tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.01 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) lb: QEMU running (pid 51) lb # Formatting '/build/vm-state-lb/tmp.YdGJx1bs2o', fmt=raw size=1073741824 lb # mke2fs 1.47.4 (6-Mar-2025) lb # Discarding device blocks: 0/262144 done lb # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Filesystem UUID: 32371818-190c-4a85-8b65-8a16c473a43c lb # Superblock backups stored on blocks: lb # 32768, 98304, 163840, 229376 lb # lb # Allocating group tables: 0/8 done lb # Writing inode tables: 0/8 done lb # Creating journal (8192 blocks): done lb # Writing superblocks and filesystem accounting information: 0/8 done lb # lb # Virtualisation disk image created. worker2 # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 52) worker2 # Formatting '/build/vm-state-worker2/tmp.ytgiCePotg', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.gYcIBsV7Te', fmt=raw size=1073741824 worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # Disk image does not exist, creating the virtualisation disk image... client # mke2fs 1.47.4 (6-Mar-2025) worker2 # Discarding device blocks: 0/262144 done worker1: QEMU running (pid 50) worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Formatting '/build/vm-state-worker1/tmp.q7zQpCjALH', fmt=raw size=1073741824 worker2 # Filesystem UUID: 46601aa9-a2da-4d97-bf78-b2a378c498a5 worker1 # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 1.06 seconds) client # Discarding device blocks: 0/262144 done worker2 # Superblock backups stored on blocks: worker1 # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service client # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # 32768, 98304, 163840, 229376 worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes client: waiting for the VM to finish booting client # Filesystem UUID: 0e25af29-1167-4ea2-9706-e005ba0f90e9 worker2 # client # Superblock backups stored on blocks: worker2 # Allocating group tables: 0/8 done client # 32768, 98304, 163840, 229376 worker2 # Writing inode tables: 0/8 done client # worker2 # Creating journal (8192 blocks): done client # Allocating group tables: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Writing inode tables: 0/8 done worker2 # client # Creating journal (8192 blocks): done worker2 # Virtualisation disk image created. client # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Filesystem UUID: 8de9c3ca-d696-4c24-be69-648c18db6bb8 client # worker1 # Superblock backups stored on blocks: client # Virtualisation disk image created. worker1 # 32768, 98304, 163840, 229376 worker1 # worker1 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker1 # Virtualisation disk image created. worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 worker2 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 lb # [ 0.000000] KASLR enabled worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] random: crng init done worker2 # [ 0.000000] DMA32 empty lb # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] Normal empty lb # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] Device empty worker2 # [ 0.000000] Movable zone start for each node lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] Early memory node ranges lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Zone ranges: worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] DMA32 empty worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] Normal empty worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] Device empty lb # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 lb # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/78c7ibbkdrnd8irpxz37agc6v4py6pgh-nixos-system-worker2-test/init regInfo=/nix/store/llaiz1z9ilncsbals7q734hl4qaxyws8-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/llaiz1z9ilncsbals7q734hl4qaxyws8-closure-info/registration", will be passed to user space. lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test/init regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] KASLR enabled worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] random: crng init done lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: client # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] Normal empty worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Device empty lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] Early memory node ranges lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] Zone ranges: worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] Early memory node ranges lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 lb # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000053] arm-pv: using stolen time PV client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.001913] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.002128] Console: colour dummy device 80x25 lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.002136] printk: legacy console [tty0] enabled client # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/15akwfj87h9x1a1mm1madsr1gizhj855-nixos-system-worker1-test/init regInfo=/nix/store/akngzink193m6x1mxil9pnzd2g7wki6n-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.002351] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.002358] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/akngzink193m6x1mxil9pnzd2g7wki6n-closure-info/registration", will be passed to user space. worker2 # [ 0.002435] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.002825] landlock: Up and running. worker2 # [ 0.002828] Yama: becoming mindful. worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.004917] LSM support for eBPF active client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test/init regInfo=/nix/store/dk026v7ww80sprshdhxp62h8c8f019ym-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000061] arm-pv: using stolen time PV worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.005236] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.002072] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.005277] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/dk026v7ww80sprshdhxp62h8c8f019ym-closure-info/registration", will be passed to user space. lb # [ 0.002291] Console: colour dummy device 80x25 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.006504] cacheinfo: Unable to detect cache hierarchy for CPU 0 lb # [ 0.002300] printk: legacy console [tty0] enabled worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.007324] rcu: Hierarchical SRCU implementation. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.007328] rcu: Max phase no-delay instances is 1000. client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.002519] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.008603] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.002526] pid_max: default: 32768 minimum: 301 client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.008702] EFI services will not be available. worker1 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.002618] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.008782] smp: Bringing up secondary CPUs ... client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.003029] landlock: Up and running. worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.008792] smp: Brought up 1 node, 1 CPU client # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] Policy zone: DMA lb # [ 0.003031] Yama: becoming mindful. lb # [ 0.004931] LSM support for eBPF active worker2 # [ 0.008795] SMP: Total of 1 processors activated. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.008801] CPU: All CPU(s) started at EL1 lb # [ 0.005197] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.008817] CPU features: detected: Branch Target Identification client # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.005235] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.008825] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.006498] cacheinfo: Unable to detect cache hierarchy for CPU 0 client # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.008828] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.007309] rcu: Hierarchical SRCU implementation. worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.007315] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.008832] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.008634] fsl-mc MSI: its@8080000 domain created client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.008752] EFI services will not be available. client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.008836] CPU features: detected: Common not Private translations lb # [ 0.008870] smp: Bringing up secondary CPUs ... worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.008839] CPU features: detected: CRC32 instructions client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages lb # [ 0.008879] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.008842] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.008882] SMP: Total of 1 processors activated. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.008888] CPU: All CPU(s) started at EL1 worker2 # [ 0.008846] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.008906] CPU features: detected: Branch Target Identification worker2 # [ 0.008849] CPU features: detected: Data independent timing control (DIT) client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.008912] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.008852] CPU features: detected: E0PD client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.008855] CPU features: detected: Enhanced Counter Virtualization client # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.008915] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.008858] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.008861] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.008918] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.008865] CPU features: detected: Fine Grained Traps client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.008922] CPU features: detected: Common not Private translations lb # [ 0.008925] CPU features: detected: CRC32 instructions worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.008868] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.008928] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.008874] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.008877] CPU features: detected: LSE atomic instructions worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.008932] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.008880] CPU features: detected: Privileged Access Never worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.008936] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.008883] CPU features: detected: PMUv3 worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.008939] CPU features: detected: E0PD client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.008886] CPU features: detected: RAS Extension Support worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.008941] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.008888] CPU features: detected: RASv1p1 Extension Support client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.008945] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.008891] CPU features: detected: Random Number Generator client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.008894] CPU features: detected: Speculation barrier (SB) lb # [ 0.008948] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.008897] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.008951] CPU features: detected: Fine Grained Traps client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.008900] CPU features: detected: TLB range maintenance instructions lb # [ 0.008956] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.008961] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.008905] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.008963] alternatives: applying system-wide alternatives lb # [ 0.008965] CPU features: detected: LSE atomic instructions worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.008968] CPU features: detected: Privileged Access Never worker2 # [ 0.012649] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.008971] CPU features: detected: PMUv3 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.008973] CPU features: detected: RAS Extension Support client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.008976] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.012902] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved) client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 lb # [ 0.008979] CPU features: detected: Random Number Generator worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.008982] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.008985] CPU features: detected: Stage-2 Force Write-Back client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.008988] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000063] arm-pv: using stolen time PV worker2 # [ 0.014009] devtmpfs: initialized lb # [ 0.008993] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.009066] alternatives: applying system-wide alternatives client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.016032] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.001208] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.013047] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.001402] Console: colour dummy device 80x25 client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.016061] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.001409] printk: legacy console [tty0] enabled client # [ 0.000046] arm-pv: using stolen time PV worker2 # [ 0.016259] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.013273] Memory: 1406520K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved) worker2 # [ 0.016263] 0 pages in range for non-PLT usage worker1 # [ 0.001631] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.001240] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.015364] devtmpfs: initialized worker2 # [ 0.016264] 508288 pages in range for PLT usage client # [ 0.001400] Console: colour dummy device 80x25 worker1 # [ 0.001638] pid_max: default: 32768 minimum: 301 worker2 # [ 0.016398] pinctrl core: initialized pinctrl subsystem lb # [ 0.017562] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.001409] printk: legacy console [tty0] enabled worker1 # [ 0.001717] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.017274] DMI not present or invalid. worker1 # [ 0.002076] landlock: Up and running. lb # [ 0.017588] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.023756] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.002083] Yama: becoming mindful. client # [ 0.001616] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.003595] LSM support for eBPF active lb # [ 0.017865] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.001623] pid_max: default: 32768 minimum: 301 worker2 # [ 0.026271] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.017871] 0 pages in range for non-PLT usage worker1 # [ 0.003795] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.001707] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.017872] 508288 pages in range for PLT usage worker2 # [ 0.026490] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.001962] landlock: Up and running. worker1 # [ 0.003829] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.018009] pinctrl core: initialized pinctrl subsystem client # [ 0.001964] Yama: becoming mindful. worker2 # [ 0.026715] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.003374] LSM support for eBPF active lb # [ 0.018855] DMI not present or invalid. worker1 # [ 0.005435] cacheinfo: Unable to detect cache hierarchy for CPU 0 worker2 # [ 0.026753] audit: initializing netlink subsys (disabled) lb # [ 0.025199] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.006724] rcu: Hierarchical SRCU implementation. client # [ 0.003646] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.027477] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.006731] rcu: Max phase no-delay instances is 1000. lb # [ 0.027438] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.003687] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.008151] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.027479] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.027636] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.008257] EFI services will not be available. client # [ 0.004979] cacheinfo: Unable to detect cache hierarchy for CPU 0 worker2 # [ 0.027483] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.008347] smp: Bringing up secondary CPUs ... client # [ 0.005790] rcu: Hierarchical SRCU implementation. lb # [ 0.027856] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.027489] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.008356] smp: Brought up 1 node, 1 CPU client # [ 0.005795] rcu: Max phase no-delay instances is 1000. lb # [ 0.027890] audit: initializing netlink subsys (disabled) worker1 # [ 0.008359] SMP: Total of 1 processors activated. worker2 # [ 0.027491] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.008365] CPU: All CPU(s) started at EL1 lb # [ 0.028578] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.007070] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.027518] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.008383] CPU features: detected: Branch Target Identification lb # [ 0.028581] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.027530] cpuidle: using governor ladder client # [ 0.007175] EFI services will not be available. worker1 # [ 0.008392] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.027535] cpuidle: using governor menu client # [ 0.007264] smp: Bringing up secondary CPUs ... lb # [ 0.028585] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.007273] smp: Brought up 1 node, 1 CPU lb # [ 0.028590] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.027743] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.008395] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.007276] SMP: Total of 1 processors activated. worker2 # [ 0.027760] ASID allocator initialised with 65536 entries lb # [ 0.028593] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.007282] CPU: All CPU(s) started at EL1 worker1 # [ 0.008399] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.029326] Serial: AMBA PL011 UART driver lb # [ 0.028618] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 client # [ 0.007295] CPU features: detected: Branch Target Identification worker1 # [ 0.008402] CPU features: detected: Common not Private translations worker2 # [ 0.035077] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.028629] cpuidle: using governor ladder client # [ 0.007300] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.008405] CPU features: detected: CRC32 instructions lb # [ 0.028635] cpuidle: using governor menu worker2 # [ 0.035263] printk: console [ttyAMA0] enabled client # [ 0.007303] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.028839] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.008408] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.028857] ASID allocator initialised with 65536 entries worker1 # [ 0.008412] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.007307] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.030188] Serial: AMBA PL011 UART driver worker1 # [ 0.008415] CPU features: detected: Data independent timing control (DIT) client # [ 0.007311] CPU features: detected: Common not Private translations worker1 # [ 0.008418] CPU features: detected: E0PD client # [ 0.007314] CPU features: detected: CRC32 instructions worker1 # [ 0.008421] CPU features: detected: Enhanced Counter Virtualization client # [ 0.007316] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.008424] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.036906] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.007320] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.037103] printk: console [ttyAMA0] enabled worker1 # [ 0.008427] CPU features: detected: Enhanced Virtualization Traps client # [ 0.007324] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.008430] CPU features: detected: Fine Grained Traps client # [ 0.007327] CPU features: detected: E0PD client # [ 0.007329] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.008433] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.008438] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.007332] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.008441] CPU features: detected: LSE atomic instructions client # [ 0.007336] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.008444] CPU features: detected: Privileged Access Never client # [ 0.007339] CPU features: detected: Fine Grained Traps worker1 # [ 0.008447] CPU features: detected: PMUv3 worker2 # [ 0.166103] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.007343] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.008449] CPU features: detected: RAS Extension Support worker2 # [ 0.166127] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.007348] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.008452] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.008455] CPU features: detected: Random Number Generator client # [ 0.007351] CPU features: detected: LSE atomic instructions worker2 # [ 0.166132] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.008457] CPU features: detected: Speculation barrier (SB) client # [ 0.007354] CPU features: detected: Privileged Access Never worker2 # [ 0.166137] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.007357] CPU features: detected: PMUv3 worker1 # [ 0.008460] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.166141] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.007359] CPU features: detected: RAS Extension Support worker1 # [ 0.008463] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.166145] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.007362] CPU features: detected: RASv1p1 Extension Support lb # [ 0.167806] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.008470] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.007364] CPU features: detected: Random Number Generator worker2 # [ 0.166149] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.008527] alternatives: applying system-wide alternatives lb # [ 0.167838] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.007367] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.166153] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.012286] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.167844] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.007369] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.167849] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.007373] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.012646] Memory: 1406508K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved) lb # [ 0.167853] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.007377] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.173937] fbcon: Taking over console worker1 # [ 0.013228] devtmpfs: initialized worker2 # [ 0.173957] ACPI: Interpreter disabled. lb # [ 0.167858] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.007434] alternatives: applying system-wide alternatives worker1 # [ 0.017958] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.167862] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.017987] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.167867] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.011458] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.018198] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.011731] Memory: 1406576K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved) worker1 # [ 0.018209] 0 pages in range for non-PLT usage client # [ 0.012368] devtmpfs: initialized worker1 # [ 0.018210] 508288 pages in range for PLT usage lb # [ 0.175731] fbcon: Taking over console client # [ 0.014333] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.018347] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.019218] DMI not present or invalid. lb # [ 0.175753] ACPI: Interpreter disabled. client # [ 0.014363] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.023865] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.014581] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.027222] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.014592] 0 pages in range for non-PLT usage client # [ 0.014593] 508288 pages in range for PLT usage worker1 # [ 0.027427] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.014698] pinctrl core: initialized pinctrl subsystem client # [ 0.015574] DMI not present or invalid. worker1 # [ 0.027646] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.182571] iommu: Default domain type: Translated client # [ 0.020611] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.027682] audit: initializing netlink subsys (disabled) worker2 # [ 0.182602] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.025659] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.183149] SCSI subsystem initialized worker1 # [ 0.028403] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.028440] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.025980] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.185265] usbcore: registered new interface driver usbfs worker1 # [ 0.028442] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.026238] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.185301] usbcore: registered new interface driver hub worker1 # [ 0.028448] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.026278] audit: initializing netlink subsys (disabled) worker2 # [ 0.185327] usbcore: registered new device driver usb worker1 # [ 0.028451] thermal_sys: Registered thermal governor 'user_space' client # [ 0.027101] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.185086] iommu: Default domain type: Translated worker2 # [ 0.185642] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.028454] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.027104] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.185103] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.028480] cpuidle: using governor ladder worker2 # [ 0.185648] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.027109] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.028488] cpuidle: using governor menu worker2 # [ 0.185659] PTP clock support registered lb # [ 0.186998] SCSI subsystem initialized client # [ 0.027114] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.185726] EDAC MC: Ver: 3.0.0 worker1 # [ 0.028727] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.027117] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.028750] ASID allocator initialised with 65536 entries worker2 # [ 0.190556] scmi_core: SCMI protocol bus registered lb # [ 0.187957] usbcore: registered new interface driver usbfs worker1 # [ 0.030228] Serial: AMBA PL011 UART driver client # [ 0.027157] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1 lb # [ 0.188002] usbcore: registered new interface driver hub worker2 # [ 0.191608] FPGA manager framework client # [ 0.027168] cpuidle: using governor ladder worker1 # [ 0.037277] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.188031] usbcore: registered new device driver usb worker2 # [ 0.192607] vgaarb: loaded client # [ 0.027174] cpuidle: using governor menu worker1 # [ 0.037464] printk: console [ttyAMA0] enabled lb # [ 0.190277] pps_core: LinuxPPS API ver. 1 registered client # [ 0.027393] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.193303] clocksource: Switched to clocksource arch_sys_counter client # [ 0.027416] ASID allocator initialised with 65536 entries lb # [ 0.190288] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.028761] Serial: AMBA PL011 UART driver lb # [ 0.190299] PTP clock support registered lb # [ 0.190357] EDAC MC: Ver: 3.0.0 client # [ 0.034571] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.034752] printk: console [ttyAMA0] enabled lb # [ 0.193207] scmi_core: SCMI protocol bus registered lb # [ 0.194274] FPGA manager framework worker2 # [ 0.198127] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.198192] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.197969] vgaarb: loaded worker2 # [ 0.200010] netfs: FS-Cache loaded worker2 # [ 0.200171] pnp: PnP ACPI: disabled worker1 # [ 0.160747] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.198702] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.199374] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.160774] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.199412] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.160780] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.160784] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.202027] netfs: FS-Cache loaded client # [ 0.157593] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.202166] pnp: PnP ACPI: disabled worker2 # [ 0.204305] NET: Registered PF_INET protocol family worker1 # [ 0.160789] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.157626] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.204569] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.157631] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.160793] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.160797] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.157636] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.160802] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.157640] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.157645] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.157649] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.168643] fbcon: Taking over console client # [ 0.157654] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.168664] ACPI: Interpreter disabled. client # [ 0.165397] fbcon: Taking over console client # [ 0.165417] ACPI: Interpreter disabled. lb # [ 0.209608] NET: Registered PF_INET protocol family lb # [ 0.209900] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.177234] iommu: Default domain type: Translated worker1 # [ 0.177250] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.177678] SCSI subsystem initialized client # [ 0.173898] iommu: Default domain type: Translated client # [ 0.173913] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.175759] SCSI subsystem initialized worker1 # [ 0.179820] usbcore: registered new interface driver usbfs client # [ 0.176541] usbcore: registered new interface driver usbfs worker1 # [ 0.179857] usbcore: registered new interface driver hub client # [ 0.176583] usbcore: registered new interface driver hub worker1 # [ 0.179881] usbcore: registered new device driver usb client # [ 0.176602] usbcore: registered new device driver usb worker1 # [ 0.182721] pps_core: LinuxPPS API ver. 1 registered client # [ 0.176927] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.182732] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.176934] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.182744] PTP clock support registered client # [ 0.176952] PTP clock support registered worker1 # [ 0.182824] EDAC MC: Ver: 3.0.0 client # [ 0.177014] EDAC MC: Ver: 3.0.0 worker1 # [ 0.187765] scmi_core: SCMI protocol bus registered client # [ 0.181902] scmi_core: SCMI protocol bus registered worker1 # [ 0.188859] FPGA manager framework worker1 # [ 0.190166] vgaarb: loaded client # [ 0.182990] FPGA manager framework client # [ 0.187040] vgaarb: loaded worker1 # [ 0.192022] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.192686] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.192722] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.187786] clocksource: Switched to clocksource arch_sys_counter client # [ 0.188492] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.195234] netfs: FS-Cache loaded worker1 # [ 0.195373] pnp: PnP ACPI: disabled client # [ 0.188527] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.190944] netfs: FS-Cache loaded client # [ 0.191084] pnp: PnP ACPI: disabled worker2 # [ 0.240716] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.240786] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.240826] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.202057] NET: Registered PF_INET protocol family worker2 # [ 0.240928] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.202333] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.241072] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.241200] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.198191] NET: Registered PF_INET protocol family worker2 # [ 0.241245] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.198500] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.241285] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.249181] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.249231] NET: Registered PF_XDP protocol family worker2 # [ 0.249253] PCI: CLS 0 bytes, default 64 worker2 # [ 0.249428] Trying to unpack rootfs image as initramfs... worker2 # [ 0.260118] kvm [1]: HYP mode not available lb # [ 0.249356] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.249434] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.249472] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.249571] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.249718] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.249845] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.249894] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.249934] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.250034] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.250066] NET: Registered PF_XDP protocol family lb # [ 0.250089] PCI: CLS 0 bytes, default 64 lb # [ 0.250404] Trying to unpack rootfs image as initramfs... lb # [ 0.269196] kvm [1]: HYP mode not available client # [ 0.232254] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.232324] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.237687] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.232361] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.237765] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.232465] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.237805] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.232611] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.237938] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.232726] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.238083] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.232770] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.238202] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.232810] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.238246] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.232912] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.238286] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.232958] NET: Registered PF_XDP protocol family worker1 # [ 0.238426] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.232981] PCI: CLS 0 bytes, default 64 worker1 # [ 0.238452] NET: Registered PF_XDP protocol family client # [ 0.233277] Trying to unpack rootfs image as initramfs... worker1 # [ 0.238475] PCI: CLS 0 bytes, default 64 worker1 # [ 0.238787] Trying to unpack rootfs image as initramfs... client # [ 0.250044] kvm [1]: HYP mode not available worker1 # [ 0.257275] kvm [1]: HYP mode not available worker2 # [ 0.466138] Initialise system trusted keyrings worker2 # [ 0.473499] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.475132] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.475925] 9p: Installing v9fs 9p2000 file system support lb # [ 0.480076] Initialise system trusted keyrings lb # [ 0.480953] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.482353] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.483337] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.497057] Key type asymmetric registered worker2 # [ 0.497096] Asymmetric key parser 'x509' registered worker2 # [ 0.497186] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.459671] Initialise system trusted keyrings worker1 # [ 0.460500] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.461940] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.462751] 9p: Installing v9fs 9p2000 file system support client # [ 0.457529] Initialise system trusted keyrings worker2 # [ 0.505402] io scheduler mq-deadline registered client # [ 0.458460] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.505433] io scheduler kyber registered lb # [ 0.503459] Key type asymmetric registered lb # [ 0.503499] Asymmetric key parser 'x509' registered client # [ 0.459962] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.503588] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.460857] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.511068] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.510789] io scheduler mq-deadline registered lb # [ 0.510820] io scheduler kyber registered worker2 # [ 0.517366] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.518237] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.518266] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.518280] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.518288] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.522935] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.518325] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.518359] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.518463] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.524598] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.518478] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.518485] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.525106] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.518490] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.525129] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.518496] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.525143] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.491841] Key type asymmetric registered worker2 # [ 0.518589] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.491879] Asymmetric key parser 'x509' registered lb # [ 0.525151] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.519091] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.491979] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.525179] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.519286] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.494196] io scheduler mq-deadline registered worker2 # [ 0.519303] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.494211] io scheduler kyber registered lb # [ 0.525210] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.489207] Key type asymmetric registered worker2 # [ 0.519333] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.525306] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.489245] Asymmetric key parser 'x509' registered worker2 # [ 0.519350] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.525319] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.489348] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.519817] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.525326] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.491506] io scheduler mq-deadline registered worker2 # [ 0.520007] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.491520] io scheduler kyber registered lb # [ 0.525332] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.520024] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.525337] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.520055] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.525416] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.520514] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint lb # [ 0.525890] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.520699] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.526083] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.520716] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.526100] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.520757] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.526131] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.521300] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint lb # [ 0.526149] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.526621] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.545802] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.545824] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.508600] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.545859] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.546374] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.546561] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.546577] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.505908] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.546610] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.507364] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.547093] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.510099] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.547280] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.547297] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.510564] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.547327] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.510583] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.547844] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.508659] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.510602] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.548030] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] client # [ 0.508710] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.548053] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.510611] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.548091] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.508738] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.510633] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.550905] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.548116] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.508763] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.550947] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.510665] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.550978] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.508833] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.548607] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.510762] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.510772] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.551545] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.508866] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.551743] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.510779] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.548805] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.508977] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.551764] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.510785] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.548836] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.508990] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.551796] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.510790] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.508996] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.549292] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.552294] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker1 # [ 0.510933] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.509002] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.552486] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.509007] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.511419] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.552503] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.511619] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.552537] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.509073] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.511637] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.553024] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.509543] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.511668] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.553212] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] client # [ 0.509738] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.511685] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.553229] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.509756] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.512159] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.553259] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.509787] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.512351] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.509805] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.553732] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.512368] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.553922] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] client # [ 0.510282] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.512399] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.553939] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.510476] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.512870] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.510493] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.553969] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.513068] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] client # [ 0.510525] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.554482] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.513085] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.554675] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] client # [ 0.510992] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.573609] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.513117] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.554693] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.511183] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.573664] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.513590] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.511200] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.554739] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.513781] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.574170] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.554756] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.511232] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.513798] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.574364] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.555232] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.511710] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker1 # [ 0.513831] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.555427] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.574633] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.514343] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint lb # [ 0.555458] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.574940] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.514536] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.574977] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.555989] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.514554] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.575009] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.556211] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.514591] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.556242] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.575494] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.575684] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.556646] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.575701] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.556837] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.575738] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.557097] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.576353] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.557409] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.576366] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.557429] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.557460] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.576371] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.557941] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.576419] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.558131] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.576468] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.558148] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.547232] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.576518] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.558179] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.547453] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.576567] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.547472] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.547503] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.576616] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.540020] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.540066] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.547990] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.576667] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.548182] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] client # [ 0.540102] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.576718] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.548198] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.540691] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.576768] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.548229] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.540887] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.548246] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.576818] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.540905] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.548754] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.576903] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.540937] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.548951] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.576958] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.541414] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.548982] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.541602] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.576987] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.541618] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.577013] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.549450] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.541648] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.549657] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.577043] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.549689] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.577086] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned client # [ 0.542118] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.542308] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.577114] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.550128] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.542324] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.577146] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.550325] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.542356] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.577178] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.550587] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.599019] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.542373] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.577208] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.550840] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.599056] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.542890] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.550858] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.577235] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.599062] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.543082] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.550888] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.577263] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.543113] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.599120] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.577291] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.543583] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.599178] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.543783] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.599229] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.543837] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.599279] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.544275] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.599330] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.544470] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.599382] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.544726] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.545094] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.599433] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.545114] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.599484] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.571485] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.545145] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.613414] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.571702] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.599534] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.545626] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.613475] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.571719] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.599671] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.545815] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.613508] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned worker1 # [ 0.571751] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.545831] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.599723] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.613543] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.572392] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.599747] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.545862] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.613579] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.572405] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.546504] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.613616] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned client # [ 0.546516] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.572411] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.599770] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.613647] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned worker2 # [ 0.613672] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned worker1 # [ 0.572461] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.599794] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.546522] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.613697] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned lb # [ 0.599821] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.572512] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.546569] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.613736] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.599844] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.572563] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.613747] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.546619] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.599867] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.613751] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.546669] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.572613] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.599891] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.615027] pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.546718] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.599913] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.572663] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.599937] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned client # [ 0.546767] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.572714] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.599961] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.546817] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.572764] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.599985] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.546866] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.572814] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.600008] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.546916] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.572864] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.600031] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.600054] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned client # [ 0.546966] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.573018] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.600076] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.573068] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.547065] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.600099] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.573092] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.547114] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.600122] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned worker1 # [ 0.573115] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.547138] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.600145] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned worker1 # [ 0.573139] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.600168] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned client # [ 0.547168] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.573164] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.600190] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned client # [ 0.547192] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.600242] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.573188] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.547220] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.600253] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.573212] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned client # [ 0.547244] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.600258] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.573236] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.547267] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.601478] pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.573259] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.547290] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.573282] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned client # [ 0.547314] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.573307] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.547340] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.573331] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.547365] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.573354] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.547390] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.573376] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.547413] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.573400] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned client # [ 0.547436] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.573423] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.547460] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned worker1 # [ 0.573446] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.547483] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.573469] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned client # [ 0.547506] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.573492] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned client # [ 0.547529] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned worker1 # [ 0.573515] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned client # [ 0.547552] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned worker1 # [ 0.573538] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned client # [ 0.547577] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned worker1 # [ 0.573569] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.547603] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned worker1 # [ 0.573579] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.547633] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.573584] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.547643] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.574829] pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.547648] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.654773] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.657279] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.619128] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 43263 usecs client # [ 0.613073] pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.658316] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.668163] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.671591] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.668513] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.641549] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.683021] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) lb # [ 0.680325] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.682488] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.645090] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.652990] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.647602] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.693556] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) worker2 # [ 0.694635] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) worker2 # [ 0.696895] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) client # [ 0.658697] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.699985] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) lb # [ 0.702307] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) worker2 # [ 0.708964] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.661837] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.663258] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.667881] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.670565] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) worker2 # [ 0.713027] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.712416] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.714635] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.718912] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.672148] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) client # [ 0.674473] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) worker1 # [ 0.681638] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) client # [ 0.678529] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) lb # [ 0.725723] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.688935] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) worker2 # [ 0.735275] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.732501] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.694386] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.699229] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.701713] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.749962] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.703756] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.706425] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.753163] msm_serial: driver initialized lb # [ 0.752021] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.753993] SuperH (H)SCI(F) driver initialized worker2 # [ 0.754061] STM32 USART driver initialized lb # [ 0.755447] msm_serial: driver initialized lb # [ 0.755616] SuperH (H)SCI(F) driver initialized lb # [ 0.755679] STM32 USART driver initialized client # [ 0.714080] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.717120] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.728103] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.731783] msm_serial: driver initialized worker1 # [ 0.731940] SuperH (H)SCI(F) driver initialized worker1 # [ 0.732001] STM32 USART driver initialized client # [ 0.736044] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.794912] loop: module loaded worker2 # [ 0.795246] virtio_blk virtio5: 1/0/0 default/read/poll queues worker2 # [ 0.796169] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.749723] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.752468] msm_serial: driver initialized client # [ 0.752638] SuperH (H)SCI(F) driver initialized client # [ 0.752698] STM32 USART driver initialized worker2 # [ 0.802160] megasas: 07.734.00.00-rc1 worker2 # [ 0.802990] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.804431] loop: module loaded lb # [ 0.804698] virtio_blk virtio5: 1/0/0 default/read/poll queues worker1 # [ 0.768336] loop: module loaded lb # [ 0.805950] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.768583] virtio_blk virtio5: 1/0/0 default/read/poll queues worker1 # [ 0.769393] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.814767] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.814827] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.816985] Using buffer write method worker2 # [ 0.817113] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.819684] megasas: 07.734.00.00-rc1 lb # [ 0.820703] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.823215] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.823303] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.830629] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.830698] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.783730] megasas: 07.734.00.00-rc1 worker2 # [ 0.832862] Using buffer write method worker2 # [ 0.832911] Concatenating MTD devices: worker2 # [ 0.832915] (0): "0.flash" worker2 # [ 0.832919] (1): "0.flash" worker2 # [ 0.832923] into device "0.flash" worker1 # [ 0.784565] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.786669] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.786752] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.788538] Using buffer write method worker1 # [ 0.788626] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.790331] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.790362] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.834763] Using buffer write method lb # [ 0.834865] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.792784] loop: module loaded client # [ 0.793015] virtio_blk virtio5: 1/0/0 default/read/poll queues lb # [ 0.838045] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.838099] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.793857] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.847857] Using buffer write method lb # [ 0.847919] Concatenating MTD devices: lb # [ 0.847923] (0): "0.flash" lb # [ 0.847928] (1): "0.flash" lb # [ 0.847931] into device "0.flash" client # [ 0.800601] megasas: 07.734.00.00-rc1 client # [ 0.801458] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.813157] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.813214] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.824532] Using buffer write method worker1 # [ 0.827644] Using buffer write method client # [ 0.824657] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.827691] Concatenating MTD devices: worker1 # [ 0.827695] (0): "0.flash" worker1 # [ 0.827700] (1): "0.flash" client # [ 0.827237] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.827704] into device "0.flash" client # [ 0.827287] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.832438] Using buffer write method client # [ 0.832478] Concatenating MTD devices: client # [ 0.832482] (0): "0.flash" client # [ 0.832486] (1): "0.flash" client # [ 0.832490] into device "0.flash" worker2 # [ 1.214036] Freeing initrd memory: 26148K client # [ 1.175993] Freeing initrd memory: 26160K worker2 # [ 1.226488] tun: Universal TUN/TAP device driver, 1.6 client # [ 1.186113] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 1.231871] thunder_xcv, ver 1.0 worker2 # [ 1.231952] thunder_bgx, ver 1.0 worker2 # [ 1.232027] nicpf, ver 1.0 worker2 # [ 1.236033] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 1.236076] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 1.195023] Freeing initrd memory: 26148K client # [ 1.190626] thunder_xcv, ver 1.0 worker2 # [ 1.236128] e1000e: Intel(R) PRO/1000 Network Driver client # [ 1.190685] thunder_bgx, ver 1.0 client # [ 1.190710] nicpf, ver 1.0 worker2 # [ 1.236140] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 1.191708] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 1.236179] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 1.191722] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 1.236187] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 1.191749] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 1.236213] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 1.191757] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 1.236221] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 1.236388] sky2: driver version 1.30 client # [ 1.195740] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 1.195753] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 1.202619] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 1.244339] usbcore: registered new interface driver usb-storage worker2 # [ 1.244456] usbcore: registered new interface driver usbserial_generic worker2 # [ 1.244472] usbserial: USB Serial support registered for generic worker2 # [ 1.245150] hv_vmbus: registering driver hyperv_keyboard client # [ 1.197346] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 1.197362] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 1.197545] sky2: driver version 1.30 worker2 # [ 1.249127] ehci-pci 0000:00:0a.0: EHCI Host Controller client # [ 1.199393] usbcore: registered new interface driver usb-storage worker2 # [ 1.249190] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 client # [ 1.199523] usbcore: registered new interface driver usbserial_generic worker2 # [ 1.249476] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 client # [ 1.199545] usbserial: USB Serial support registered for generic worker1 # [ 1.208846] thunder_xcv, ver 1.0 worker1 # [ 1.208926] thunder_bgx, ver 1.0 client # [ 1.200598] ehci-pci 0000:00:0a.0: EHCI Host Controller worker1 # [ 1.209012] nicpf, ver 1.0 worker1 # [ 1.210457] e1000: Intel(R) PRO/1000 Network Driver client # [ 1.200640] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 worker2 # [ 1.253025] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 1.210473] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 1.200875] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 worker1 # [ 1.210506] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 1.253067] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:44:58 UTC (1790073898) worker1 # [ 1.210514] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 1.250520] Freeing initrd memory: 26140K worker1 # [ 1.210553] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 1.255287] i2c_dev: i2c /dev entries driver client # [ 1.208320] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.210559] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 1.210586] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 1.210594] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 1.210753] sky2: driver version 1.30 client # [ 1.210220] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 1.210259] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:44:58 UTC (1790073898) client # [ 1.210612] i2c_dev: i2c /dev entries driver worker2 # [ 1.259247] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 1.259265] sdhci: Copyright(c) Pierre Ossman client # [ 1.211845] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 client # [ 1.212278] hub 1-0:1.0: USB hub found worker2 # [ 1.259548] Synopsys Designware Multimedia Card Interface Driver client # [ 1.212303] hub 1-0:1.0: 6 ports detected worker2 # [ 1.259967] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 1.264234] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 lb # [ 1.262411] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 1.220066] usbcore: registered new interface driver usb-storage worker2 # [ 1.264690] hub 1-0:1.0: USB hub found worker1 # [ 1.220885] usbcore: registered new interface driver usbserial_generic worker2 # [ 1.264717] hub 1-0:1.0: 6 ports detected worker1 # [ 1.220907] usbserial: USB Serial support registered for generic worker1 # [ 1.221573] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.223081] ehci-pci 0000:00:0a.0: EHCI Host Controller worker2 # [ 1.267653] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 1.223117] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 worker2 # [ 1.267997] usbcore: registered new interface driver usbhid worker1 # [ 1.223665] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 worker2 # [ 1.268006] usbhid: USB HID core driver worker2 # [ 1.271317] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 1.231454] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 1.273109] drop_monitor: Initializing network drop monitor service lb # [ 1.267223] thunder_xcv, ver 1.0 worker1 # [ 1.231497] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:44:58 UTC (1790073898) client # [ 1.224244] sdhci: Secure Digital Host Controller Interface driver lb # [ 1.267283] thunder_bgx, ver 1.0 lb # [ 1.267308] nicpf, ver 1.0 worker1 # [ 1.232006] i2c_dev: i2c /dev entries driver client # [ 1.224272] sdhci: Copyright(c) Pierre Ossman worker2 # [ 1.275571] NET: Registered PF_INET6 protocol family lb # [ 1.267949] e1000: Intel(R) PRO/1000 Network Driver client # [ 1.224682] Synopsys Designware Multimedia Card Interface Driver lb # [ 1.267957] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 1.225444] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 1.276945] Segment Routing with IPv6 lb # [ 1.267994] e1000e: Intel(R) PRO/1000 Network Driver client # [ 1.227576] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 1.276969] In-situ OAM (IOAM) with IPv6 lb # [ 1.268002] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 1.277012] NET: Registered PF_PACKET protocol family worker1 # [ 1.235134] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 lb # [ 1.268038] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 1.236754] hub 1-0:1.0: USB hub found client # [ 1.231235] usbcore: registered new interface driver usbhid worker1 # [ 1.236780] hub 1-0:1.0: 6 ports detected client # [ 1.231254] usbhid: USB HID core driver lb # [ 1.268046] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 1.268072] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 1.268081] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 1.278807] 9pnet: Installing 9P2000 support lb # [ 1.268274] sky2: driver version 1.30 lb # [ 1.270695] usbcore: registered new interface driver usb-storage lb # [ 1.271693] ehci-pci 0000:00:0a.0: EHCI Host Controller client # [ 1.234604] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 1.271775] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 worker1 # [ 1.241450] sdhci: Secure Digital Host Controller Interface driver lb # [ 1.272022] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 worker1 # [ 1.241466] sdhci: Copyright(c) Pierre Ossman client # [ 1.237813] drop_monitor: Initializing network drop monitor service worker1 # [ 1.241971] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 1.282207] Key type dns_resolver registered client # [ 1.238027] NET: Registered PF_INET6 protocol family worker1 # [ 1.242397] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 1.282630] usbcore: registered new interface driver usbserial_generic lb # [ 1.282652] usbserial: USB Serial support registered for generic lb # [ 1.282808] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 client # [ 1.241452] Segment Routing with IPv6 lb # [ 1.283239] hub 1-0:1.0: USB hub found client # [ 1.241487] In-situ OAM (IOAM) with IPv6 lb # [ 1.283265] hub 1-0:1.0: 6 ports detected client # [ 1.241560] NET: Registered PF_PACKET protocol family client # [ 1.243311] 9pnet: Installing 9P2000 support worker1 # [ 1.248901] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 1.249231] usbcore: registered new interface driver usbhid lb # [ 1.288087] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.249238] usbhid: USB HID core driver worker2 # [ 1.289866] registered taskstats version 1 worker2 # [ 1.290087] Loading compiled-in X.509 certificates client # [ 1.246477] Key type dns_resolver registered lb # [ 1.291183] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 1.291218] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:44:58 UTC (1790073898) worker1 # [ 1.252938] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 1.291585] i2c_dev: i2c /dev entries driver worker1 # [ 1.254722] drop_monitor: Initializing network drop monitor service worker1 # [ 1.257002] NET: Registered PF_INET6 protocol family worker1 # [ 1.258362] Segment Routing with IPv6 worker1 # [ 1.258385] In-situ OAM (IOAM) with IPv6 worker1 # [ 1.258418] NET: Registered PF_PACKET protocol family client # [ 1.253976] registered taskstats version 1 client # [ 1.254205] Loading compiled-in X.509 certificates worker1 # [ 1.260367] 9pnet: Installing 9P2000 support worker2 # [ 1.303081] Demotion targets for Node 0: null worker1 # [ 1.262675] Key type dns_resolver registered worker2 # [ 1.303589] Key type .fscrypt registered worker2 # [ 1.303601] Key type fscrypt-provisioning registered lb # [ 1.298973] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 1.303727] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.298988] sdhci: Copyright(c) Pierre Ossman worker2 # [ 1.303752] ima: Allocated hash algorithm: sha1 lb # [ 1.299319] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 1.303791] ima: No architecture policies found worker2 # [ 1.308360] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 1.270510] registered taskstats version 1 client # [ 1.264660] Demotion targets for Node 0: null worker1 # [ 1.270705] Loading compiled-in X.509 certificates client # [ 1.264830] Key type .fscrypt registered client # [ 1.264838] Key type fscrypt-provisioning registered lb # [ 1.308714] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 1.264966] ima: No TPM chip found, activating TPM-bypass! client # [ 1.264996] ima: Allocated hash algorithm: sha1 client # [ 1.265029] ima: No architecture policies found lb # [ 1.311734] hid: raw HID events driver (C) Jiri Kosina client # [ 1.269538] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 1.312126] usbcore: registered new interface driver usbhid lb # [ 1.312136] usbhid: USB HID core driver lb # [ 1.315462] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 1.280009] Demotion targets for Node 0: null lb # [ 1.317405] drop_monitor: Initializing network drop monitor service worker1 # [ 1.280166] Key type .fscrypt registered lb # [ 1.317652] NET: Registered PF_INET6 protocol family worker1 # [ 1.280173] Key type fscrypt-provisioning registered worker1 # [ 1.280289] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.321210] Segment Routing with IPv6 worker1 # [ 1.280315] ima: Allocated hash algorithm: sha1 lb # [ 1.321237] In-situ OAM (IOAM) with IPv6 worker1 # [ 1.280345] ima: No architecture policies found lb # [ 1.321279] NET: Registered PF_PACKET protocol family worker1 # [ 1.285122] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 1.323057] 9pnet: Installing 9P2000 support lb # [ 1.325614] Key type dns_resolver registered worker2 # [ 1.339280] clk: Disabling unused clocks lb # [ 1.336777] registered taskstats version 1 worker2 # [ 1.339334] PM: genpd: Disabling unused power domains lb # [ 1.336992] Loading compiled-in X.509 certificates client # [ 1.293294] clk: Disabling unused clocks client # [ 1.293347] PM: genpd: Disabling unused power domains client # [ 1.298350] Freeing unused kernel memory: 4736K client # [ 1.299082] Run /init as init process worker2 # [ 1.347105] Freeing unused kernel memory: 4736K worker2 # [ 1.347537] Run /init as init process lb # [ 1.347380] Demotion targets for Node 0: null lb # [ 1.347560] Key type .fscrypt registered lb # [ 1.347567] Key type fscrypt-provisioning registered worker1 # [ 1.310887] clk: Disabling unused clocks lb # [ 1.347693] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.347715] ima: Allocated hash algorithm: sha1 lb # [ 1.347748] ima: No architecture policies found worker1 # [ 1.311556] PM: genpd: Disabling unused power domains lb # [ 1.352169] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 1.316796] Freeing unused kernel memory: 4736K worker1 # [ 1.317544] Run /init as init process client # [ 1.318830] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.368623] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 1.343864] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.382903] clk: Disabling unused clocks lb # [ 1.382940] PM: genpd: Disabling unused power domains lb # [ 1.388596] Freeing unused kernel memory: 4736K lb # [ 1.389010] Run /init as init process lb # [ 1.416812] systemd[1]: Successfully made /usr/ read-only. client # [ 1.459931] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.509411] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.483088] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.530772] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.618280] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.665976] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.645735] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.685420] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.654260] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.654339] systemd[1]: Detected virtualization qemu. client # [ 1.654452] systemd[1]: Detected architecture arm64. worker2 # [ 1.708296] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.654486] systemd[1]: Running in initrd. client # [ 1.655753] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.721190] systemd[1]: Detected virtualization qemu. client # [ 1.678041] systemd[1]: Hostname set to . worker2 # [ 1.721360] systemd[1]: Detected architecture arm64. worker2 # [ 1.721389] systemd[1]: Running in initrd. worker2 # [ 1.723082] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.679558] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.723546] systemd[1]: Hostname set to . worker1 # [ 1.692439] systemd[1]: Detected virtualization qemu. worker1 # [ 1.694629] systemd[1]: Detected architecture arm64. worker1 # [ 1.696971] systemd[1]: Running in initrd. worker1 # [ 1.701766] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.706400] systemd[1]: Hostname set to . client # [ 1.704489] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 worker2 # [ 1.753875] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 lb # [ 1.752344] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.765344] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 lb # [ 1.770692] systemd[1]: Detected virtualization qemu. lb # [ 1.772893] systemd[1]: Detected architecture arm64. worker1 # [ 1.731517] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 lb # [ 1.774926] systemd[1]: Running in initrd. lb # [ 1.778213] systemd[1]: Initializing machine ID from random generator. lb # [ 1.787751] systemd[1]: Hostname set to . client # [ 1.843882] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.897417] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.863034] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.914858] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.998904] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 2.001018] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 client # [ 2.016702] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 2.026097] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 2.072505] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 2.032388] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 lb # [ 2.075341] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 2.082331] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 lb # [ 2.081304] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 worker1 # [ 2.080715] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 2.148423] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 2.113308] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 2.158598] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 2.122809] systemd[1]: Created slice Slice /system/modprobe. client # [ 2.123967] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 2.125011] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 2.125897] systemd[1]: Reached target Path Units. client # [ 2.126537] systemd[1]: Reached target Slice Units. client # [ 2.127183] systemd[1]: Reached target Swaps. client # [ 2.127778] systemd[1]: Reached target Timer Units. client # [ 2.128693] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 2.129778] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 2.130697] systemd[1]: Listening on Journal Sockets. client # [ 2.131490] systemd[1]: Listening on udev Control Socket. client # [ 2.132390] systemd[1]: Listening on udev Kernel Socket. client # [ 2.133094] systemd[1]: Reached target Socket Units. client # [ 2.135335] systemd[1]: Starting Create List of Static Device Nodes... client # [ 2.144403] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 2.145383] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 2.156115] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 2.180793] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 2.176124] systemd[1]: Starting Journal Service... worker1 # [ 2.189200] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 2.190312] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 2.191551] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 2.191599] systemd[1]: Reached target Path Units. worker1 # [ 2.191619] systemd[1]: Reached target Slice Units. worker1 # [ 2.191637] systemd[1]: Reached target Swaps. worker1 # [ 2.191656] systemd[1]: Reached target Timer Units. worker1 # [ 2.191860] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 2.191998] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 2.192121] systemd[1]: Listening on Journal Sockets. worker1 # [ 2.192235] systemd[1]: Listening on udev Control Socket. worker1 # [ 2.192339] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 2.192358] systemd[1]: Reached target Socket Units. worker1 # [ 2.201446] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 2.211224] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 2.211359] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 2.211763] systemd[1]: Starting Load Kernel Modules... client # [ 2.212048] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 2.215754] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 2.224526] systemd[1]: Mounting Kernel Configuration File System... lb # [ 2.263791] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 2.273128] systemd[1]: Created slice Slice /system/modprobe. lb # [ 2.274204] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 2.275363] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 2.276348] systemd[1]: Reached target Path Units. lb # [ 2.277064] systemd[1]: Reached target Slice Units. lb # [ 2.277792] systemd[1]: Reached target Swaps. lb # [ 2.278413] systemd[1]: Reached target Timer Units. lb # [ 2.279458] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 2.280530] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 2.281467] systemd[1]: Listening on Journal Sockets. lb # [ 2.282282] systemd[1]: Listening on udev Control Socket. client # [ 2.236217] systemd[1]: Finished Create List of Static Device Nodes. client # [ 2.237601] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 2.283224] systemd[1]: Listening on udev Kernel Socket. lb # [ 2.283974] systemd[1]: Reached target Socket Units. lb # [ 2.286448] systemd[1]: Starting Create List of Static Device Nodes... client # [ 2.249235] systemd-journald[73]: Collecting audit messages is disabled. worker2 # [ 2.300578] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 2.298578] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 2.259301] systemd[1]: Starting Journal Service... lb # [ 2.299823] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 2.314717] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 2.316121] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 2.317717] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 2.318817] systemd[1]: Reached target Path Units. worker1 # [ 2.278202] systemd[1]: Starting Load Kernel Modules... worker2 # [ 2.319668] systemd[1]: Reached target Slice Units. worker2 # [ 2.320510] systemd[1]: Reached target Swaps. worker1 # [ 2.279397] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 2.321288] systemd[1]: Reached target Timer Units. lb # [ 2.312559] systemd[1]: Mounting Kernel Configuration File System... client # [ 2.272309] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker2 # [ 2.322476] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 2.323840] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 2.325029] systemd[1]: Listening on Journal Sockets. worker2 # [ 2.326213] systemd[1]: Listening on udev Control Socket. worker2 # [ 2.326342] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 2.326365] systemd[1]: Reached target Socket Units. worker2 # [ 2.331248] systemd[1]: Starting Create List of Static Device Nodes... client # [ 2.284440] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 2.285851] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 2.338694] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 2.336934] systemd[1]: Starting Journal Service... worker1 # [ 2.295195] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 2.339881] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 2.351707] systemd[1]: Mounting Kernel Configuration File System... lb # [ 2.349479] systemd[1]: Starting Load Kernel Modules... lb # [ 2.350393] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 2.315222] systemd[1]: Finished Create List of Static Device Nodes. client # [ 2.329452] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 2.339451] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 2.343586] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 2.344198] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 2.391485] systemd[1]: Starting Journal Service... lb # [ 2.391837] systemd[1]: Starting Coldplug All udev Devices... lb # [ 2.396873] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 2.401643] systemd[1]: Starting Load Kernel Modules... lb # [ 2.399981] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker2 # [ 2.402632] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 2.351305] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 2.363424] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 2.402700] systemd-journald[73]: Collecting audit messages is disabled. worker1 # [ 2.367243] systemd-journald[73]: Collecting audit messages is disabled. worker2 # [ 2.418902] systemd[1]: Starting Coldplug All udev Devices... client # [ 2.367162] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 client # [ 2.367426] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 2.367438] [drm] features: -context_init worker1 # [ 2.383821] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 2.427092] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker2 # [ 2.428772] systemd[1]: Finished Create List of Static Device Nodes. client # [ 2.392460] [drm] number of scanouts: 1 client # [ 2.392500] [drm] number of cap sets: 0 client # [ 2.395433] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic client # [ 2.395466] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker1 # [ 2.403225] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 2.444459] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 2.443351] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 2.455642] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 2.424132] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 2.474258] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 2.432673] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker1 # [ 2.432943] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 2.432958] [drm] features: -context_init worker1 # [ 2.433793] [drm] number of scanouts: 1 worker1 # [ 2.433816] [drm] number of cap sets: 0 worker2 # [ 2.486124] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 2.451847] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.436374] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 2.459419] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic worker1 # [ 2.459452] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 lb # [ 2.507441] systemd[1]: Mounted Kernel Configuration File System. client # [ 2.461475] Console: switching to colour frame buffer device 160x50 client # [ 2.462257] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.524045] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 2.539025] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 2.548285] systemd-journald[73]: Collecting audit messages is disabled. worker1 # [ 2.499666] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.506552] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.519763] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 2.571648] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.510372] systemd-modules-load[75]: Inserted module 'dm_mod' lb # [ 2.575341] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 2.537614] systemd[1]: Finished Load Kernel Modules. client # [ 2.532181] systemd[1]: Finished Load Kernel Modules. client # [ 2.535155] systemd[1]: Started Journal Service. lb # [ 2.587125] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker2 # [ 2.590192] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 2.587405] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 2.592487] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 2.587419] [drm] features: -context_init lb # [ 2.588416] [drm] number of scanouts: 1 lb # [ 2.588441] [drm] number of cap sets: 0 lb # [ 2.592227] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic lb # [ 2.592250] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker1 # [ 2.555615] systemd[1]: Starting Apply Kernel Variables... client # [ 2.520523] systemd-modules-load[75]: Module 'virtio_balloon' is built in client # [ 2.536967] systemd-modules-load[75]: Module 'virtio_console' is built in client # [ 2.538291] systemd-modules-load[75]: Inserted module 'virtio_gpu' client # [ 2.539382] systemd-modules-load[75]: Module 'virtio_rng' is built in worker2 # [ 2.606385] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.548696] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 2.549865] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.550928] systemd[1]: Reached target Local File Systems. client # [ 2.551873] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.619223] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker2 # [ 2.619607] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 2.619623] [drm] features: -context_init worker2 # [ 2.620476] [drm] number of scanouts: 1 worker2 # [ 2.620498] [drm] number of cap sets: 0 worker2 # [ 2.624780] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 2.560420] systemd[1]: Starting Create System Files and Directories... client # [ 2.570366] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.635184] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.602356] systemd[1]: Started Journal Service. worker1 # [ 2.596696] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.650884] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic worker2 # [ 2.650925] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker1 # [ 2.605707] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.660524] systemd[1]: Started Journal Service. worker1 # [ 2.607654] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.643365] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.663871] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.649169] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.651102] systemd-modules-load[74]: Module 'virtio_console' is built in worker1 # [ 2.620766] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.622021] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.623160] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.661728] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.632658] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 2.633952] systemd[1]: Reached target Local File Systems. lb # [ 2.668606] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.634916] systemd[1]: Starting Create System Files and Directories... lb # [ 2.672912] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.641622] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.677346] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 2.643661] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.682515] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 2.706161] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.706423] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 2.706453] systemd[1]: Reached target Local File Systems. lb # [ 2.692579] systemd[1]: Reached target Local File Systems. client # [ 2.649772] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.711045] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.697759] systemd[1]: Starting Apply Kernel Variables... lb # [ 2.699329] systemd[1]: Starting Create System Files and Directories... client # [ 2.672589] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.733945] Console: switching to colour frame buffer device 160x50 lb # [ 2.725450] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.695678] systemd-udevd[81]: Using default interface naming scheme 'v261'. worker2 # [ 2.762244] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.713489] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.774203] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.777133] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.746153] systemd-udevd[82]: Using default interface naming scheme 'v261'. client # [ 2.743348] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.789874] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.800906] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.780260] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.833852] systemd-udevd[83]: Using default interface naming scheme 'v261'. worker2 # [ 2.836480] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.859100] systemd[1]: Started Journal Service. worker2 # [ 2.841834] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 2.843008] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.856459] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.857858] systemd-modules-load[74]: Module 'virtio_rng' is built in worker2 # [ 2.858930] systemd[1]: Starting Create System Files and Directories... lb # [ 2.866503] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.869513] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.878267] systemd[1]: Finished Apply Kernel Variables. client # [ 2.840370] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.880906] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.937926] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.944576] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.908774] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.960234] systemd[1]: Starting Virtual Console Setup... client # [ 2.916804] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.948713] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.956794] systemd[1]: Finished Virtual Console Setup. lb # [ 3.021697] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 3.024887] systemd[1]: Starting Virtual Console Setup... lb # [ 3.028750] systemd[1]: Finished Virtual Console Setup. worker2 # [ 3.107571] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 3.109533] systemd[1]: Finished Virtual Console Setup. client # [ 3.621847] systemd[1]: Finished Coldplug All udev Devices. client # [ 3.622862] systemd[1]: Reached target System Initialization. client # [ 3.623689] systemd[1]: Reached target Basic System. worker1 # [ 3.686646] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 3.689946] systemd[1]: Reached target System Initialization. worker1 # [ 3.692397] systemd[1]: Reached target Basic System. lb # [ 3.800986] systemd[1]: Finished Coldplug All udev Devices. lb # [ 3.803300] systemd[1]: Reached target System Initialization. lb # [ 3.805654] systemd[1]: Reached target Basic System. worker2 # [ 3.882577] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 3.884526] systemd[1]: Reached target System Initialization. worker2 # [ 3.885956] systemd[1]: Reached target Basic System. worker1 # [ 4.092854] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 4.104303] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 4.107226] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 4.360777] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 4.469194] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 4.491066] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. lb # [ 4.713539] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 4.715521] systemd[1]: Reached target Initrd Root Device. lb # [ 4.716598] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 4.949168] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 4.951982] systemd[1]: Reached target Initrd Root Device. worker2 # [ 4.957584] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 4.978302] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 4.988690] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. lb # [ 5.002436] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line. lb # [ 5.050713] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 5.070577] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 5.075253] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 5.088329] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 5.096610] systemd[1]: Mounting /sysroot... lb # [ 5.100912] systemd[1]: Mounting /sysroot... worker2 # [ 5.315068] EXT4-fs (vda): mounted filesystem 46601aa9-a2da-4d97-bf78-b2a378c498a5 r/w with ordered data mode. Quota mode: none. worker2 # [ 5.315045] systemd[1]: Mounted /sysroot. worker2 # [ 5.318945] systemd[1]: Reached target Initrd Root File System. worker2 # [ 5.324950] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 5.342026] EXT4-fs (vda): mounted filesystem 32371818-190c-4a85-8b65-8a16c473a43c r/w with ordered data mode. Quota mode: none. lb # [ 5.361681] systemd[1]: Mounted /sysroot. lb # [ 5.374570] systemd[1]: Reached target Initrd Root File System. lb # [ 5.382520] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 5.404470] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 5.417907] systemd[1]: Mounting /sysroot/run... worker2 # [ 5.428492] systemd-sysroot-fstab-check[121]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 5.432208] systemd[1]: Reload requested from client PID 121 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 5.435532] systemd[1]: Reloading... lb # [ 5.462043] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 5.534147] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 5.664493] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 5.673968] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 5.729272] systemd[1]: Reached target Initrd Root Device. worker1 # [ 5.730422] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 5.772304] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 5.849861] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 5.852768] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 5.860608] systemd[1]: Mounted /sysroot/run. lb # [ 5.904834] systemd-sysroot-fstab-check[125]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 5.948325] systemd[1]: Reload requested from client PID 125 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 5.950193] systemd[1]: Reloading... worker1 # [ 5.934863] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 5.954696] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 5.956550] systemd[1]: Mounting /sysroot... worker2 # [ 5.994704] systemd[1]: Reloading finished in 563 ms. worker2 # [ 6.021966] systemd-sysroot-fstab-check[121]: Requesting initrd-fs.target/start/replace... worker2 # [ 6.026355] systemd-sysroot-fstab-check[121]: Requesting swap.target/start/replace... worker2 # [ 6.030213] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 6.031851] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 5.986358] (udev-worker)[110]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 6.033273] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 6.015021] EXT4-fs (vda): mounted filesystem 8de9c3ca-d696-4c24-be69-648c18db6bb8 r/w with ordered data mode. Quota mode: none. client # [ 5.995810] (udev-worker)[110]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.043595] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 6.010212] systemd[1]: Mounted /sysroot. worker1 # [ 6.011532] systemd[1]: Reached target Initrd Root File System. worker1 # [ 6.016306] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 6.052510] systemd[1]: Mounting /sysroot/run... client # [ 6.009299] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 6.012832] systemd[1]: Reached target Initrd Root Device. worker2 # [ 6.060584] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 6.015810] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 6.085307] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 6.051266] systemd-sysroot-fstab-check[132]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 6.088771] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 6.056317] systemd[1]: Reload requested from client PID 132 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 6.062357] systemd[1]: Reloading... worker2 # [ 6.108572] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 6.106900] systemd[1]: Reloading finished in 154 ms. worker2 # [ 6.111925] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 6.067957] systemd-fsck[122]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 6.121709] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 6.076240] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 6.079300] systemd[1]: Mounting /sysroot... worker2 # [ 6.128847] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 6.132811] systemd[1]: Mounted /sysroot/run. lb # [ 6.133112] systemd-sysroot-fstab-check[125]: Requesting initrd-fs.target/start/replace... lb # [ 6.136775] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 6.140502] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 6.143854] systemd-sysroot-fstab-check[125]: Requesting swap.target/start/replace... lb # [ 6.153614] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 6.155373] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 6.158353] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 6.163232] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 6.139217] EXT4-fs (vda): mounted filesystem 0e25af29-1167-4ea2-9706-e005ba0f90e9 r/w with ordered data mode. Quota mode: none. worker2 # [ 6.168495] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 6.129332] systemd[1]: Mounted /sysroot. client # [ 6.131678] systemd[1]: Reached target Initrd Root File System. client # [ 6.140219] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 6.189254] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 6.192130] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 6.195403] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 6.200382] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 6.214880] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 6.216757] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 6.171383] systemd-sysroot-fstab-check[130]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 6.219819] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 6.177141] systemd[1]: Reload requested from client PID 130 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 6.223447] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 6.181522] systemd[1]: Reloading... lb # [ 6.246510] systemd[1]: Mounted /sysroot/nix/store. lb # [ 6.249597] systemd[1]: Reached target Initrd File Systems. lb # [ 6.256256] systemd[1]: Starting Find NixOS closure... lb # [ 6.276320] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 6.314890] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 6.340465] systemd[1]: Finished Find NixOS closure. lb # [ 6.344467] systemd[1]: Reached target Initrd Default Target. lb # [ 6.350522] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 6.320141] systemd[1]: Reloading finished in 258 ms. worker1 # [ 6.351251] systemd-sysroot-fstab-check[132]: Requesting initrd-fs.target/start/replace... worker1 # [ 6.356320] systemd-sysroot-fstab-check[132]: Requesting swap.target/start/replace... lb # [ 6.394580] systemd[1]: Stopped target Initrd Default Target. lb # [ 6.398173] systemd[1]: Stopped target Basic System. lb # [ 6.399372] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 6.363924] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 6.366824] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 6.405357] systemd[1]: Stopped target Path Units. lb # [ 6.406193] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 6.407475] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 6.376489] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 6.412311] systemd[1]: Stopped target Slice Units. worker1 # [ 6.378137] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 6.414994] systemd[1]: Stopped target Socket Units. lb # [ 6.420296] systemd[1]: Stopped target System Initialization. lb # [ 6.424138] systemd[1]: Stopped target Swaps. lb # [ 6.424918] systemd[1]: Stopped target Timer Units. lb # [ 6.425685] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 6.429410] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 6.432192] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 6.436415] systemd[1]: Stopped Find NixOS closure. lb # [ 6.441292] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 6.407569] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 6.443539] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 6.412203] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 6.446891] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 6.451063] systemd[1]: Stopped Apply Kernel Variables. lb # [ 6.453554] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 6.457693] systemd[1]: Stopped Load Kernel Modules. lb # [ 6.464634] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 6.469005] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 6.434897] systemd[1]: Reloading finished in 254 ms. lb # [ 6.480880] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 6.483900] systemd[1]: Stopped Create System Files and Directories. lb # [ 6.488278] systemd[1]: Stopped target Local File Systems. lb # [ 6.489168] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 6.490164] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 6.491167] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 6.491955] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 6.500648] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 6.503063] systemd[1]: Stopped Virtual Console Setup. lb # [ 6.512376] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 6.467591] systemd-sysroot-fstab-check[130]: Requesting initrd-fs.target/start/replace... client # [ 6.473980] systemd-sysroot-fstab-check[130]: Requesting swap.target/start/replace... lb # [ 6.521246] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 6.523928] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 6.479425] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 6.527970] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 6.484455] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 6.531389] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 6.486503] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 6.537063] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 6.501216] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 6.572885] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 6.576208] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 6.578798] systemd[1]: systemd-udevd.service: Consumed 1.288s CPU time over 3.848s wall clock time, 15.3M memory peak. client # [ 6.536533] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 6.539417] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 6.584895] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 6.586051] systemd[1]: Closed udev Control Socket. lb # [ 6.586815] systemd[1]: Starting Cleanup udev Database... lb # [ 6.587663] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 6.590245] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 6.591305] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 6.592607] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 6.593641] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 6.594619] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 6.618348] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 6.620759] systemd[1]: Finished Cleanup udev Database. lb # [ 6.624713] systemd[1]: Reached target Switch Root. lb # [ 6.627645] systemd[1]: Starting NixOS Activation... lb # [ 6.827693] initrd-nixos-activation-start[183]: booting system configuration /nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test worker1 # [ 6.812121] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 6.824188] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 6.851231] systemd[1]: Mounting /sysroot/run... lb # [ 6.906191] initrd-nixos-activation-start[183]: running activation script... worker1 # [ 6.873941] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 6.889369] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 6.936334] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 6.981926] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 6.995721] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 6.998705] systemd[1]: Mounted /sysroot/run. worker2 # [ 7.039241] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 7.013002] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 7.024203] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 7.043743] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 7.044678] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 7.062993] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 7.075767] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 7.078711] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 7.081849] systemd[1]: Mounting /sysroot/run... worker2 # [ 7.147067] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 7.160920] systemd[1]: Reached target Initrd File Systems. client # [ 7.116291] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 7.172237] systemd[1]: Starting Find NixOS closure... worker2 # [ 7.192997] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 7.172530] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 7.270591] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 7.278856] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 7.254559] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 7.261791] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 7.264219] systemd[1]: Mounted /sysroot/run. client # [ 7.300952] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 7.360479] systemd[1]: Finished Find NixOS closure. client # [ 7.315134] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 7.364322] systemd[1]: Reached target Initrd Default Target. client # [ 7.324858] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 7.370531] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 7.357757] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 7.361728] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 7.372381] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 7.437552] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 7.440397] systemd[1]: Stopped target Basic System. worker2 # [ 7.442091] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 7.443138] systemd[1]: Stopped target Path Units. worker2 # [ 7.444190] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 7.449829] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.452154] systemd[1]: Stopped target Slice Units. worker2 # [ 7.452918] systemd[1]: Stopped target Socket Units. worker2 # [ 7.456183] systemd[1]: Stopped target System Initialization. worker2 # [ 7.461409] systemd[1]: Stopped target Swaps. worker2 # [ 7.462217] systemd[1]: Stopped target Timer Units. worker2 # [ 7.464178] systemd[1]: dbus.socket: Deactivated successfully. client # [ 7.419031] systemd[1]: Mounted /sysroot/nix/store. client # [ 7.422412] systemd[1]: Reached target Initrd File Systems. worker2 # [ 7.468268] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 7.469252] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 7.428164] systemd[1]: Starting Find NixOS closure... worker2 # [ 7.476688] systemd[1]: Stopped Find NixOS closure. worker2 # [ 7.479674] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 7.480787] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 7.484308] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 7.488265] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 7.489174] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 7.488002] initrd-nixos-activation-start[206]: setting up /etc... client # [ 7.448873] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 7.495137] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 7.497301] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 7.500514] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 7.502765] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 7.516336] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 7.523304] systemd[1]: Stopped target Local File Systems. worker2 # [ 7.531712] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 7.538618] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 7.539814] systemd[1]: Stopped Coldplug All udev Devices. client # [ 7.494410] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 7.544431] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 7.549056] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 7.552481] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 7.556385] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 7.559853] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 7.524968] (udev-worker)[122]: mtd0ro: Failed to find and pin callout binary "/nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 7.562297] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 7.565109] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 7.568297] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 7.531470] (udev-worker)[122]: mtd0ro: /nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 7.569441] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 7.528252] systemd[1]: Finished Find NixOS closure. client # [ 7.529181] systemd[1]: Reached target Initrd Default Target. client # [ 7.532412] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 7.588466] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 7.590495] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 7.594124] systemd[1]: systemd-udevd.service: Consumed 2.495s CPU time over 4.900s wall clock time, 24.5M memory peak. worker2 # [ 7.597069] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 7.600570] systemd[1]: Closed udev Control Socket. worker2 # [ 7.601466] systemd[1]: Starting Cleanup udev Database... worker2 # [ 7.604233] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 7.605476] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 7.608265] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 7.574503] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 7.609764] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 7.610943] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 7.611959] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 7.578530] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 7.580179] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 7.584377] systemd[1]: Starting Virtual Console Setup... client # [ 7.579306] systemd[1]: Stopped target Initrd Default Target. client # [ 7.582091] systemd[1]: Stopped target Basic System. client # [ 7.584980] systemd[1]: Stopped target Initrd Root Device. client # [ 7.589248] systemd[1]: Stopped target Path Units. client # [ 7.592806] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 7.604210] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 7.605899] systemd[1]: Stopped Virtual Console Setup. client # [ 7.594977] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.609920] systemd[1]: Starting Virtual Console Setup... client # [ 7.599238] systemd[1]: Stopped target Slice Units. client # [ 7.605109] systemd[1]: Stopped target Socket Units. client # [ 7.606903] systemd[1]: Stopped target System Initialization. client # [ 7.608792] systemd[1]: Stopped target Swaps. client # [ 7.609462] systemd[1]: Stopped target Timer Units. worker2 # [ 7.656501] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 7.611387] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 7.659190] systemd[1]: Finished Cleanup udev Database. client # [ 7.617378] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 7.618249] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 7.619278] systemd[1]: Stopped Find NixOS closure. client # [ 7.619975] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 7.661926] systemd[1]: Reached target Switch Root. worker2 # [ 7.663167] systemd[1]: Starting NixOS Activation... client # [ 7.625979] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 7.640665] systemd-vconsole-setup[168]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 7.643940] systemd[1]: Finished Virtual Console Setup. client # [ 7.632743] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 7.634609] systemd[1]: Stopped Apply Kernel Variables. client # [ 7.636349] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 7.638557] systemd[1]: Stopped Load Kernel Modules. client # [ 7.641832] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 7.646317] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 7.657368] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 7.661664] systemd[1]: Stopped Create System Files and Directories. client # [ 7.670015] systemd[1]: Stopped target Local File Systems. client # [ 7.675802] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 7.678909] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 7.679901] systemd[1]: Stopped Coldplug All udev Devices. client # [ 7.685540] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 7.689084] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.692494] systemd[1]: Stopped Virtual Console Setup. client # [ 7.695062] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 7.696636] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 7.700574] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 7.701660] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 7.704133] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 7.705217] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 7.752287] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 7.755367] systemd[1]: Finished NixOS Activation. lb # [ 7.756812] systemd[1]: Starting Switch Root... client # [ 7.712271] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 7.715117] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 7.717067] systemd[1]: systemd-udevd.service: Consumed 2.611s CPU time over 5.142s wall clock time, 25.2M memory peak. client # [ 7.720578] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 7.724507] systemd[1]: Closed udev Control Socket. client # [ 7.725346] systemd[1]: Starting Cleanup udev Database... client # [ 7.726117] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 7.728413] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 7.729592] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 7.730832] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 7.731833] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 7.732943] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 7.782364] systemd[1]: Switching root. client # [ 7.770381] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 7.772587] systemd[1]: Finished Cleanup udev Database. client # [ 7.775596] systemd[1]: Reached target Switch Root. client # [ 7.780743] systemd[1]: Starting NixOS Activation... worker1 # [ 7.808151] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 7.845046] initrd-nixos-activation-start[186]: booting system configuration /nix/store/78c7ibbkdrnd8irpxz37agc6v4py6pgh-nixos-system-worker2-test worker1 # [ 7.862879] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 7.866034] systemd[1]: Reached target Initrd File Systems. worker1 # [ 7.871082] systemd[1]: Starting Find NixOS closure... worker1 # [ 7.894275] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 7.932325] initrd-nixos-activation-start[186]: running activation script... lb # [ 7.978876] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). worker1 # [ 7.944850] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 7.950550] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 7.975181] systemd[1]: Finished Find NixOS closure. worker1 # [ 7.976248] systemd[1]: Reached target Initrd Default Target. worker1 # [ 7.977647] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 7.990387] initrd-nixos-activation-start[183]: booting system configuration /nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test worker1 # [ 8.015275] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 8.020326] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 8.023482] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 8.028620] systemd[1]: Stopped target Basic System. worker1 # [ 8.029391] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 8.030194] systemd[1]: Stopped target Path Units. worker1 # [ 8.030899] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 8.033439] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 8.036222] systemd[1]: Stopped target Slice Units. worker1 # [ 8.040285] systemd[1]: Stopped target Socket Units. worker1 # [ 8.041228] systemd[1]: Stopped target System Initialization. worker1 # [ 8.042090] systemd[1]: Stopped target Swaps. worker1 # [ 8.042749] systemd[1]: Stopped target Timer Units. worker1 # [ 8.044697] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 8.049380] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 8.052886] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 8.056853] systemd[1]: Stopped Find NixOS closure. worker1 # [ 8.058299] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 8.064226] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 8.065235] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 8.066181] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 8.066989] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 8.067989] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 8.081343] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 8.085148] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 8.092928] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 8.084188] initrd-nixos-activation-start[183]: running activation script... worker1 # [ 8.098004] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 8.108582] systemd[1]: Stopped target Local File Systems. worker1 # [ 8.112431] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 8.113915] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 8.115134] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 8.121873] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 8.128354] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.129582] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.130364] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 8.131829] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 8.134159] systemd[1]: systemd-udevd.service: Consumed 2.520s CPU time over 5.461s wall clock time, 30.3M memory peak. worker1 # [ 8.136723] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 8.139320] systemd[1]: Closed udev Control Socket. worker1 # [ 8.140929] systemd[1]: Starting Cleanup udev Database... worker1 # [ 8.143769] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 8.146688] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 8.147689] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 8.149050] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 8.150066] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 8.151029] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 8.151902] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 8.159561] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 8.160701] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 8.164221] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 8.204671] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 8.206742] systemd[1]: Finished Cleanup udev Database. worker1 # [ 8.210597] systemd[1]: Reached target Switch Root. worker1 # [ 8.212395] systemd[1]: Starting NixOS Activation... worker1 # [ 8.427609] initrd-nixos-activation-start[193]: booting system configuration /nix/store/15akwfj87h9x1a1mm1madsr1gizhj855-nixos-system-worker1-test worker2 # [ 8.495527] initrd-nixos-activation-start[209]: setting up /etc... worker1 # [ 8.502586] initrd-nixos-activation-start[193]: running activation script... lb # [ 8.625163] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 8.638284] systemd[1]: Detected virtualization qemu. lb # [ 8.643418] systemd[1]: Detected architecture arm64. lb # [ 8.647637] systemd[1]: Detected first boot. lb # [ 8.656593] systemd[1]: Initializing machine ID from random generator. client # [ 8.602905] initrd-nixos-activation-start[206]: setting up /etc... worker2 # [ 8.800562] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 8.805307] systemd[1]: Finished NixOS Activation. worker2 # [ 8.812428] systemd[1]: Starting Switch Root... worker2 # [ 8.834712] systemd[1]: Switching root. client # [ 8.908813] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 8.911876] systemd[1]: Finished NixOS Activation. client # [ 8.913457] systemd[1]: Starting Switch Root... client # [ 8.938158] systemd[1]: Switching root. lb # [ 9.003389] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 8.984957] initrd-nixos-activation-start[216]: setting up /etc... worker2 # [ 9.042016] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). client # [ 9.126136] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). lb # [ 9.228945] systemd[1]: Applying preset policy. worker1 # [ 9.271161] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 9.274354] systemd[1]: Finished NixOS Activation. worker1 # [ 9.275604] systemd[1]: Starting Switch Root... worker1 # [ 9.305695] systemd[1]: Switching root. worker1 # [ 9.502630] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). worker2 # [ 9.660426] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 9.673267] systemd[1]: Detected virtualization qemu. worker2 # [ 9.678117] systemd[1]: Detected architecture arm64. worker2 # [ 9.681904] systemd[1]: Detected first boot. worker2 # [ 9.690794] systemd[1]: Initializing machine ID from random generator. lb # [ 9.792228] systemd[1]: Populated /etc with preset unit settings. client # [ 9.739695] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 9.752406] systemd[1]: Detected virtualization qemu. client # [ 9.757183] systemd[1]: Detected architecture arm64. client # [ 9.759577] systemd[1]: Detected first boot. client # [ 9.769048] systemd[1]: Initializing machine ID from random generator. worker2 # [ 10.061194] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 10.102260] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 10.123899] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 10.139366] systemd[1]: Detected virtualization qemu. worker1 # [ 10.141700] systemd[1]: Detected architecture arm64. worker1 # [ 10.146453] systemd[1]: Detected first boot. worker1 # [ 10.155406] systemd[1]: Initializing machine ID from random generator. worker2 # [ 10.277017] systemd[1]: Applying preset policy. client # [ 10.281158] systemd[1]: Applying preset policy. lb # [ 10.386598] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 10.388494] systemd[1]: Stopped initrd-switch-root.service. lb # [ 10.393227] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 10.399778] systemd[1]: Created slice Slice /system/getty. lb # [ 10.402065] systemd[1]: Created slice User and Session Slice. lb # [ 10.404771] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 10.407531] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 10.410136] systemd[1]: Expecting device /dev/hvc0... lb # [ 10.412309] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 10.414422] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 10.416685] systemd[1]: Stopped target initrd-fs.target. lb # [ 10.420086] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 10.422601] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 10.425221] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 10.427921] systemd[1]: Reached target Path Units. lb # [ 10.430257] systemd[1]: Reached target Remote File Systems. lb # [ 10.432620] systemd[1]: Reached target Slice Units. lb # [ 10.433760] systemd[1]: Reached target Swaps. lb # [ 10.439677] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 10.447251] systemd[1]: Listening on Process Core Dump Socket. lb # [ 10.452130] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 10.457144] systemd[1]: Listening on Factory Reset Management. lb # [ 10.458637] systemd[1]: Listening on Hostname Service Socket. lb # [ 10.466355] systemd[1]: Starting Journal Log Access Socket... lb # [ 10.470527] systemd[1]: Listening on Journal Audit Socket. lb # [ 10.478316] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 10.482783] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 10.485775] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 10.489241] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 10.503945] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 10.507008] systemd[1]: Listening on udev Control Socket. lb # [ 10.509854] systemd[1]: Listening on udev Varlink Socket. lb # [ 10.516568] systemd[1]: Mounting Huge Pages File System... lb # [ 10.524030] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 10.535474] systemd[1]: Mounting Kernel Debug File System... lb # [ 10.553221] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 10.521418] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 10.569338] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 10.584213] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 10.591250] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.607299] systemd[1]: Mounting Kernel Configuration File System... lb # [ 10.615424] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 10.622616] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 10.679241] systemd[1]: Starting Load Kernel Module fuse... lb # [ 10.683198] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 10.792075] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 10.825016] systemd[1]: Applying preset policy. client # [ 10.849558] systemd[1]: Populated /etc with preset unit settings. lb # [ 10.988394] systemd[1]: Starting Journal Service... lb # [ 11.066507] systemd[1]: Starting Load Kernel Modules... lb # [ 11.128016] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 11.192772] systemd-journald[278]: Collecting audit messages is enabled. lb # [ 11.199857] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 11.200376] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 11.194231] systemd[1]: Queued start job for default target Multi-User System. lb # [ 11.200741] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 11.232932] fuse: init (API version 7.45) lb # [ 11.249096] systemd[1]: Starting Coldplug All udev Devices... lb # [ 11.277294] systemd[1]: Started Journal Service. lb # [ 11.265732] systemd-modules-load[279]: Module 'atkbd' is built in lb # [ 11.267327] systemd-modules-load[279]: Module 'loop' is built in lb # [ 11.277539] systemd[1]: Listening on Journal Log Access Socket. lb # [ 11.279775] systemd[1]: Mounted Huge Pages File System. lb # [ 11.286234] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 11.288995] systemd[1]: Mounted Kernel Debug File System. lb # [ 11.297063] systemd[1]: Mounted Kernel Trace File System. lb # [ 11.302568] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 11.312618] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 11.313772] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 11.314671] systemd[1]: Mounted Kernel Configuration File System. lb # [ 11.315538] systemd[1]: modprobe@fuse.service: Deactivated successfully. worker2 # [ 11.343150] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 11.344611] systemd[1]: Stopped initrd-switch-root.service. lb # [ 11.328530] systemd[1]: Finished Load Kernel Module fuse. worker2 # [ 11.348445] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 11.353850] systemd[1]: Created slice Slice /system/getty. worker2 # [ 11.355870] systemd[1]: Created slice User and Session Slice. worker2 # [ 11.356924] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 11.360304] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 11.335206] systemd[1]: Finished Load Kernel Modules. worker2 # [ 11.361386] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 11.362056] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 11.364074] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 11.366119] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 11.369115] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 11.371453] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 11.373893] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 11.376262] systemd[1]: Reached target Path Units. worker2 # [ 11.378823] systemd[1]: Reached target Remote File Systems. lb # [ 11.360414] systemd[1]: Mounting FUSE Control File System... worker2 # [ 11.381473] systemd[1]: Reached target Slice Units. worker2 # [ 11.384043] systemd[1]: Reached target Swaps. worker2 # [ 11.390075] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 11.396659] systemd[1]: Listening on Process Core Dump Socket. lb # [ 11.375457] systemd-oomd[280]: No swap; memory pressure usage will be degraded worker2 # [ 11.402483] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 11.407513] systemd[1]: Listening on Factory Reset Management. lb # [ 11.384737] systemd[1]: Starting Firewall... worker2 # [ 11.410691] systemd[1]: Listening on Hostname Service Socket. lb # [ 11.407816] EXT4-fs (vda): re-mounted 32371818-190c-4a85-8b65-8a16c473a43c. worker2 # [ 11.418718] systemd[1]: Starting Journal Log Access Socket... lb # [ 11.400531] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 11.421267] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 11.426476] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 11.429164] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 11.431657] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 11.435940] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 11.416490] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 11.426116] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 11.450011] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 11.452173] systemd[1]: Listening on udev Control Socket. worker2 # [ 11.454468] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 11.461446] systemd[1]: Mounting Huge Pages File System... worker2 # [ 11.467173] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 11.454179] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 11.458274] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 11.482160] systemd[1]: Mounting Kernel Debug File System... client # [ 11.438312] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 11.439772] systemd[1]: Stopped initrd-switch-root.service. client # [ 11.444887] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 11.451720] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 11.494518] systemd[1]: Mounting Kernel Trace File System... client # [ 11.449611] systemd[1]: Created slice Slice /system/getty. client # [ 11.451393] systemd[1]: Created slice User and Session Slice. client # [ 11.452854] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 11.456177] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 11.457265] systemd[1]: Expecting device /dev/hvc0... client # [ 11.459073] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 11.488545] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 11.461084] systemd[1]: Reached target Local Encrypted Volumes. client # [ 11.462405] systemd[1]: Stopped target initrd-fs.target. client # [ 11.462744] systemd[1]: Stopped target initrd-root-fs.target. client # [ 11.463055] systemd[1]: Stopped target initrd-switch-root.target. client # [ 11.463369] systemd[1]: Reached target Virtual Machines and Containers. client # [ 11.463698] systemd[1]: Reached target Path Units. client # [ 11.468443] systemd[1]: Reached target Remote File Systems. worker2 # [ 11.510511] systemd[1]: Starting Create List of Static Device Nodes... client # [ 11.471310] systemd[1]: Reached target Slice Units. lb # [ 11.491296] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 11.474573] systemd[1]: Reached target Swaps. client # [ 11.479754] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 11.486039] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 11.533074] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 11.491400] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 11.542417] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 11.496537] systemd[1]: Listening on Factory Reset Management. client # [ 11.499080] systemd[1]: Listening on Hostname Service Socket. client # [ 11.506215] systemd[1]: Starting Journal Log Access Socket... lb # [ 11.536552] systemd[1]: Starting Load/Save OS Random Seed... client # [ 11.510182] systemd[1]: Listening on Journal Audit Socket. lb # [ 11.540383] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 11.558929] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 11.559476] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 11.559838] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 11.519204] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 11.522211] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 11.525125] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 11.528085] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 11.579794] systemd[1]: Starting Load Kernel Module fuse... client # [ 11.540080] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 11.585655] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 11.542928] systemd[1]: Listening on udev Control Socket. client # [ 11.545608] systemd[1]: Listening on udev Varlink Socket. client # [ 11.552050] systemd[1]: Mounting Huge Pages File System... worker2 # [ 11.603661] systemd[1]: Starting Journal Service... client # [ 11.558611] systemd[1]: Mounting POSIX Message Queue File System... client # [ 11.566688] systemd[1]: Mounting Kernel Debug File System... client # [ 11.584080] systemd[1]: Mounting Kernel Trace File System... lb # [ 11.615625] systemd[1]: Mounted FUSE Control File System. client # [ 11.597636] systemd[1]: Starting Create List of Static Device Nodes... client # [ 11.612184] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 11.614900] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 11.649376] systemd[1]: Finished Apply Kernel Variables. client # [ 11.637324] systemd[1]: Mounting Kernel Configuration File System... client # [ 11.642613] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 11.645944] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 11.690149] systemd-journald[278]: Received client request to flush runtime journal. lb # [ 11.760208] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 11.780633] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 11.782894] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 11.788834] systemd[1]: Reached target First Boot Complete. lb # [ 11.790250] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 11.832788] systemd[1]: Starting Load Kernel Modules... client # [ 11.823242] systemd[1]: Starting Load Kernel Module fuse... client # [ 11.825584] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 11.881970] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 11.879297] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 11.880864] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 11.888772] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 11.909170] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 11.870888] systemd[1]: Starting Journal Service... worker2 # [ 11.917688] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 11.939110] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 11.952046] systemd[1]: Listening on Journal Log Access Socket. client # [ 11.907369] systemd[1]: Starting Load Kernel Modules... worker2 # [ 11.957940] systemd[1]: Mounted Huge Pages File System. worker2 # [ 11.961931] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 11.970030] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 11.973499] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 11.977293] systemd[1]: Mounted Kernel Configuration File System. client # [ 11.938418] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 11.961636] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 11.962161] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 12.052431] systemd[1]: Finished Create List of Static Device Nodes. client # [ 12.006448] fuse: init (API version 7.45) worker2 # [ 12.064658] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 12.029043] systemd[1]: Starting Coldplug All udev Devices... client # [ 12.038038] systemd-journald[278]: Collecting audit messages is enabled. client # [ 12.037846] systemd[1]: Queued start job for default target Multi-User System. client # [ 12.041175] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 12.067624] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 12.069031] systemd[1]: Stopped initrd-switch-root.service. client # [ 12.063384] systemd[1]: Started Journal Service. worker1 # [ 12.072750] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 12.049651] systemd-modules-load[279]: Module 'atkbd' is built in worker1 # [ 12.077422] systemd[1]: Created slice Slice /system/getty. worker1 # [ 12.079576] systemd[1]: Created slice User and Session Slice. client # [ 12.054115] systemd-modules-load[279]: Module 'loop' is built in worker1 # [ 12.080585] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 12.082824] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 12.060445] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 12.085070] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 12.087244] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 12.090109] systemd[1]: Reached target Local Encrypted Volumes. client # [ 12.065964] systemd[1]: Mounted Huge Pages File System. worker1 # [ 12.091446] systemd[1]: Stopped target initrd-fs.target. client # [ 12.069390] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 12.094244] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 12.096555] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 12.098885] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 12.101889] systemd[1]: Reached target Path Units. client # [ 12.078856] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 12.104325] systemd[1]: Reached target Remote File Systems. worker1 # [ 12.106671] systemd[1]: Reached target Slice Units. worker1 # [ 12.108859] systemd[1]: Reached target Swaps. worker2 # [ 12.153513] EXT4-fs (vda): re-mounted 46601aa9-a2da-4d97-bf78-b2a378c498a5. client # [ 12.082774] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 12.113933] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 12.120070] systemd[1]: Listening on Process Core Dump Socket. client # [ 12.091969] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 12.125030] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 12.100989] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 12.102253] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 12.103238] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 12.130096] systemd[1]: Listening on Factory Reset Management. worker1 # [ 12.131597] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 12.170056] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 12.170825] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 12.171258] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 12.140076] systemd[1]: Starting Journal Log Access Socket... client # [ 12.116613] systemd[1]: modprobe@fuse.service: Deactivated successfully. worker1 # [ 12.142842] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 12.182931] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 12.186238] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 12.149223] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 12.121388] systemd[1]: Finished Load Kernel Module fuse. worker2 # [ 12.190648] systemd-journald[281]: Collecting audit messages is enabled. worker1 # [ 12.151358] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 12.193306] systemd[1]: Finished Load Kernel Modules. worker1 # [ 12.153101] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 12.128585] systemd[1]: Finished Load Kernel Modules. worker1 # [ 12.155339] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 12.205979] systemd[1]: Starting Firewall... worker1 # [ 12.166190] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 12.166800] systemd[1]: Listening on udev Control Socket. worker1 # [ 12.169154] systemd[1]: Listening on udev Varlink Socket. client # [ 12.164223] EXT4-fs (vda): re-mounted 0e25af29-1167-4ea2-9706-e005ba0f90e9. worker1 # [ 12.174419] systemd[1]: Mounting Huge Pages File System... client # [ 12.155406] systemd[1]: Mounting FUSE Control File System... worker1 # [ 12.183180] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 12.234419] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 12.193065] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 12.237690] systemd[1]: Started Journal Service. client # [ 12.177235] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 12.207314] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 12.232340] systemd[1]: Queued start job for default target Multi-User System. client # [ 12.198205] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 12.224777] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 12.252446] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 12.213319] systemd-oomd[281]: No swap; memory pressure usage will be degraded worker2 # [ 12.263219] systemd-modules-load[282]: Module 'atkbd' is built in worker1 # [ 12.244236] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 12.247744] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 12.274012] systemd-modules-load[282]: Module 'loop' is built in client # [ 12.231366] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 12.264220] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 12.268111] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 12.239466] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 12.273289] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 12.257037] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 12.285630] systemd[1]: Starting Load Kernel Module fuse... worker1 # [ 12.286149] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 12.293647] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 12.305052] systemd[1]: Starting Journal Service... worker2 # [ 12.336424] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 12.294434] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 12.361469] fuse: init (API version 7.45) client # [ 12.295684] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 12.351954] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 12.307615] systemd[1]: Starting Load/Save OS Random Seed... client # [ 12.312546] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 12.383677] systemd-udevd[318]: Using default interface naming scheme 'v261'. worker2 # [ 12.383652] systemd-oomd[283]: No swap; memory pressure usage will be degraded worker2 # [ 12.413298] systemd[1]: modprobe@fuse.service: Deactivated successfully. client # [ 12.375921] systemd[1]: Finished Apply Kernel Variables. lb # [ 12.423971] systemd[1]: Mounting /run/wrappers... client # [ 12.383204] systemd[1]: Mounted FUSE Control File System. worker2 # [ 12.462284] systemd-journald[281]: Received client request to flush runtime journal. client # [ 12.450746] systemd-journald[278]: Received client request to flush runtime journal. lb # [ 12.489806] systemd[1]: Mounted /run/wrappers. lb # [ 12.491910] systemd[1]: Reached target Local File Systems. lb # [ 12.497183] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 12.505377] systemd[1]: Starting register-nix-paths.service... lb # [ 12.517998] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 12.522436] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 12.518225] systemd[1]: Starting Load Kernel Modules... lb # [ 12.544834] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 12.548340] systemd[1]: Finished Load Kernel Module fuse. worker2 # [ 12.557716] systemd[1]: Mounting FUSE Control File System... worker2 # [ 12.563873] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 12.520628] systemd[1]: Finished Load/Save OS Random Seed. client # [ 12.523406] systemd[1]: Reached target First Boot Complete. client # [ 12.529047] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 12.575343] systemd[1]: Starting Create System Files and Directories... client # [ 12.532624] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 12.575925] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 12.582631] systemd[1]: Reached target First Boot Complete. client # [ 12.537926] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 12.591690] systemd[1]: Mounted FUSE Control File System. worker2 # [ 12.598428] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 12.604243] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 12.608663] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 12.588871] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 12.613850] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 12.605867] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 12.609369] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 12.636315] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 12.654402] systemd[1]: Listening on Journal Log Access Socket. client # [ 12.633112] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 12.657975] systemd[1]: Mounted Huge Pages File System. client # [ 12.635708] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 12.661301] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 12.667587] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 12.690084] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 12.646371] systemd[1]: Mounting /run/wrappers... worker2 # [ 12.692129] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 12.670894] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 12.674027] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 12.697625] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 12.652775] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 12.749839] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 12.770208] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 12.744447] systemd[1]: Mounted /run/wrappers. worker1 # [ 12.771219] systemd[1]: Finished Create List of Static Device Nodes. client # [ 12.752230] systemd[1]: Reached target Local File Systems. worker1 # [ 12.783396] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 12.766928] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 12.816218] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 12.772249] systemd[1]: Starting register-nix-paths.service... client # [ 12.791196] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 12.794091] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 12.848392] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 12.852509] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 12.855119] EXT4-fs (vda): re-mounted 8de9c3ca-d696-4c24-be69-648c18db6bb8. client # [ 12.834680] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 12.891527] systemd[1]: Finished Create System Files and Directories. worker1 # [ 12.871791] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 12.875558] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 12.879342] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 12.906563] systemd-udevd[321]: Using default interface naming scheme 'v261'. lb # [ 12.904649] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 12.894022] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 12.897089] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 12.875401] systemd[1]: Starting Create System Files and Directories... worker1 # [ 12.902183] systemd[1]: Finished Load Kernel Modules. lb # [ 12.928398] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 12.917193] systemd[1]: Starting Firewall... worker1 # [ 12.930180] fuse: init (API version 7.45) worker1 # [ 12.943019] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 12.958296] systemd-journald[287]: Collecting audit messages is enabled. worker1 # [ 12.989492] systemd[1]: modprobe@fuse.service: Deactivated successfully. worker1 # [ 12.992672] systemd[1]: Finished Load Kernel Module fuse. client # [ 12.991677] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 13.016403] systemd[1]: Started Journal Service. worker1 # [ 13.015569] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 13.018454] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 13.030180] systemd-modules-load[288]: Module 'atkbd' is built in worker1 # [ 13.041588] systemd-modules-load[288]: Module 'loop' is built in lb # [ 13.102888] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 13.069430] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 13.060099] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker2 # [ 13.109290] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 13.092151] systemd[1]: Mounting FUSE Control File System... client # [ 13.105484] systemd[1]: Finished Create System Files and Directories. worker1 # [ 13.120744] systemd-oomd[289]: No swap; memory pressure usage will be degraded lb # [ 13.175934] mousedev: PS/2 mouse device common for all mice client # [ 13.120377] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 13.134074] systemd[1]: Mounted FUSE Control File System. worker1 # [ 13.140369] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 13.130382] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 13.165152] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 13.172627] systemd[1]: Reached target First Boot Complete. lb # [ 13.224210] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 13.234137] systemd[1]: Starting Update is Completed... worker1 # [ 13.202887] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 13.220906] systemd-journald[287]: Received client request to flush runtime journal. worker2 # [ 13.329365] systemd[1]: Mounting /run/wrappers... client # [ 13.310241] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 13.309766] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 13.348469] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 13.345482] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 13.360530] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 13.359076] systemd[1]: Starting Update is Completed... client # [ 13.381128] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 13.427854] systemd[1]: Finished Update is Completed. worker2 # [ 13.444980] systemd[1]: Mounted /run/wrappers. worker2 # [ 13.446808] systemd[1]: Reached target Local File Systems. worker2 # [ 13.456539] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 13.462173] systemd[1]: Starting register-nix-paths.service... worker2 # [ 13.474265] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 13.428656] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 13.484266] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 13.492939] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 13.459209] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 13.461142] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 13.465835] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 13.549240] systemd[1]: Starting Create System Files and Directories... client # [ 13.512143] systemd[1]: Finished Update is Completed. worker2 # [ 13.720190] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 13.732199] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 13.792063] systemd-udevd[328]: Using default interface naming scheme 'v261'. worker2 # [ 13.943411] systemd[1]: Finished Create System Files and Directories. worker2 # [ 13.953920] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 13.969498] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 13.996203] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 14.014403] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 14.062980] systemd[1]: Mounting /run/wrappers... worker2 # [ 14.182147] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 14.177476] systemd[1]: Mounted /run/wrappers. worker1 # [ 14.180419] systemd[1]: Reached target Local File Systems. worker1 # [ 14.187856] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 14.278611] systemd[1]: Starting register-nix-paths.service... worker2 # [ 14.322681] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 14.292284] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 14.296723] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 14.304210] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 14.339571] systemd[1]: Starting Update is Completed... worker1 # [ 14.307539] systemd[1]: Starting Create System Files and Directories... worker2 # [ 14.545344] systemd[1]: Finished Update is Completed. lb # [ 14.572613] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 14.576457] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 14.576448] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 14.590141] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 14.580386] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 14.599587] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 14.742253] systemd[1]: Finished Create System Files and Directories. worker1 # [ 14.747675] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 14.758279] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 14.810471] systemd[1]: Finished Coldplug All udev Devices. client # [ 14.785079] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 14.933522] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 14.968309] systemd[1]: Finished Firewall. lb # [ 14.996182] systemd[1]: Finished register-nix-paths.service. lb # [ 14.997233] systemd[1]: Reached target System Initialization. lb # [ 15.000393] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 15.002284] systemd[1]: Started niks3 garbage collection timer. client # [ 14.974701] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 15.005085] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 15.007931] systemd[1]: Reached target Timer Units. lb # [ 15.010846] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 15.016188] systemd[1]: Listening on niks3 server socket. lb # [ 15.018798] systemd[1]: Listening on Nix Daemon Socket. lb # [ 15.020470] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 15.023072] systemd[1]: Reached target Socket Units. lb # [ 15.025201] systemd[1]: Reached target Basic System. lb # [ 15.030267] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 15.043153] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 15.057223] systemd[1]: Starting Post-Boot Actions... lb # [ 15.095309] systemd[1]: Started Reset console on configuration changes. worker2 # [ 15.110337] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 15.076849] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 15.111765] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 15.081543] systemd[1]: Starting Update is Completed... lb # [ 15.119537] systemd[1]: Starting resolvconf update... lb # [ 15.150583] systemd[1]: Started rustfs.service. lb # [ 15.198977] systemd[1]: Starting rustfs-bucket.service... lb # [ 15.210480] nsncd[507]: Sep 22 10:45:12.437 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 15.222570] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 15.238385] systemd[1]: Finished Post-Boot Actions. worker1 # [ 15.231736] systemd[1]: Finished Update is Completed. worker2 # [ 15.285936] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 15.295573] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 15.300206] systemd[1]: Starting Virtual Console Setup... lb # [ 15.302900] systemd[1]: Reached target User and Group Name Lookups. lb # [ 15.317688] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 15.362839] systemd[1]: Finished Firewall. lb # [ 15.370860] systemd[1]: Starting User Login Management... worker2 # [ 15.370876] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 15.378412] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 15.387262] systemd[1]: Starting Virtual Console Setup... lb # [ 15.425637] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 15.479818] systemd[1]: Finished register-nix-paths.service. client # [ 15.483417] systemd[1]: Reached target System Initialization. client # [ 15.487641] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 15.492573] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 15.496978] systemd[1]: Reached target Timer Units. client # [ 15.499554] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 15.502782] systemd[1]: Listening on Nix Daemon Socket. client # [ 15.508923] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 15.512328] systemd[1]: Reached target Socket Units. client # [ 15.516391] systemd[1]: Reached target Basic System. client # [ 15.520880] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 15.526578] systemd[1]: Starting nftables firewall... client # [ 15.541336] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 15.580688] systemd[1]: Finished Coldplug All udev Devices. client # [ 15.574434] systemd[1]: Starting Post-Boot Actions... lb # [ 15.654416] dbus-broker-launch[529]: Looking up NSS user entry for 'systemd-timesync'... client # [ 15.611229] systemd[1]: Started Reset console on configuration changes. lb # [ 15.665467] dbus-broker-launch[529]: NSS returned no entry for 'systemd-timesync' lb # [ 15.671922] dbus-broker-launch[529]: Invalid user-name in /nix/store/kxmx3smba321l9v2dpmh9zahhsbc77yh-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 15.641208] systemd[1]: Starting resolvconf update... lb # [ 15.689116] systemd-logind[530]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 15.698915] systemd-logind[530]: New seat seat0. worker2 # [ 15.713311] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 15.710751] systemd[1]: Started User Login Management. lb # [ 15.727111] systemd[1]: Starting linger-users.service... lb # [ 15.740748] systemd[1]: Started D-Bus System Message Bus. client # [ 15.698474] nsncd[405]: Sep 22 10:45:13.006 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 15.714230] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 15.734807] systemd[1]: Finished Post-Boot Actions. worker1 # [ 15.758195] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 15.799325] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 15.800478] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 15.813260] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 15.823448] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 15.829176] systemd[1]: Stopping User and Group Name Lookups... lb # [ 15.830147] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 15.831040] systemd[1]: nscd.service: Deactivated successfully. lb # [ 15.831867] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 15.857675] dbus-broker-launch[529]: Ready worker1 # [ 15.833969] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 15.823809] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 15.838223] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 15.890855] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 15.894949] systemd[1]: Finished linger-users.service. client # [ 15.852900] systemd[1]: Reached target Host and Network Name Lookups. client # [ 15.859186] systemd[1]: Reached target User and Group Name Lookups. client # [ 15.870123] systemd[1]: Starting D-Bus System Message Bus... client # [ 15.873305] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 15.933464] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 15.957644] (udev-worker)[355]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 15.917716] systemd[1]: Starting User Login Management... worker2 # [ 15.964690] (udev-worker)[355]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 15.978226] (udev-worker)[350]: Network interface NamePolicy= disabled on kernel command line. client # [ 15.955191] systemd[1]: Starting Virtual Console Setup... worker1 # [ 16.017574] systemd[1]: Finished Firewall. lb # [ 16.054635] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 16.067484] systemd-vconsole-setup[503]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 16.071936] nsncd[605]: Sep 22 10:45:13.284 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 16.079270] systemd[1]: Finished resolvconf update. worker2 # [ 16.079255] systemd[1]: Finished Virtual Console Setup. lb # [ 16.087089] systemd[1]: Reached target Preparation for Network. lb # [ 16.090016] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 16.095807] systemd[1]: Reached target User and Group Name Lookups. lb # [ 16.098247] systemd[1]: Starting DHCP Client... lb # [ 16.102381] systemd[1]: Starting Extra networking commands.... worker2 # [ 16.105288] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 16.109117] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 16.115047] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 16.120684] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 16.124429] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 16.132429] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 16.136621] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 16.141885] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 16.110288] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 16.178404] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 16.203356] dbus-broker-launch[431]: Looking up NSS user entry for 'systemd-timesync'... client # [ 16.227295] dbus-broker-launch[431]: NSS returned no entry for 'systemd-timesync' client # [ 16.235888] dbus-broker-launch[431]: Invalid user-name in /nix/store/ydh4ywywm30g15n5m6x5vfz42682s2cc-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 16.282824] systemd[1]: Started D-Bus System Message Bus. client # [ 16.299598] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 16.308945] systemd[1]: Stopping Host and Network Name Lookups... client # [ 16.320483] systemd[1]: Stopped target User and Group Name Lookups. client # [ 16.331478] systemd[1]: Stopping User and Group Name Lookups... client # [ 16.345379] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 16.351773] systemd[1]: nscd.service: Deactivated successfully. client # [ 16.361044] systemd-logind[433]: New seat seat0. client # [ 16.369584] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 16.381105] systemd[1]: Started User Login Management. client # [ 16.391846] dbus-broker-launch[431]: Ready client # [ 16.400418] systemd[1]: Starting linger-users.service... lb # [ 16.451313] dhcpcd[631]: dhcpcd-10.3.2 starting worker2 # [ 16.467015] systemd[1]: Finished register-nix-paths.service. client # [ 16.422919] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 16.477411] dhcpcd[665]: dev: loaded udev lb # [ 16.488497] dhcpcd[665]: no valid interfaces found lb # [ 16.492604] dhcpcd[665]: no valid interfaces found lb # [ 16.495669] dhcpcd[665]: libudev: received NULL device lb # [ 16.499991] dhcpcd[665]: libudev: received NULL device worker1 # [ 16.475159] (udev-worker)[355]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 16.497677] (udev-worker)[352]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 16.503812] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. client # [ 16.529578] systemd[1]: Finished resolvconf update. lb # [ 16.589480] systemd[1]: Finished Extra networking commands.. client # [ 16.550767] systemd[1]: Starting DHCP Client... lb # [ 16.595186] systemd[1]: Reached target Network. lb # [ 16.611958] systemd[1]: Starting PostgreSQL Server... client # [ 16.565820] systemd[1]: linger-users.service: Deactivated successfully. client # [ 16.569875] systemd[1]: Finished linger-users.service. lb # [ 16.623534] systemd[1]: Starting Permit User Sessions... client # [ 16.592656] nsncd[497]: Sep 22 10:45:13.899 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 16.605666] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 16.615067] systemd[1]: Reached target Host and Network Name Lookups. client # [ 16.622414] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 16.758160] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 16.761610] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 16.802434] systemd[1]: Finished Permit User Sessions. worker1 # [ 16.769210] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 16.778107] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 16.817981] systemd[1]: Started Getty on tty1. worker1 # [ 16.784220] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 16.826923] systemd[1]: Reached target Login Prompts. worker1 # [ 16.793617] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 16.798002] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 16.800860] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 16.869077] mousedev: PS/2 mouse device common for all mice worker2 # [ 16.954902] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 16.914496] dhcpcd[537]: dhcpcd-10.3.2 starting worker2 # [ 16.960469] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 16.963664] systemd[1]: Reached target System Initialization. worker2 # [ 16.966387] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 16.968246] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 16.934659] systemd[1]: Finished register-nix-paths.service. worker2 # [ 16.974199] systemd[1]: Reached target Timer Units. worker2 # [ 16.976485] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 16.932921] dhcpcd[547]: dev: loaded udev worker2 # [ 16.981805] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 16.987580] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 16.996408] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 16.956418] systemd[1]: Finished nftables firewall. client # [ 16.964725] systemd[1]: Reached target Preparation for Network. worker2 # [ 17.011649] systemd[1]: Reached target Socket Units. worker2 # [ 17.016412] systemd[1]: Reached target Basic System. client # [ 16.981542] systemd[1]: Starting Extra networking commands.... worker2 # [ 17.031477] systemd[1]: Started backdoor.service. client # [ 17.008620] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 17.036233] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 17.050479] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 17.011075] systemd-vconsole-setup[437]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 17.016856] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 17.065366] systemd[1]: Starting Post-Boot Actions... client # [ 17.025857] systemd[1]: Finished Virtual Console Setup. worker2 # [ 17.075525] systemd[1]: Started Reset console on configuration changes. worker2 # [ 17.086549] systemd[1]: Starting resolvconf update... worker2 # [ 17.090558] systemd[1]: Starting D-Bus System Message Bus... client # [ 17.066764] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 17.080937] systemd[1]: Started backdoor.service. worker1 # [ 17.140497] mousedev: PS/2 mouse device common for all mice lb # [ 17.173679] postgresql-pre-start[700]: The files belonging to this database system will be owned by user "postgres". lb # [ 17.178604] postgresql-pre-start[700]: This user must also own the server process. lb # [ 17.191223] postgresql-pre-start[700]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 17.194682] postgresql-pre-start[700]: The default database encoding has accordingly been set to "UTF8". lb # [ 17.199595] postgresql-pre-start[700]: The default text search configuration will be set to "english". lb # [ 17.204424] postgresql-pre-start[700]: Data page checksums are enabled. lb # [ 17.211060] postgresql-pre-start[700]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 17.215696] postgresql-pre-start[700]: creating subdirectories ... ok lb # [ 17.222873] postgresql-pre-start[700]: selecting dynamic shared memory implementation ... posix worker2 # [ 17.227027] systemd[1]: Finished Post-Boot Actions. client # [ 17.202157] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # connecting to host... client # [ 17.258387] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 17.286571] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 17.287647] systemd[1]: Reached target Host and Network Name Lookups. client # [ 17.258907] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 17.264667] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 17.265046] cfg80211: failed to load regulatory.db client # connecting to host... worker2 # [ 17.302684] systemd[1]: Reached target User and Group Name Lookups. client # [ 17.259008] dhcpcd[547]: no valid interfaces found client # [ 17.263174] dhcpcd[547]: no valid interfaces found worker2 # [ 17.309838] nsncd[585]: Sep 22 10:45:14.553 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 17.322666] systemd[1]: Starting User Login Management... client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 17.335680] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 17.361264] (udev-worker)[510]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 17.421879] dbus-broker-launch[589]: Looking up NSS user entry for 'systemd-timesync'... client # [ 17.378019] (udev-worker)[496]: Network interface NamePolicy= disabled on kernel command line. client # [ 17.382403] (udev-worker)[510]: Network interface NamePolicy= disabled on kernel command line. client: connected to guest root shell worker2 # [ 17.447833] dbus-broker-launch[589]: NSS returned no entry for 'systemd-timesync' client: (connecting took 18.54 seconds) client: (finished: waiting for the VM to finish booting, in 18.54 seconds) worker2 # [ 17.454629] dbus-broker-launch[589]: Invalid user-name in /nix/store/a9ijqx61hpssavy5d9lh27li51m8hnl7-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 17.504818] systemd[1]: Started D-Bus System Message Bus. client # [ 17.465662] systemd[1]: Finished Extra networking commands.. lb # [ 17.510594] postgresql-pre-start[700]: selecting default "max_connections" ... 100 client # [ 17.469212] systemd[1]: Reached target Network. client # [ 17.483138] systemd[1]: Starting Permit User Sessions... worker1 # [ 17.504674] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 17.508296] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 17.510207] systemd[1]: Reached target System Initialization. worker1 # [ 17.512339] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 17.514561] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 17.517873] systemd[1]: Reached target Timer Units. worker1 # [ 17.521997] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 17.529602] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 17.537762] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 17.543623] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 17.551370] systemd[1]: Reached target Socket Units. worker2 # [ 17.591980] dbus-broker-launch[589]: Ready worker1 # [ 17.560615] systemd[1]: Reached target Basic System. worker1 # [ 17.569850] systemd[1]: Started backdoor.service. worker2 # [ 17.602202] systemd-logind[610]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 17.606005] systemd-logind[610]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 17.607155] systemd-logind[610]: New seat seat0. worker1 # [ 17.576587] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 17.617169] systemd[1]: Started User Login Management. worker2 # [ 17.622160] systemd[1]: Starting linger-users.service... client # [ 17.579303] systemd[1]: Finished Permit User Sessions. worker1 # [ 17.597231] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 17.590957] systemd[1]: Started Getty on tty1. client # [ 17.591759] systemd[1]: Reached target Login Prompts. worker1 # [ 17.613533] systemd[1]: Starting Post-Boot Actions... worker1 # [ 17.625374] systemd[1]: Started Reset console on configuration changes. worker1 # [ 17.632589] systemd[1]: Starting resolvconf update... worker2 # [ 17.671679] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 17.677062] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 17.678181] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 17.679191] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 17.647536] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 17.680000] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 17.707522] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 17.718018] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 17.719312] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 17.760312] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 17.761457] systemd[1]: Finished linger-users.service. client # [ 17.724377] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 17.769930] postgresql-pre-start[700]: selecting default "shared_buffers" ... 128MB client # [ 17.732855] systemd[1]: Starting Address configuration of eth1... worker1 # [ 17.759679] systemd[1]: Finished Post-Boot Actions. worker2 # [ 17.815542] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 17.823827] nsncd[657]: Sep 22 10:45:15.083 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 17.841082] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 17.846652] systemd[1]: Reached target User and Group Name Lookups. worker1 # connecting to host... worker1 # [ 17.889585] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 17.891056] nsncd[583]: Sep 22 10:45:15.172 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 17.932262] systemd[1]: Finished resolvconf update. worker2 # [ 17.933642] systemd[1]: Reached target Preparation for Network. worker2 # [ 17.939190] systemd[1]: Starting DHCP Client... worker2 # [ 17.944280] systemd[1]: Starting Address configuration of eth1... worker1 # [ 17.911637] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 17.950361] systemd[1]: Starting Extra networking commands.... worker1 # [ 17.918979] systemd[1]: Reached target User and Group Name Lookups. client # [ 17.933769] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 17.929064] systemd[1]: Starting User Login Management... worker1 # [ 17.929910] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 17.962133] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 17.947159] dhcpcd[547]: eth0: waiting for carrier client # [ 17.948338] dhcpcd[547]: eth0: carrier acquired client # [ 17.962012] network-addresses-eth1-start[626]: adding address 192.168.1.1/24... done worker1 # [ 17.974596] dbus-broker-launch[587]: Looking up NSS user entry for 'systemd-timesync'... client # [ 17.973325] dhcpcd[547]: DUID 00:01:00:01:32:45:18:bb:52:54:00:12:34:56 client # [ 17.976391] dhcpcd[547]: eth0: IAID 00:12:34:56 client # [ 17.980536] systemd-logind[433]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 17.981731] dhcpcd[547]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 17.999305] dbus-broker-launch[587]: NSS returned no entry for 'systemd-timesync' client # [ 17.987299] network-addresses-eth1-start[626]: adding address 2001:db8:1::1/64... done worker1 # [ 18.005381] dbus-broker-launch[587]: Invalid user-name in /nix/store/a9ijqx61hpssavy5d9lh27li51m8hnl7-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 18.014441] systemd[1]: Finished Address configuration of eth1. worker1 # [ 18.046648] systemd[1]: Started D-Bus System Message Bus. client # [ 18.098730] mousedev: PS/2 mouse device common for all mice worker2 # [ 18.147278] network-addresses-eth1-start[685]: adding address 192.168.1.4/24... done worker1 # [ 18.120503] dbus-broker-launch[587]: Ready client # [ 18.109355] systemd-logind[433]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 18.136591] systemd-logind[608]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 18.145036] systemd-logind[608]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 18.146269] systemd-logind[608]: New seat seat0. worker2 # [ 18.185068] network-addresses-eth1-start[685]: adding address 2001:db8:1::4/64... done worker1 # [ 18.153784] systemd[1]: Started User Login Management. worker1 # [ 18.159720] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 18.176518] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 18.177422] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 18.178718] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 18.190124] systemd[1]: Starting linger-users.service... worker1 # [ 18.191055] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 18.191918] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 18.205022] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 18.206022] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 18.250307] systemd[1]: Finished Address configuration of eth1. worker2 # [ 18.279382] dhcpcd[693]: dhcpcd-10.3.2 starting lb # [ 18.295847] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 worker2 # [ 18.301582] dhcpcd[734]: dev: loaded udev worker1 # [ 18.295004] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 18.299144] systemd[1]: Finished linger-users.service. worker1 # [ 18.311165] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 18.314788] nsncd[654]: Sep 22 10:45:15.594 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 18.370175] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 18.370615] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 18.322526] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 18.326429] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 18.416692] systemd[1]: Finished Extra networking commands.. worker1 # [ 18.392776] systemd[1]: Finished resolvconf update. worker2 # [ 18.419832] systemd[1]: Reached target Network. worker2 # [ 18.428399] systemd[1]: Starting Permit User Sessions... worker1 # [ 18.393734] systemd[1]: Reached target Preparation for Network. worker1 # [ 18.402775] systemd[1]: Starting DHCP Client... worker1 # [ 18.408506] systemd[1]: Starting Address configuration of eth1... worker1 # [ 18.419352] systemd[1]: Starting Extra networking commands.... worker2 # [ 18.496452] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 18.535177] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 18.535720] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 18.541297] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 18.541745] cfg80211: failed to load regulatory.db worker2 # [ 18.543642] systemd[1]: Finished Permit User Sessions. worker2 # [ 18.549010] systemd[1]: Started Getty on tty1. worker2 # [ 18.553000] systemd[1]: Reached target Login Prompts. worker2 # [ 18.660843] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 18.643384] dhcpcd[734]: eth0: waiting for carrier worker2 # [ 18.646592] dhcpcd[734]: libudev: received NULL device worker2 # [ 18.648288] dhcpcd[734]: libudev: received NULL device worker2 # [ 18.650964] dhcpcd[734]: eth0: carrier acquired worker1 # [ 18.623703] network-addresses-eth1-start[683]: adding address 192.168.1.3/24... done worker2 # [ 18.669205] dhcpcd[734]: DUID 00:01:00:01:32:45:18:bb:52:54:00:12:34:56 worker2 # [ 18.670321] dhcpcd[734]: eth0: IAID 00:12:34:56 worker2 # [ 18.670974] dhcpcd[734]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 18.657624] network-addresses-eth1-start[683]: adding address 2001:db8:1::3/64... done worker1 # [ 18.701132] systemd[1]: Finished Address configuration of eth1. client: (finished: waiting for unit network-addresses-eth1.service, in 19.87 seconds) worker1 # [ 18.752832] dhcpcd[691]: dhcpcd-10.3.2 starting client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } worker1 # [ 18.773511] dhcpcd[735]: dev: loaded udev client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.06 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting worker1 # [ 18.831516] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 18.831956] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 18.863415] systemd[1]: Finished Extra networking commands.. worker1 # [ 18.870999] systemd[1]: Reached target Network. worker1 # [ 18.877888] systemd[1]: Starting Permit User Sessions... client # [ 18.872548] dhcpcd[547]: eth0: soliciting a DHCP lease client # [ 18.880785] dhcpcd[547]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 18.888333] dhcpcd[547]: eth0: probing address 10.0.2.15/24 worker1 # [ 18.945197] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 18.974039] systemd[1]: Starting Virtual Console Setup... worker1 # [ 18.984193] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 18.984824] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 18.990489] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 19.006855] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 18.990989] cfg80211: failed to load regulatory.db lb # [ 19.022213] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 18.994483] systemd[1]: Finished Permit User Sessions. lb # [ 19.030749] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 18.998711] systemd[1]: Started Getty on tty1. worker1 # [ 18.999971] systemd[1]: Reached target Login Prompts. lb # [ 19.042683] systemd[1]: Starting Virtual Console Setup... worker1 # [ 19.126661] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 19.115866] dhcpcd[735]: eth0: waiting for carrier worker1 # [ 19.117691] dhcpcd[735]: eth0: carrier acquired worker1 # [ 19.140175] dhcpcd[735]: DUID 00:01:00:01:32:45:18:bc:52:54:00:12:34:56 worker1 # [ 19.141252] dhcpcd[735]: eth0: IAID 00:12:34:56 worker1 # [ 19.141940] dhcpcd[735]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 19.446528] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 19.525248] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 19.518730] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 19.588704] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 19.602315] systemd[1]: Started backdoor.service. client # [ 19.584552] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 19.584717] systemd[1]: Stopped Virtual Console Setup. client # [ 19.584771] systemd[1]: Stopping Virtual Console Setup... client # [ 19.592557] systemd[1]: Starting Virtual Console Setup... client # [ 19.625194] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 19.632122] systemd[1]: Stopped Virtual Console Setup. client # [ 19.638803] systemd[1]: Starting Virtual Console Setup... lb # [ 19.721911] 8021q: 802.1Q VLAN Support v1.8 client # [ 19.673166] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 19.698681] systemd-logind[433]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 19.761552] systemd-vconsole-setup[730]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # connecting to host... lb # [ 19.776634] systemd[1]: Finished Virtual Console Setup. client # [ 19.756777] dhcpcd[547]: eth0: soliciting an IPv6 router worker2 # [ 19.803122] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 19.758406] dhcpcd[547]: eth0: Router Advertisement from fe80::2 client # [ 19.759391] dhcpcd[547]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 19.760440] dhcpcd[547]: eth0: adding route to fec0::/64 client # [ 19.761330] dhcpcd[547]: eth0: adding default route via fe80::2 worker2 # [ 19.807968] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 19.810564] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 19.815640] systemd[1]: Starting Virtual Console Setup... worker2 # [ 19.832333] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.99 seconds) lb: (finished: waiting for the VM to finish booting, in 0.99 seconds) worker2 # [ 19.855715] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 19.858198] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 19.862672] systemd[1]: Starting Virtual Console Setup... worker2 # [ 19.915536] systemd-logind[610]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 19.933244] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 20.015094] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 20.015682] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 20.022140] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 20.022561] cfg80211: failed to load regulatory.db worker1 # [ 20.007704] systemd[1]: Starting Virtual Console Setup... worker1 # [ 20.039100] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 20.041437] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 20.043990] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 20.050721] systemd[1]: Starting Virtual Console Setup... client # [ 20.043839] systemd-vconsole-setup[670]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 20.048671] systemd[1]: Finished Virtual Console Setup. lb # [ 20.165788] (udev-worker)[721]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 20.167979] (udev-worker)[721]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 20.144362] systemd-logind[608]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 20.179600] (udev-worker)[724]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 20.269587] systemd-vconsole-setup[784]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 20.276404] systemd[1]: Finished Virtual Console Setup. worker2 # [ 20.365799] dhcpcd[734]: eth0: soliciting a DHCP lease worker2 # [ 20.368985] dhcpcd[734]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 20.376472] dhcpcd[734]: eth0: probing address 10.0.2.15/24 lb # [ 20.426745] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 20.439785] systemd[1]: Starting Address configuration of eth1... lb # [ 20.510342] systemd-logind[530]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 20.587677] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 20.538220] systemd-vconsole-setup[782]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 20.574189] dhcpcd[665]: eth0: waiting for carrier lb # [ 20.577452] dhcpcd[665]: eth0: carrier acquired worker1 # [ 20.543772] systemd[1]: Finished Virtual Console Setup. lb # [ 20.613987] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 20.600436] dhcpcd[665]: DUID 00:01:00:01:32:45:18:bd:52:54:00:12:34:56 lb # [ 20.601457] dhcpcd[665]: eth0: IAID 00:12:34:56 lb # [ 20.602090] dhcpcd[665]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 20.621872] network-addresses-eth1-start[759]: adding address 192.168.1.2/24... done lb # [ 20.641684] network-addresses-eth1-start[759]: adding address 2001:db8:1::2/64... done lb # [ 20.670138] systemd[1]: Finished Address configuration of eth1. lb # [ 20.865184] systemd-logind[530]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 21.076453] dhcpcd[734]: eth0: soliciting an IPv6 router worker2 # [ 21.079889] dhcpcd[734]: eth0: Router Advertisement from fe80::2 worker2 # [ 21.083654] dhcpcd[734]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 21.086776] dhcpcd[734]: eth0: adding route to fec0::/64 worker2 # [ 21.089142] dhcpcd[734]: eth0: adding default route via fe80::2 worker1 # [ 21.068750] dhcpcd[735]: eth0: soliciting a DHCP lease worker1 # [ 21.077648] dhcpcd[735]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 21.084724] dhcpcd[735]: eth0: probing address 10.0.2.15/24 lb: (finished: waiting for unit network-addresses-eth1.service, in 2.41 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker1 # [ 21.242243] dhcpcd[735]: eth0: soliciting an IPv6 router worker1 # [ 21.247927] dhcpcd[735]: eth0: Router Advertisement from fe80::2 worker1 # [ 21.251521] dhcpcd[735]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 21.255054] dhcpcd[735]: eth0: adding route to fec0::/64 worker1 # [ 21.258917] dhcpcd[735]: eth0: adding default route via fe80::2 lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.13 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 21.449653] postgresql-pre-start[700]: selecting default time zone ... UTC lb # [ 21.463913] postgresql-pre-start[700]: creating configuration files ... ok worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.15 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.08 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.23 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.14 seconds) lb: waiting for unit niks3.service lb # [ 22.313660] dhcpcd[665]: eth0: soliciting an IPv6 router lb # [ 22.322177] dhcpcd[665]: eth0: Router Advertisement from fe80::2 lb # [ 22.335937] dhcpcd[665]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 22.338326] dhcpcd[665]: eth0: adding route to fec0::/64 lb # [ 22.382573] dhcpcd[665]: eth0: adding default route via fe80::2 lb # [ 22.421482] dhcpcd[665]: eth0: soliciting a DHCP lease lb # [ 22.460489] dhcpcd[665]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 22.462959] dhcpcd[665]: eth0: probing address 10.0.2.15/24 lb # [ 23.498980] postgresql-pre-start[700]: running bootstrap script ... ok client # [ 23.934773] dhcpcd[547]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 23.936326] dhcpcd[547]: eth0: adding route to 10.0.2.0/24 client # [ 23.937099] dhcpcd[547]: eth0: adding default route via 10.0.2.2 client # [ 24.160255] systemd[1]: Started DHCP Client. client # [ 24.162400] systemd[1]: Reached target Multi-User System. client # [ 24.163577] systemd[1]: Startup finished in 1.286s (kernel) + 7.945s (initrd) + 14.931s (userspace) = 24.163s. worker2 # [ 25.185812] dhcpcd[734]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 25.187602] dhcpcd[734]: eth0: adding route to 10.0.2.0/24 worker2 # [ 25.189496] dhcpcd[734]: eth0: adding default route via 10.0.2.2 worker2 # [ 25.318472] systemd[1]: Started DHCP Client. worker2 # [ 25.321066] systemd[1]: Reached target Multi-User System. worker2 # [ 25.322400] systemd[1]: Startup finished in 1.332s (kernel) + 7.829s (initrd) + 16.158s (userspace) = 25.320s. lb # [ 25.810644] postgresql-pre-start[700]: performing post-bootstrap initialization ... ok lb # [ 25.865168] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 25.871595] systemd[1]: Stopped Virtual Console Setup. lb # [ 25.876329] systemd[1]: Stopping Virtual Console Setup... lb # [ 25.886073] systemd[1]: Starting Virtual Console Setup... lb # [ 25.952674] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 25.970309] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 25.973681] systemd[1]: Stopped Virtual Console Setup. lb # [ 25.983868] systemd-logind[530]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 25.997327] systemd[1]: Starting Virtual Console Setup... lb # [ 26.097511] postgresql-pre-start[700]: syncing data to disk ... ok lb # [ 26.099953] postgresql-pre-start[700]: initdb: warning: enabling "trust" authentication for local connections lb # [ 26.101693] postgresql-pre-start[700]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 26.103945] postgresql-pre-start[700]: Success. You can now start the database server using: lb # [ 26.105351] postgresql-pre-start[700]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 26.304352] postgres[857]: [857] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 26.308499] postgres[857]: [857] LOG: listening on IPv6 address "::1", port 5432 lb # [ 26.309677] postgres[857]: [857] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 26.311967] postgres[857]: [857] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 26.333101] postgres[866]: [866] LOG: database system was shut down at 2026-09-22 10:45:22 GMT lb # [ 26.346909] postgres[857]: [857] LOG: database system is ready to accept connections lb # [ 26.352906] systemd[1]: Started PostgreSQL Server. lb # [ 26.366094] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 26.471538] systemd-vconsole-setup[852]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 26.476253] systemd[1]: Finished Virtual Console Setup. worker1 # [ 26.461955] dhcpcd[735]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 26.467738] dhcpcd[735]: eth0: adding route to 10.0.2.0/24 worker1 # [ 26.471815] dhcpcd[735]: eth0: adding default route via 10.0.2.2 lb # [ 26.677156] postgresql-setup-start[878]: CREATE DATABASE worker1 # [ 26.647206] systemd[1]: Started DHCP Client. worker1 # [ 26.650217] systemd[1]: Reached target Multi-User System. worker1 # [ 26.651586] systemd[1]: Startup finished in 1.308s (kernel) + 8.315s (initrd) + 17.025s (userspace) = 26.649s. lb # [ 26.796474] postgresql-setup-start[883]: CREATE ROLE lb # [ 26.852745] postgresql-setup-start[889]: ALTER DATABASE lb # [ 26.870077] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 26.872346] systemd[1]: Reached target PostgreSQL. lb # [ 27.790450] dhcpcd[665]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 27.793730] dhcpcd[665]: eth0: adding route to 10.0.2.0/24 lb # [ 27.795077] dhcpcd[665]: eth0: adding default route via 10.0.2.2 lb # [ 28.230986] systemd[1]: Started DHCP Client. lb # [ 28.234655] systemd[1]: Reached target Network is Online. lb # [ 28.257400] systemd[1]: Started Envoy reverse proxy. lb # [ 28.267550] systemd[1]: Started mock-oidc.service. lb # [ 28.488151] mock-oidc-server[962]: Mock OIDC Server running lb # [ 28.490078] mock-oidc-server[962]: OIDC Address: 192.168.1.2:8080 lb # [ 28.491296] mock-oidc-server[962]: Issue Address: 0.0.0.0:8081 lb # [ 28.494028] mock-oidc-server[962]: Issuer: http://192.168.1.2:8080/oidc lb # [ 28.497122] mock-oidc-server[962]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 28.499229] mock-oidc-server[962]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 28.501548] mock-oidc-server[962]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 30.128700] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 30.128901] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 30.128943] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 30.128977] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 30.129011] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 30.129043] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 30.129088] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 30.129123] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 30.129157] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 30.129275] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 30.225300] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 30.245435] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 30.249665] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 30.254499] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 30.257441] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 30.265875] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 30.271065] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 30.274377] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 30.275918] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 30.278473] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 30.280789] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 30.284052] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 30.286830] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 30.295028] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 30.297588] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 30.302292] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 30.305841] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 30.310291] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 30.312738] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 30.315829] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 30.318183] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 30.320716] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 30.327901] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 30.332862] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 30.339921] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 30.343731] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 30.345957] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 30.348263] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 30.351068] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 30.355081] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 30.359397] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 30.362117] envoy[961]: [2026-09-22 10:45:27.354][961][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 30.364712] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 30.366825] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 30.368872] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 30.371127] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 30.375289] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 30.377906] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 30.384716] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 30.390727] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 30.394689] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 30.397215] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 30.400202] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 30.418854] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 30.428585] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 30.431501] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 30.433596] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 30.436004] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 30.445367] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 30.474332] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 30.521451] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 31.050352] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 31.679010] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 31.709259] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 31.776093] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 31.780166] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 31.798906] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 31.910344] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 31.925334] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 32.006292] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 32.027857] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 32.070638] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 32.089006] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 32.091094] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 32.175413] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 32.228672] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 32.240709] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 32.249784] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 32.292248] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 33.167418] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 35.757060] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 35.777444] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 35.780786] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 35.788643] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 35.790961] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 35.801901] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 35.807482] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 35.811501] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 35.815642] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 35.829170] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 35.831137] envoy[961]: [2026-09-22 10:45:27.355][961][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 35.833058] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 35.837112] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 35.840889] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 35.844006] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 35.858018] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 35.863137] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 35.867225] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 35.870353] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 35.878424] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 35.885441] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 35.887762] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 35.890739] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 35.893642] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 35.897146] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 35.910521] envoy[961]: [2026-09-22 10:45:27.404][961][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 35.921484] envoy[961]: [2026-09-22 10:45:27.472][961][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 35.923509] envoy[961]: [2026-09-22 10:45:27.492][961][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 36.018312] envoy[961]: [2026-09-22 10:45:27.492][961][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 36.023159] envoy[961]: [2026-09-22 10:45:27.492][961][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 36.050926] envoy[961]: [2026-09-22 10:45:27.492][961][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 38.395995] envoy[961]: [2026-09-22 10:45:35.572][961][info][main] [source/server/server.cc:968] runtime: {} lb # [ 38.768580] envoy[961]: [2026-09-22 10:45:35.988][961][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 38.834020] envoy[961]: [2026-09-22 10:45:36.016][961][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 39.494184] envoy[961]: [2026-09-22 10:45:36.030][961][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 41.423832] envoy[961]: [2026-09-22 10:45:36.178][961][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 41.435545] envoy[961]: [2026-09-22 10:45:36.184][961][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 42.252724] envoy[961]: [2026-09-22 10:45:36.184][961][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) worker1 # [ 52.628943] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 52.665730] envoy[961]: [2026-09-22 10:45:49.892][961][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker2 # [ 52.672747] systemd[1]: Reached target Network is Online. worker2 # [ 52.682461] systemd[1]: Starting scheduler-token.service... lb # [ 52.694053] envoy[961]: [2026-09-22 10:45:49.921][961][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 52.700677] envoy[961]: [2026-09-22 10:45:49.928][961][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 52.709320] envoy[961]: [2026-09-22 10:45:49.932][961][info][main] [source/server/server.cc:1077] starting main dispatch loop worker2 # [ 52.781627] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 52.787151] systemd[1]: Finished scheduler-token.service. worker2 # [ 52.796582] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 101ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 52.798820] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 52.850749] rustfs-bucket-start[999]: mb s3://farm lb # [ 52.862404] systemd[1]: Finished rustfs-bucket.service. lb # [ 53.122609] systemd[1]: Starting niks3 server... worker1 # [ 53.111181] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 53.126858] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 53.129617] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 53.173071] nix-grpc-daemon[860]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 53.209316] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 53.175092] nix-grpc-daemon[860]: E0000 00:00:1790073950.456102 860 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 53.222926] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 53.190265] nix-grpc-daemon[860]: E0000 00:00:1790073950.473387 860 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 53.225649] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 53.195938] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 53.201556] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 53.264353] nix-grpc-daemon[867]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 53.266055] nix-grpc-daemon[867]: E0000 00:00:1790073950.531167 867 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 53.267063] envoy[961]: [2026-09-22 10:45:50.494][961][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 53.269530] envoy[961]: [2026-09-22 10:45:50.496][961][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized worker1 # [ 53.236122] nix-grpc-daemon[860]: E0000 00:00:1790073950.518595 903 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 53.271910] envoy[961]: [2026-09-22 10:45:50.499][961][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager worker2 # [ 53.281386] nix-grpc-daemon[867]: E0000 00:00:1790073950.549355 867 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 53.279893] envoy[961]: [2026-09-22 10:45:50.507][961][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 53.286320] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 53.291390] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 53.257294] nix-grpc-daemon[860]: E0000 00:00:1790073950.540328 903 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 53.264098] nix-grpc-daemon[860]: E0000 00:00:1790073950.543730 903 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 53.282973] nix-grpc-daemon[860]: E0000 00:00:1790073950.564665 903 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 53.325984] nix-grpc-daemon[867]: E0000 00:00:1790073950.593153 908 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 53.295380] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 53.298465] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 53.347844] nix-grpc-daemon[867]: E0000 00:00:1790073950.615672 908 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 53.352084] nix-grpc-daemon[867]: E0000 00:00:1790073950.619214 908 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 53.374788] nix-grpc-daemon[867]: E0000 00:00:1790073950.641646 908 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 53.386036] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 53.388180] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 53.429702] postgres[1018]: [1018] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 53.431674] postgres[1018]: [1018] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 53.473132] niks3-server[1010]: 2026/09/22 10:45:50 OK 20241026095416_initial_model.sql (26.54ms) lb # [ 53.479325] niks3-server[1010]: 2026/09/22 10:45:50 OK 20251210153512_drop_unused_gin_index.sql (5.47ms) lb # [ 53.484635] niks3-server[1010]: 2026/09/22 10:45:50 OK 20251218171726_add_pins.sql (7.34ms) lb # [ 53.491446] niks3-server[1010]: 2026/09/22 10:45:50 OK 20260628120000_add_object_size_and_stats.sql (6.75ms) lb # [ 53.498956] niks3-server[1010]: 2026/09/22 10:45:50 OK 20260905000000_add_claims.sql (7.45ms) lb # [ 53.506618] niks3-server[1010]: 2026/09/22 10:45:50 OK 20260920000000_drop_claims.sql (7.46ms) lb # [ 53.509658] niks3-server[1010]: 2026/09/22 10:45:50 goose: successfully migrated database to version: 20260920000000 lb # [ 53.515816] niks3-server[1010]: 2026/09/22 10:45:50 OK 1_commit_pending_closure.sql (9.18ms) lb # [ 53.519954] niks3-server[1010]: 2026/09/22 10:45:50 OK 2_object_stats_trigger.sql (3.94ms) lb # [ 53.521896] niks3-server[1010]: 2026/09/22 10:45:50 goose: up to current file version: 2 lb # [ 53.526663] niks3-server[1010]: 2026/09/22 10:45:50 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key worker1 # [ 53.500598] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 53.506299] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 53.584850] niks3-server[1010]: 2026/09/22 10:45:50 INFO Read proxy enabled — serving cache objects from S3 lb # [ 53.587564] niks3-server[1010]: 2026/09/22 10:45:50 INFO Using socket-activated listener address=0.0.0.0:5751 worker2 # [ 53.590931] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=scheduler_connected addr=lb:50051 lb # [ 53.596419] niks3-server[1010]: 2026/09/22 10:45:50 INFO systemd watchdog enabled interval=15s lb # [ 53.598435] systemd[1]: Started niks3 server. lb # [ 53.599303] systemd[1]: Reached target Multi-User System. worker2 # [ 53.597571] nix-grpc-daemon[867]: ts=2026-09-22T10:45:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 53.602069] systemd[1]: Startup finished in 1.376s (kernel) + 6.729s (initrd) + 45.489s (userspace) = 53.595s. lb # [ 53.605286] niks3-server[1010]: 2026/09/22 10:45:50 INFO Starting HTTP server address=0.0.0.0:5751 worker1 # [ 53.714354] nix-grpc-daemon[860]: ts=2026-09-22T10:45:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 53.799070] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 53.804721] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 31.91 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service worker2 # [ 54.005887] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 54.011447] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit envoy.service, in 0.07 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 54.092088] envoy[961]: conn peer=[::1]:44944 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds) worker1 # [ 54.117652] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.121623] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 54.230805] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 54.238857] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 54.320305] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.327093] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 54.463253] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 54.537832] nix-grpc-daemon[867]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 54.529252] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.555413] nix-grpc-daemon[860]: ts=2026-09-22T10:45:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 54.622925] systemd[1]: Starting scheduler-token.service... worker2 # [ 54.751875] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.757578] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.770187] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 54.806920] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 55.009593] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.982626] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 54.993904] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 55.041838] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 55.116293] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 55.118358] systemd[1]: Finished scheduler-token.service. worker2 # [ 55.119818] systemd[1]: scheduler-token.service: Consumed 217ms CPU time over 539ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 55.231436] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 55.233305] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 55.197027] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.11 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 55.202169] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 55.432781] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 55.398212] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 55.438321] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 55.403704] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 55.605602] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 55.612658] nix-grpc-daemon[860]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 55.650477] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 55.661222] nix-grpc-daemon[867]: ts=2026-09-22T10:45:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 55.812804] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 55.815256] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 55.859800] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 55.866326] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 56.067310] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 56.030868] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 56.040180] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 56.077323] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 56.242517] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 56.279530] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 56.247504] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 56.287485] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 56.448380] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 56.451932] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 56.488533] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 56.495814] nix-grpc-daemon[867]: ts=2026-09-22T10:45:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 56.691225] nix-grpc-daemon[860]: ts=2026-09-22T10:45:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 56.737191] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 57.110564] nix-grpc-daemon[860]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 57.147208] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 57.373969] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 57.355866] nix-grpc-daemon[860]: ts=2026-09-22T10:45:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 57.427954] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 57.474375] nix-grpc-daemon[860]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 57.634950] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 57.652368] nix-grpc-daemon[867]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 57.689674] nix-grpc-daemon[860]: ts=2026-09-22T10:45:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 57.714887] nix-grpc-daemon[860]: ts=2026-09-22T10:45:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 57.917017] nix-grpc-daemon[860]: ts=2026-09-22T10:45:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 57.946864] nix-grpc-daemon[867]: ts=2026-09-22T10:45:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 57.959802] nix-grpc-daemon[860]: ts=2026-09-22T10:45:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 58.004947] nix-grpc-daemon[867]: ts=2026-09-22T10:45:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 58.231285] nix-grpc-daemon[867]: ts=2026-09-22T10:45:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 58.293436] nix-grpc-daemon[860]: ts=2026-09-22T10:45:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 58.791469] nix-grpc-daemon[867]: ts=2026-09-22T10:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 59.025073] nix-grpc-daemon[867]: ts=2026-09-22T10:45:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 58.882549] nix-grpc-daemon[860]: ts=2026-09-22T10:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 59.249212] nix-grpc-daemon[860]: ts=2026-09-22T10:45:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 59.307032] nix-grpc-daemon[867]: ts=2026-09-22T10:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 59.470846] nix-grpc-daemon[860]: ts=2026-09-22T10:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 59.549141] nix-grpc-daemon[867]: ts=2026-09-22T10:45:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 59.606919] nix-grpc-daemon[867]: ts=2026-09-22T10:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 59.718542] nix-grpc-daemon[860]: ts=2026-09-22T10:45:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 59.806621] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 59.868966] nix-grpc-daemon[867]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 59.959446] nix-grpc-daemon[867]: ts=2026-09-22T10:45:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 60.025795] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 2.70 seconds) worker1 # [ 60.046704] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 60.163680] nix-grpc-daemon[867]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 60.349568] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 60.468712] nix-grpc-daemon[867]: ts=2026-09-22T10:45:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 60.450508] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 60.667275] nix-grpc-daemon[867]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 60.698753] nix-grpc-daemon[860]: ts=2026-09-22T10:45:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 60.754071] nix-grpc-daemon[867]: ts=2026-09-22T10:45:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 60.990453] nix-grpc-daemon[867]: ts=2026-09-22T10:45:58Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 61.182085] nix-grpc-daemon[867]: ts=2026-09-22T10:45:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 61.129893] nix-grpc-daemon[860]: ts=2026-09-22T10:45:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 61.489396] nix-grpc-daemon[860]: ts=2026-09-22T10:45:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 61.555390] nix-grpc-daemon[867]: ts=2026-09-22T10:45:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 61.737743] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 61.819407] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 61.946495] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 62.020504] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 62.107933] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 62.159487] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 62.280695] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 62.282287] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 62.345824] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 62.348517] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 1.42 seconds) worker1 # [ 62.506397] nix-grpc-daemon[860]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 62.582107] nix-grpc-daemon[867]: ts=2026-09-22T10:45:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 62.835743] nix-grpc-daemon[867]: ts=2026-09-22T10:46:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 62.854489] nix-grpc-daemon[860]: ts=2026-09-22T10:46:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 63.069972] nix-grpc-daemon[867]: ts=2026-09-22T10:46:00Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 63.918113] nix-grpc-daemon[860]: ts=2026-09-22T10:46:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 67.871500] nix-grpc-daemon[860]: ts=2026-09-22T10:46:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 67.925855] nix-grpc-daemon[867]: ts=2026-09-22T10:46:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 68.151964] nix-grpc-daemon[867]: ts=2026-09-22T10:46:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 68.137372] nix-grpc-daemon[860]: ts=2026-09-22T10:46:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 68.582435] nix-grpc-daemon[867]: ts=2026-09-22T10:46:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 68.697769] nix-grpc-daemon[860]: ts=2026-09-22T10:46:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 68.824877] nix-grpc-daemon[867]: ts=2026-09-22T10:46:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 68.960775] nix-grpc-daemon[860]: ts=2026-09-22T10:46:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 69.298789] nix-grpc-daemon[867]: ts=2026-09-22T10:46:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 69.333830] nix-grpc-daemon[860]: ts=2026-09-22T10:46:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 69.543060] nix-grpc-daemon[867]: ts=2026-09-22T10:46:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 69.715833] nix-grpc-daemon[860]: ts=2026-09-22T10:46:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 70.018156] nix-grpc-daemon[867]: ts=2026-09-22T10:46:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 70.543365] nix-grpc-daemon[867]: ts=2026-09-22T10:46:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 78.867644] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 78.906575] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 78.915508] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_take_over lb # [ 78.911812] niks3-server[1010]: 2026/09/22 10:46:16 INFO lead: acquired remote=192.168.1.4:48870 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 15.43 seconds) worker1 # [ 79.065093] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 79.071402] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.107079] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.113255] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 79.272977] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 79.278979] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.314950] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.321402] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 79.479619] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 79.483089] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.522726] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.527892] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 79.684563] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.724821] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.727752] nix-grpc-daemon[867]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 79.690406] nix-grpc-daemon[860]: ts=2026-09-22T10:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 79.892895] nix-grpc-daemon[860]: ts=2026-09-22T10:46:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.930894] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.941382] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:50082 worker2 # [ 79.948693] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 79.966144] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 79.971106] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:50082 worker2 # [ 79.976926] nix-grpc-daemon[867]: ts=2026-09-22T10:46:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 25.75 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 81.408909] nix-grpc-daemon[867]: ts=2026-09-22T10:46:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 81.423152] systemd[1]: Started Nix Daemon. worker2 # [ 81.977156] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 82.021318] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5) worker2 # [ 82.064351] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 185 ms (attempt 2/5) worker2 # [ 82.254476] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 3/5) worker2 # [ 82.306860] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 4/5) worker2 # [ 82.595503] nix-grpc-daemon[867]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 82.599221] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 82.653888] nix-grpc-daemon[860]: ts=2026-09-22T10:46:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 82.665696] systemd[1]: Started Nix Daemon. worker1 # [ 83.304543] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 83.337088] nix-grpc-daemon[860]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5) worker1 # [ 83.344938] nix-grpc-daemon[860]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5) worker1 # [ 83.503276] nix-grpc-daemon[860]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 3/5) worker1 # [ 83.642547] nix-grpc-daemon[860]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 671 ms (attempt 4/5) worker1 # [ 84.330370] nix-grpc-daemon[860]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 84.337615] nix-grpc-daemon[860]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 84.360139] nix-grpc-daemon[860]: ts=2026-09-22T10:46:21Z level=debug event=path_info_miss path=7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test error="error: path '/nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 84.418013] nix-grpc-daemon[867]: ts=2026-09-22T10:46:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 84.430191] nix-grpc-daemon[867]: ts=2026-09-22T10:46:21Z level=debug event=path_info_miss path=7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test error="error: path '/nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds) client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds) worker1 # [ 84.846764] nix-grpc-daemon[860]: ts=2026-09-22T10:46:22Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 84.853834] nix-grpc-daemon[860]: ts=2026-09-22T10:46:22Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34910 worker2 # [ 84.914155] nix-grpc-daemon[867]: ts=2026-09-22T10:46:22Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40376 worker2 # [ 84.927602] nix-grpc-daemon[867]: ts=2026-09-22T10:46:22Z level=debug event=path_info_miss path=7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test error="error: path '/nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 84.945998] nix-grpc-daemon[867]: ts=2026-09-22T10:46:22Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40376 worker2 # [ 84.955823] nix-grpc-daemon[867]: ts=2026-09-22T10:46:22Z level=debug event=path_info_miss path=7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test error="error: path '/nix/store/7k0a7n8x1zd6x0xsp0lh395b7bhivpl8-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds) worker2 # [ 85.376007] nix-grpc-daemon[867]: ts=2026-09-22T10:46:22Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 85.769981] nix-grpc-daemon[867]: ts=2026-09-22T10:46:23Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.82 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2 worker2 # [ 86.116822] nix-grpc-daemon[867]: ts=2026-09-22T10:46:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 86.133115] nix-grpc-daemon[867]: ts=2026-09-22T10:46:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 86.185953] nix-grpc-daemon[867]: ts=2026-09-22T10:46:23Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker2 # [ 86.193024] nix-grpc-daemon[867]: ts=2026-09-22T10:46:23Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 86.214097] nix-grpc-daemon[860]: ts=2026-09-22T10:46:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 86.240517] nix-grpc-daemon[860]: ts=2026-09-22T10:46:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying 1 paths to 'lb:50051'... worker1 # [ 86.260993] nix-grpc-daemon[860]: ts=2026-09-22T10:46:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 86.291926] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 86.342850] nix-grpc-daemon[860]: ts=2026-09-22T10:46:23Z level=info event=niks3_push_started pid=933 worker1 # [ 86.754313] nix-grpc-daemon[933]: time=2026-09-22T10:46:24.035Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 87.057953] nix-daemon[921]: accepted connection from pid 946, user nix-grpc-daemon (trusted) worker1 # [ 87.088282] nix-daemon[921]: reaped child process 949, status = succeeded worker1 # [ 87.359296] nix-daemon[921]: accepted connection from pid 951, user nix-grpc-daemon (trusted) lb # [ 87.438837] niks3-server[1010]: 2026/09/22 10:46:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 87.408533] nix-daemon[921]: reaped child process 954, status = succeeded worker1 # [ 87.449901] nix-grpc-daemon[933]: time=2026-09-22T10:46:24.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 87.469091] nix-grpc-daemon[933]: time=2026-09-22T10:46:24.737Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 87.609825] niks3-server[1010]: 2026/09/22 10:46:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 87.628462] niks3-server[1010]: 2026/09/22 10:46:24 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 87.666794] niks3-server[1010]: 2026/09/22 10:46:24 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 87.683619] niks3-server[1010]: 2026/09/22 10:46:24 INFO Signed narinfos id=1 count=1 worker1 # [ 87.661180] nix-grpc-daemon[933]: time=2026-09-22T10:46:24.942Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.715531] niks3-server[1010]: 2026/09/22 10:46:24 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 87.731266] niks3-server[1010]: 2026/09/22 10:46:24 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo lb # [ 87.752695] niks3-server[1010]: 2026/09/22 10:46:24 INFO Completed upload id=1 worker1 # [ 87.734922] nix-grpc-daemon[933]: time=2026-09-22T10:46:25.018Z level=INFO msg="Upload complete. (933ms)" worker1 # [ 87.765369] nix-grpc-daemon[860]: ts=2026-09-22T10:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 paths=1 nar_bytes_in=504 worker1 # [ 87.781933] nix-grpc-daemon[860]: ts=2026-09-22T10:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 87.801135] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 87.804719] nix-daemon[921]: reaped child process 931, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 87.823380] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 88.910407] envoy[961]: conn peer=[2001:db8:1::1]:50856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=3075 lb # [ 88.917302] envoy[961]: conn peer=[2001:db8:1::1]:50872 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=388 ms=116 lb # [ 88.922818] envoy[961]: conn peer=[2001:db8:1::1]:50888 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=22 lb # [ 88.929011] envoy[961]: conn peer=[2001:db8:1::1]:50892 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=182 ms=26 client # farm-a> LOG-farm-a-t1 worker1 # [ 89.102084] nix-daemon[921]: reaped child process 958, status = succeeded worker1 # [ 89.293242] nix-daemon[921]: accepted connection from pid 974, user nix-grpc-daemon (trusted) worker1 # [ 89.317912] nix-daemon[921]: reaped child process 977, status = succeeded lb # [ 89.366563] niks3-server[1010]: 2026/09/22 10:46:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 89.349713] nix-grpc-daemon[933]: time=2026-09-22T10:46:26.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 89.351947] nix-grpc-daemon[933]: time=2026-09-22T10:46:26.631Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 89.444297] niks3-server[1010]: 2026/09/22 10:46:26 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 89.453961] niks3-server[1010]: 2026/09/22 10:46:26 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 89.462596] niks3-server[1010]: 2026/09/22 10:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 89.468713] niks3-server[1010]: 2026/09/22 10:46:26 INFO Signed narinfos id=2 count=1 worker1 # [ 89.435167] nix-grpc-daemon[933]: time=2026-09-22T10:46:26.716Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.471503] niks3-server[1010]: 2026/09/22 10:46:26 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 89.482682] niks3-server[1010]: 2026/09/22 10:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 89.491308] niks3-server[1010]: 2026/09/22 10:46:26 INFO Completed upload id=2 worker1 # [ 89.457116] nix-grpc-daemon[933]: time=2026-09-22T10:46:26.740Z level=INFO msg="Upload complete. (414ms)" lb # [ 89.496317] niks3-server[1010]: 2026/09/22 10:46:26 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 89.469486] nix-grpc-daemon[860]: ts=2026-09-22T10:46:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 89.522782] nix-grpc-daemon[867]: ts=2026-09-22T10:46:26Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 89.529795] nix-grpc-daemon[867]: ts=2026-09-22T10:46:26Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 89.501921] nix-daemon[921]: reaped child process 956, status = succeeded worker1 # [ 89.542850] nix-grpc-daemon[860]: ts=2026-09-22T10:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 89.566091] nix-grpc-daemon[860]: ts=2026-09-22T10:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 89.591420] nix-grpc-daemon[933]: time=2026-09-22T10:46:26.874Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 89.606313] nix-grpc-daemon[860]: ts=2026-09-22T10:46:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 89.635864] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 89.884955] nix-daemon[921]: accepted connection from pid 981, user nix-grpc-daemon (trusted) worker1 # [ 89.915749] nix-daemon[921]: reaped child process 984, status = succeeded worker1 # [ 90.076149] nix-daemon[921]: accepted connection from pid 986, user nix-grpc-daemon (trusted) worker1 # [ 90.112234] nix-daemon[921]: reaped child process 989, status = succeeded lb # [ 90.154368] niks3-server[1010]: 2026/09/22 10:46:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.139166] nix-grpc-daemon[933]: time=2026-09-22T10:46:27.421Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 90.140951] nix-grpc-daemon[933]: time=2026-09-22T10:46:27.421Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 90.206031] niks3-server[1010]: 2026/09/22 10:46:27 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst worker1 # [ 90.184794] nix-grpc-daemon[933]: time=2026-09-22T10:46:27.467Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.221875] niks3-server[1010]: 2026/09/22 10:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 90.231423] niks3-server[1010]: 2026/09/22 10:46:27 INFO Signed narinfos id=3 count=1 lb # [ 90.236822] niks3-server[1010]: 2026/09/22 10:46:27 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 90.246772] niks3-server[1010]: 2026/09/22 10:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 90.254743] niks3-server[1010]: 2026/09/22 10:46:27 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo lb # [ 90.259748] niks3-server[1010]: 2026/09/22 10:46:27 INFO Completed upload id=3 worker1 # [ 90.225846] nix-grpc-daemon[933]: time=2026-09-22T10:46:27.509Z level=INFO msg="Upload complete. (562ms)" worker1 # [ 90.238376] nix-grpc-daemon[860]: ts=2026-09-22T10:46:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 90.253019] nix-grpc-daemon[860]: ts=2026-09-22T10:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 90.267338] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 90.275543] nix-daemon[921]: reaped child process 979, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker1 # [ 90.291186] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 91.525053] nix-daemon[921]: reaped child process 993, status = succeeded worker1 # [ 91.700665] nix-daemon[921]: accepted connection from pid 1009, user nix-grpc-daemon (trusted) worker1 # [ 91.729699] nix-daemon[921]: reaped child process 1012, status = succeeded lb # [ 91.777667] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 91.756558] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.038Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 91.758313] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.038Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 91.819647] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 91.836809] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 91.845661] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 91.854032] niks3-server[1010]: 2026/09/22 10:46:29 INFO Signed narinfos id=4 count=1 worker1 # [ 91.820238] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.102Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.862679] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 91.873893] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 91.878891] niks3-server[1010]: 2026/09/22 10:46:29 INFO Completed upload id=4 worker1 # [ 91.844384] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.127Z level=INFO msg="Upload complete. (335ms)" lb # [ 91.881232] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 91.858363] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 91.897417] nix-grpc-daemon[867]: ts=2026-09-22T10:46:29Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker2 # [ 91.903700] nix-grpc-daemon[867]: ts=2026-09-22T10:46:29Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 91.876500] nix-daemon[921]: reaped child process 991, status = succeeded worker1 # [ 91.907282] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 91.929038] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 91.950775] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.233Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 91.966027] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 91.994881] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 92.233261] nix-daemon[921]: accepted connection from pid 1016, user nix-grpc-daemon (trusted) worker1 # [ 92.263734] nix-daemon[921]: reaped child process 1019, status = succeeded worker1 # [ 92.464877] nix-daemon[921]: accepted connection from pid 1021, user nix-grpc-daemon (trusted) worker1 # [ 92.487908] nix-daemon[921]: reaped child process 1024, status = succeeded lb # [ 92.533047] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 92.512111] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.794Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 92.513774] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.794Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 92.572837] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 92.588634] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker1 # [ 92.561178] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.844Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.602573] niks3-server[1010]: 2026/09/22 10:46:29 INFO Signed narinfos id=5 count=1 lb # [ 92.609332] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 92.619383] niks3-server[1010]: 2026/09/22 10:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 92.628357] niks3-server[1010]: 2026/09/22 10:46:29 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo lb # [ 92.635952] niks3-server[1010]: 2026/09/22 10:46:29 INFO Completed upload id=5 worker1 # [ 92.601912] nix-grpc-daemon[933]: time=2026-09-22T10:46:29.885Z level=INFO msg="Upload complete. (583ms)" worker1 # [ 92.613174] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 92.632450] nix-grpc-daemon[860]: ts=2026-09-22T10:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 92.645402] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 92.656502] nix-daemon[921]: reaped child process 1014, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 92.668338] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 93.713612] nix-daemon[921]: reaped child process 1028, status = succeeded worker1 # [ 93.891215] nix-daemon[921]: accepted connection from pid 1044, user nix-grpc-daemon (trusted) worker1 # [ 93.915167] nix-daemon[921]: reaped child process 1047, status = succeeded lb # [ 93.962023] niks3-server[1010]: 2026/09/22 10:46:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.940805] nix-grpc-daemon[933]: time=2026-09-22T10:46:31.222Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 93.945399] nix-grpc-daemon[933]: time=2026-09-22T10:46:31.222Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 94.015226] niks3-server[1010]: 2026/09/22 10:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 93.996571] nix-grpc-daemon[933]: time=2026-09-22T10:46:31.279Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.030914] niks3-server[1010]: 2026/09/22 10:46:31 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 94.043077] niks3-server[1010]: 2026/09/22 10:46:31 INFO Signed narinfos id=6 count=1 lb # [ 94.048633] niks3-server[1010]: 2026/09/22 10:46:31 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 94.060461] niks3-server[1010]: 2026/09/22 10:46:31 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 94.063583] niks3-server[1010]: 2026/09/22 10:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 94.070443] niks3-server[1010]: 2026/09/22 10:46:31 INFO Completed upload id=6 lb # [ 94.074407] niks3-server[1010]: 2026/09/22 10:46:31 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 94.036523] nix-grpc-daemon[933]: time=2026-09-22T10:46:31.319Z level=INFO msg="Upload complete. (353ms)" worker1 # [ 94.049604] nix-grpc-daemon[860]: ts=2026-09-22T10:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 94.069547] nix-daemon[921]: reaped child process 1026, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 8.33 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.25 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 9.03 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 95.743191] nix-grpc-daemon[867]: ts=2026-09-22T10:46:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 95.713196] nix-grpc-daemon[860]: ts=2026-09-22T10:46:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 95.728363] nix-grpc-daemon[933]: time=2026-09-22T10:46:33.010Z level=INFO msg="All 2 paths already cached" worker2 # [ 95.784941] nix-grpc-daemon[867]: ts=2026-09-22T10:46:33Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.36 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.60 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 96.911725] nix-grpc-daemon[867]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 96.882959] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 96.896472] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.178Z level=INFO msg="All 2 paths already cached" worker2 # [ 96.949229] nix-grpc-daemon[867]: ts=2026-09-22T10:46:34Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker2 # [ 96.956142] nix-grpc-daemon[867]: ts=2026-09-22T10:46:34Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 96.962511] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 96.979570] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 96.995586] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.278Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 97.008147] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 97.024471] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 97.245258] nix-daemon[921]: accepted connection from pid 1064, user nix-grpc-daemon (trusted) worker1 # [ 97.269824] nix-daemon[921]: reaped child process 1067, status = succeeded worker1 # [ 97.459917] nix-daemon[921]: accepted connection from pid 1069, user nix-grpc-daemon (trusted) worker1 # [ 97.481398] nix-daemon[921]: reaped child process 1072, status = succeeded lb # [ 97.527713] niks3-server[1010]: 2026/09/22 10:46:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.504942] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.787Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 97.506552] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.787Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 97.562119] niks3-server[1010]: 2026/09/22 10:46:34 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 97.543699] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.826Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.579264] niks3-server[1010]: 2026/09/22 10:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 97.591825] niks3-server[1010]: 2026/09/22 10:46:34 INFO Signed narinfos id=7 count=1 lb # [ 97.595057] niks3-server[1010]: 2026/09/22 10:46:34 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 97.604506] niks3-server[1010]: 2026/09/22 10:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 97.612224] niks3-server[1010]: 2026/09/22 10:46:34 INFO Completed upload id=7 worker1 # [ 97.578305] nix-grpc-daemon[933]: time=2026-09-22T10:46:34.861Z level=INFO msg="Upload complete. (533ms)" lb # [ 97.615005] niks3-server[1010]: 2026/09/22 10:46:34 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 97.588943] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 97.596407] nix-grpc-daemon[860]: ts=2026-09-22T10:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 97.611899] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 97.614369] nix-daemon[921]: reaped child process 1062, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 97.628603] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 98.483780] nix-daemon[921]: reaped child process 1076, status = succeeded worker1 # [ 98.663987] nix-daemon[921]: accepted connection from pid 1092, user nix-grpc-daemon (trusted) worker1 # [ 98.687350] nix-daemon[921]: reaped child process 1095, status = succeeded lb # [ 98.738239] niks3-server[1010]: 2026/09/22 10:46:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 98.713100] nix-grpc-daemon[933]: time=2026-09-22T10:46:35.996Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 98.715777] nix-grpc-daemon[933]: time=2026-09-22T10:46:35.998Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 98.781873] niks3-server[1010]: 2026/09/22 10:46:36 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 98.794937] niks3-server[1010]: 2026/09/22 10:46:36 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 98.769971] nix-grpc-daemon[933]: time=2026-09-22T10:46:36.052Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.804805] niks3-server[1010]: 2026/09/22 10:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 98.813721] niks3-server[1010]: 2026/09/22 10:46:36 INFO Signed narinfos id=8 count=1 lb # [ 98.819292] niks3-server[1010]: 2026/09/22 10:46:36 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 98.829053] niks3-server[1010]: 2026/09/22 10:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 98.840399] niks3-server[1010]: 2026/09/22 10:46:36 INFO Completed upload id=8 lb # [ 98.843566] niks3-server[1010]: 2026/09/22 10:46:36 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 98.806445] nix-grpc-daemon[933]: time=2026-09-22T10:46:36.089Z level=INFO msg="Upload complete. (364ms)" worker1 # [ 98.821819] nix-grpc-daemon[860]: ts=2026-09-22T10:46:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 98.841131] nix-daemon[921]: reaped child process 1074, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.25 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath lb # [ 98.909362] envoy[961]: conn peer=[2001:db8:1::1]:50904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4070 ms=8000 lb # [ 98.911626] envoy[961]: conn peer=[2001:db8:1::1]:57980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=65 lb # [ 98.914351] envoy[961]: conn peer=[2001:db8:1::1]:57982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1806 ms=1956 client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.25 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.89 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup client # [ 99.372890] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 3e11f11fe0894dd3b264dd41ee33c492 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup worker2 # [ 99.704733] nix-grpc-daemon[867]: ts=2026-09-22T10:46:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 client # [ 99.667551] nix[898]: don't know how to build these paths: client # [ 99.668797] nix[898]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 99.727454] nix-grpc-daemon[867]: ts=2026-09-22T10:46:36Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 99.736130] nix-grpc-daemon[867]: ts=2026-09-22T10:46:36Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 99.774093] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 99.796761] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 client # [ 99.768238] nix[898]: copying 1 paths to 'lb:50051'... worker2 # [ 99.814608] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 client # [ 99.770884] nix[898]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker2 # [ 99.835984] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 99.868694] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=info event=niks3_push_started pid=997 worker2 # [ 100.155950] nix-grpc-daemon[997]: time=2026-09-22T10:46:37.422Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 100.368290] nix-daemon[947]: accepted connection from pid 1010, user nix-grpc-daemon (trusted) worker2 # [ 100.392895] nix-daemon[947]: reaped child process 1013, status = succeeded client # [ 100.501966] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup. worker2 # [ 100.576396] nix-daemon[947]: accepted connection from pid 1015, user nix-grpc-daemon (trusted) client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds) worker2 # [ 100.597477] nix-daemon[947]: reaped child process 1018, status = succeeded lb # [ 100.608838] niks3-server[1010]: 2026/09/22 10:46:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.619232] nix-grpc-daemon[997]: time=2026-09-22T10:46:37.887Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 100.622561] nix-grpc-daemon[997]: time=2026-09-22T10:46:37.889Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 100.639325] niks3-server[1010]: 2026/09/22 10:46:37 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 100.645978] niks3-server[1010]: 2026/09/22 10:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign worker2 # [ 100.650181] nix-grpc-daemon[997]: time=2026-09-22T10:46:37.917Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.651051] niks3-server[1010]: 2026/09/22 10:46:37 INFO Signed narinfos id=9 count=1 lb # [ 100.656245] niks3-server[1010]: 2026/09/22 10:46:37 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 100.665506] niks3-server[1010]: 2026/09/22 10:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 100.670186] niks3-server[1010]: 2026/09/22 10:46:37 INFO Completed upload id=9 lb # [ 100.672482] niks3-server[1010]: 2026/09/22 10:46:37 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker2 # [ 100.670342] nix-grpc-daemon[997]: time=2026-09-22T10:46:37.938Z level=INFO msg="Upload complete. (506ms)" worker2 # [ 100.676491] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 100.688820] nix-grpc-daemon[867]: ts=2026-09-22T10:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 100.696349] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 100.699483] nix-daemon[947]: reaped child process 995, status = succeeded client # Running as unit: dedup2.service; invocation ID: 8d144596af204c179c8bd89d303c9bee client # [ 100.665474] nix[898]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 100.717926] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker1 # [ 100.967797] nix-grpc-daemon[860]: ts=2026-09-22T10:46:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34910 client # [ 100.965593] nix[914]: don't know how to build these paths: client # [ 100.967431] nix[914]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 101.026628] nix-grpc-daemon[867]: ts=2026-09-22T10:46:38Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 100.989806] nix[914]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 101.034851] nix-grpc-daemon[867]: ts=2026-09-22T10:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.08 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 104.380810] nix-daemon[947]: reaped child process 1026, status = succeeded worker2 # [ 104.571447] nix-daemon[947]: accepted connection from pid 1054, user nix-grpc-daemon (trusted) worker2 # [ 104.592913] nix-daemon[947]: reaped child process 1057, status = succeeded lb # [ 104.606072] niks3-server[1010]: 2026/09/22 10:46:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 104.619452] nix-grpc-daemon[997]: time=2026-09-22T10:46:41.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 104.621846] nix-grpc-daemon[997]: time=2026-09-22T10:46:41.885Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 104.657815] niks3-server[1010]: 2026/09/22 10:46:41 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 104.669085] niks3-server[1010]: 2026/09/22 10:46:41 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 104.679854] niks3-server[1010]: 2026/09/22 10:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 104.686963] niks3-server[1010]: 2026/09/22 10:46:41 INFO Signed narinfos id=10 count=1 worker2 # [ 104.688133] nix-grpc-daemon[997]: time=2026-09-22T10:46:41.955Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.690347] niks3-server[1010]: 2026/09/22 10:46:41 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 104.701950] niks3-server[1010]: 2026/09/22 10:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker2 # [ 104.711414] nix-grpc-daemon[997]: time=2026-09-22T10:46:41.979Z level=INFO msg="Upload complete. (376ms)" lb # [ 104.711613] niks3-server[1010]: 2026/09/22 10:46:41 INFO Completed upload id=10 lb # [ 104.714890] niks3-server[1010]: 2026/09/22 10:46:41 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker2 # [ 104.726345] nix-grpc-daemon[867]: ts=2026-09-22T10:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker2 # [ 104.740832] nix-grpc-daemon[867]: ts=2026-09-22T10:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker2 # [ 104.748831] nix-daemon[947]: reaped child process 1022, status = succeeded client # [ 104.730563] systemd[1]: dedup2.service: Deactivated successfully. client # [ 104.738254] systemd[1]: dedup2.service: Consumed 261ms CPU time over 4.235s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic. client # [ 104.789500] systemd[1]: dedup1.service: Deactivated successfully. client # [ 104.794089] systemd[1]: dedup1.service: Consumed 200ms CPU time over 5.420s wall clock time, 39.3M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.16 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.39 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 105.994396] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 106.049095] nix-grpc-daemon[867]: ts=2026-09-22T10:46:43Z level=debug event=want client=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 106.049476] nix-grpc-daemon[867]: ts=2026-09-22T10:46:43Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 106.059832] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 106.077531] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 106.094888] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 106.111628] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 106.327287] nix-daemon[921]: accepted connection from pid 1120, user nix-grpc-daemon (trusted) worker1 # [ 106.351433] nix-daemon[921]: reaped child process 1123, status = succeeded worker1 # [ 106.526773] nix-daemon[921]: accepted connection from pid 1125, user nix-grpc-daemon (trusted) worker1 # [ 106.547616] nix-daemon[921]: reaped child process 1128, status = succeeded lb # [ 106.598861] niks3-server[1010]: 2026/09/22 10:46:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 106.576496] nix-grpc-daemon[933]: time=2026-09-22T10:46:43.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 106.579107] nix-grpc-daemon[933]: time=2026-09-22T10:46:43.859Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 106.642687] niks3-server[1010]: 2026/09/22 10:46:43 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker1 # [ 106.621824] nix-grpc-daemon[933]: time=2026-09-22T10:46:43.904Z level=INFO msg="Uploading 1 narinfos" lb # [ 106.662113] niks3-server[1010]: 2026/09/22 10:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 106.674118] niks3-server[1010]: 2026/09/22 10:46:43 INFO Signed narinfos id=11 count=1 lb # [ 106.680506] niks3-server[1010]: 2026/09/22 10:46:43 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls worker1 # [ 106.657782] nix-grpc-daemon[933]: time=2026-09-22T10:46:43.940Z level=INFO msg="Upload complete. (527ms)" lb # [ 106.691983] niks3-server[1010]: 2026/09/22 10:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 106.696211] niks3-server[1010]: 2026/09/22 10:46:43 INFO Completed upload id=11 lb # [ 106.699844] niks3-server[1010]: 2026/09/22 10:46:43 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker1 # [ 106.665723] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 106.684523] nix-grpc-daemon[860]: ts=2026-09-22T10:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 106.693340] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 106.699470] nix-daemon[921]: reaped child process 1118, status = succeeded worker1 # [ 106.717108] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) lb # [ 108.913468] envoy[961]: conn peer=[2001:db8:1::1]:58824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=3741 lb # [ 108.918883] envoy[961]: conn peer=[2001:db8:1::1]:57998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3022 tx=1471 ms=5101 worker1 # [ 112.737696] nix-grpc-daemon[860]: ts=2026-09-22T10:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=6 outputs=0 worker1 # [ 112.758275] nix-daemon[921]: reaped child process 1130, status = succeeded worker1 # [ 112.759490] nix-daemon[921]: reaped child process 1132, status = succeeded worker1 # [ 113.066195] nix-grpc-daemon[860]: ts=2026-09-22T10:46:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 113.121771] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=debug event=want client=7 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker2 # [ 113.130065] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 113.137723] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 113.152927] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 113.167253] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 113.185129] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 113.401308] nix-daemon[947]: accepted connection from pid 1074, user nix-grpc-daemon (trusted) worker2 # [ 113.422742] nix-daemon[947]: reaped child process 1077, status = succeeded worker2 # [ 113.604429] nix-daemon[947]: accepted connection from pid 1079, user nix-grpc-daemon (trusted) worker2 # [ 113.624477] nix-daemon[947]: reaped child process 1082, status = succeeded lb # [ 113.642261] niks3-server[1010]: 2026/09/22 10:46:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 113.655116] nix-grpc-daemon[997]: time=2026-09-22T10:46:50.922Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 113.656823] nix-grpc-daemon[997]: time=2026-09-22T10:46:50.922Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 113.678749] niks3-server[1010]: 2026/09/22 10:46:50 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 113.692612] niks3-server[1010]: 2026/09/22 10:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker2 # [ 113.698978] nix-grpc-daemon[997]: time=2026-09-22T10:46:50.966Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.700359] niks3-server[1010]: 2026/09/22 10:46:50 INFO Signed narinfos id=12 count=1 lb # [ 113.708873] niks3-server[1010]: 2026/09/22 10:46:50 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 113.719557] niks3-server[1010]: 2026/09/22 10:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete worker2 # [ 113.726038] nix-grpc-daemon[997]: time=2026-09-22T10:46:50.994Z level=INFO msg="Upload complete. (523ms)" lb # [ 113.728464] niks3-server[1010]: 2026/09/22 10:46:50 INFO Completed upload id=12 lb # [ 113.731565] niks3-server[1010]: 2026/09/22 10:46:50 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker2 # [ 113.736409] nix-grpc-daemon[867]: ts=2026-09-22T10:46:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 113.747561] nix-grpc-daemon[867]: ts=2026-09-22T10:46:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 113.760942] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 113.766349] nix-daemon[947]: reaped child process 1072, status = succeeded worker2 # [ 113.782603] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) lb # [ 118.914080] envoy[961]: conn peer=[2001:db8:1::1]:58830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=953 ms=6769 worker2 # [ 120.021009] nix-grpc-daemon[867]: ts=2026-09-22T10:46:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=7 outputs=0 worker2 # [ 120.128551] nix-daemon[947]: reaped child process 1084, status = succeeded worker2 # [ 120.133587] nix-daemon[947]: reaped child process 1086, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 14.77 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share client # [ 120.567788] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 074cb3f9a7304d94b1e4f426e0f2a903 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 0.15 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share worker1 # [ 120.931166] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34910 client # [ 120.928968] nix[988]: don't know how to build these paths: client # [ 120.931492] nix[988]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 120.991453] nix-grpc-daemon[867]: ts=2026-09-22T10:46:58Z level=debug event=want client=8 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 120.967271] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 120.988845] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 client # [ 120.992657] nix[988]: copying 1 paths to 'lb:50051'... client # [ 120.995539] nix[988]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker1 # [ 121.005076] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 121.019683] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 121.245543] nix-daemon[921]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker1 # [ 121.269679] nix-daemon[921]: reaped child process 1158, status = succeeded worker1 # [ 121.431030] nix-daemon[921]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker1 # [ 121.449451] nix-daemon[921]: reaped child process 1163, status = succeeded lb # [ 121.499279] niks3-server[1010]: 2026/09/22 10:46:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.476747] nix-grpc-daemon[933]: time=2026-09-22T10:46:58.759Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 121.478297] nix-grpc-daemon[933]: time=2026-09-22T10:46:58.759Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 121.537732] niks3-server[1010]: 2026/09/22 10:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker1 # [ 121.512209] nix-grpc-daemon[933]: time=2026-09-22T10:46:58.795Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.553294] niks3-server[1010]: 2026/09/22 10:46:58 INFO Signed narinfos id=13 count=1 lb # [ 121.560170] niks3-server[1010]: 2026/09/22 10:46:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 121.571932] niks3-server[1010]: 2026/09/22 10:46:58 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst worker1 # [ 121.541979] nix-grpc-daemon[933]: time=2026-09-22T10:46:58.825Z level=INFO msg="Upload complete. (496ms)" lb # [ 121.576705] niks3-server[1010]: 2026/09/22 10:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 121.581002] niks3-server[1010]: 2026/09/22 10:46:58 INFO Completed upload id=13 worker1 # [ 121.546529] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 121.583689] niks3-server[1010]: 2026/09/22 10:46:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker1 # [ 121.558172] nix-grpc-daemon[860]: ts=2026-09-22T10:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 121.562577] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 121.566582] nix-daemon[921]: reaped child process 1153, status = succeeded client # [ 121.563751] nix[988]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker1 # [ 121.577162] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker2 # [ 120.998028] nix-grpc-daemon[867]: ts=2026-09-22T10:46:58Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=1 client # [ 121.729143] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 72ce58ce3b864bd5b01f224e06aafda1 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 1.16 seconds) worker2 # [ 122.251830] nix-grpc-daemon[867]: ts=2026-09-22T10:46:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40376 client # [ 122.212834] nix[1004]: don't know how to build these paths: client # [ 122.214309] nix[1004]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 122.294468] nix-grpc-daemon[867]: ts=2026-09-22T10:46:59Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 122.285100] nix-grpc-daemon[860]: ts=2026-09-22T10:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 client # [ 122.278789] nix[1004]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 125.300626] systemd[1]: share1.service: Sent signal SIGINT to main process 988 (nix) on client request. client # [ 125.309728] nix[988]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.12 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 125.327917] nix[988]: error: interrupted by the user client # [ 125.381769] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 125.387900] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 125.391875] systemd[1]: share1.service: Consumed 236ms CPU time over 4.818s wall clock time, 41.2M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 0.11 seconds) worker1 # [ 125.808454] nix-daemon[921]: reaped child process 1167, status = succeeded client: waiting for success: ! systemctl is-active share2 worker1 # [ 126.001739] nix-daemon[921]: accepted connection from pid 1201, user nix-grpc-daemon (trusted) worker1 # [ 126.033868] nix-daemon[921]: reaped child process 1204, status = succeeded lb # [ 126.081967] niks3-server[1010]: 2026/09/22 10:47:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 126.064257] nix-grpc-daemon[933]: time=2026-09-22T10:47:03.344Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 126.065886] nix-grpc-daemon[933]: time=2026-09-22T10:47:03.344Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" worker1 # [ 126.117313] nix-grpc-daemon[933]: time=2026-09-22T10:47:03.399Z level=INFO msg="Uploading 1 narinfos" lb # [ 126.144263] niks3-server[1010]: 2026/09/22 10:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 126.158246] niks3-server[1010]: 2026/09/22 10:47:03 INFO Signed narinfos id=14 count=1 lb # [ 126.163289] niks3-server[1010]: 2026/09/22 10:47:03 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 126.171384] niks3-server[1010]: 2026/09/22 10:47:03 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 126.179407] niks3-server[1010]: 2026/09/22 10:47:03 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 126.183980] niks3-server[1010]: 2026/09/22 10:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 126.188925] niks3-server[1010]: 2026/09/22 10:47:03 INFO Completed upload id=14 worker1 # [ 126.154376] nix-grpc-daemon[933]: time=2026-09-22T10:47:03.437Z level=INFO msg="Upload complete. (364ms)" lb # [ 126.192235] niks3-server[1010]: 2026/09/22 10:47:03 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker1 # [ 126.160729] nix-grpc-daemon[860]: ts=2026-09-22T10:47:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=1 worker1 # [ 126.172594] nix-grpc-daemon[860]: ts=2026-09-22T10:47:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=1 worker1 # [ 126.175602] nix-daemon[921]: reaped child process 1165, status = succeeded client # [ 126.185750] systemd[1]: share2.service: Deactivated successfully. client # [ 126.188954] systemd[1]: share2.service: Consumed 345ms CPU time over 4.459s wall clock time, 33.2M memory peak, 5.8K incoming IP traffic, 5.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.14 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 6.58 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 127.262691] systemd[1]: Started Nix Daemon. client # [ 127.504287] nix-daemon[1051]: accepted connection from pid 1048, user root (trusted) worker2 # [ 127.829528] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 127.804665] nix-grpc-daemon[860]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 127.865152] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=want client=10 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 127.870814] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 127.915976] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 127.939443] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 127.961004] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 127.971371] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 128.216697] nix-daemon[947]: accepted connection from pid 1131, user nix-grpc-daemon (trusted) worker2 # [ 128.245697] nix-daemon[947]: reaped child process 1134, status = succeeded worker2 # [ 128.488046] nix-daemon[947]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker2 # [ 128.510265] nix-daemon[947]: reaped child process 1139, status = succeeded lb # [ 128.529044] niks3-server[1010]: 2026/09/22 10:47:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 128.544098] nix-grpc-daemon[997]: time=2026-09-22T10:47:05.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 128.545792] nix-grpc-daemon[997]: time=2026-09-22T10:47:05.811Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 128.558791] niks3-server[1010]: 2026/09/22 10:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 128.562280] niks3-server[1010]: 2026/09/22 10:47:05 INFO Signed narinfos id=15 count=1 worker2 # [ 128.564171] nix-grpc-daemon[997]: time=2026-09-22T10:47:05.831Z level=INFO msg="Uploading 1 narinfos" lb # [ 128.564747] niks3-server[1010]: 2026/09/22 10:47:05 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 128.573895] niks3-server[1010]: 2026/09/22 10:47:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 128.580145] niks3-server[1010]: 2026/09/22 10:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 128.587621] niks3-server[1010]: 2026/09/22 10:47:05 INFO Completed upload id=15 worker2 # [ 128.588170] nix-grpc-daemon[997]: time=2026-09-22T10:47:05.856Z level=INFO msg="Upload complete. (601ms)" lb # [ 128.591307] niks3-server[1010]: 2026/09/22 10:47:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker2 # [ 128.596271] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 128.599468] nix-grpc-daemon[867]: ts=2026-09-22T10:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 128.613767] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 128.618451] nix-daemon[947]: reaped child process 1129, status = succeeded worker2 # [ 128.631315] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) lb # [ 128.913376] envoy[961]: conn peer=[2001:db8:1::1]:39340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=954 ms=6972 lb # [ 128.915114] envoy[961]: conn peer=[2001:db8:1::1]:54438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=789 ms=4415 lb # [ 128.917222] envoy[961]: conn peer=[2001:db8:1::1]:54440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1087 ms=3990 worker2 # [ 129.888195] nix-daemon[947]: reaped child process 1143, status = succeeded worker2 # [ 130.080911] nix-daemon[947]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 130.105605] nix-daemon[947]: reaped child process 1162, status = succeeded lb # [ 130.119784] niks3-server[1010]: 2026/09/22 10:47:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 130.132660] nix-grpc-daemon[997]: time=2026-09-22T10:47:07.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 130.135687] nix-grpc-daemon[997]: time=2026-09-22T10:47:07.398Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 130.159302] niks3-server[1010]: 2026/09/22 10:47:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 130.176571] niks3-server[1010]: 2026/09/22 10:47:07 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 130.186780] niks3-server[1010]: 2026/09/22 10:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker2 # [ 130.193913] nix-grpc-daemon[997]: time=2026-09-22T10:47:07.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.193650] niks3-server[1010]: 2026/09/22 10:47:07 INFO Signed narinfos id=16 count=1 lb # [ 130.201292] niks3-server[1010]: 2026/09/22 10:47:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 130.212511] niks3-server[1010]: 2026/09/22 10:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 130.221373] niks3-server[1010]: 2026/09/22 10:47:07 INFO Completed upload id=16 worker2 # [ 130.221200] nix-grpc-daemon[997]: time=2026-09-22T10:47:07.489Z level=INFO msg="Upload complete. (373ms)" lb # [ 130.224719] niks3-server[1010]: 2026/09/22 10:47:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 130.206453] nix-grpc-daemon[860]: ts=2026-09-22T10:47:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 130.237536] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=9 outputs=1 worker2 # [ 130.262394] nix-daemon[947]: reaped child process 1141, status = succeeded worker1 # [ 130.233967] nix-grpc-daemon[860]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 130.320173] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 130.331255] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 130.650548] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 130.625022] nix-grpc-daemon[860]: ts=2026-09-22T10:47:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 130.641825] nix-grpc-daemon[933]: time=2026-09-22T10:47:07.924Z level=INFO msg="All 1 paths already cached" worker2 # [ 130.699571] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=debug event=want client=11 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker2 # [ 130.710189] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 130.723052] nix-grpc-daemon[867]: ts=2026-09-22T10:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 130.740190] nix-grpc-daemon[867]: ts=2026-09-22T10:47:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 130.909156] nix-grpc-daemon[997]: time=2026-09-22T10:47:08.177Z level=INFO msg="All 1 paths already cached" worker2 # [ 130.941112] nix-grpc-daemon[867]: ts=2026-09-22T10:47:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 130.958941] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 131.198883] nix-daemon[947]: accepted connection from pid 1168, user nix-grpc-daemon (trusted) worker2 # [ 131.240632] nix-daemon[947]: reaped child process 1171, status = succeeded worker2 # [ 131.474844] nix-daemon[947]: accepted connection from pid 1173, user nix-grpc-daemon (trusted) worker2 # [ 131.511629] nix-daemon[947]: reaped child process 1176, status = succeeded lb # [ 131.521783] niks3-server[1010]: 2026/09/22 10:47:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.538363] nix-grpc-daemon[997]: time=2026-09-22T10:47:08.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 131.541678] nix-grpc-daemon[997]: time=2026-09-22T10:47:08.809Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 131.568848] niks3-server[1010]: 2026/09/22 10:47:08 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker2 # [ 131.584630] nix-grpc-daemon[997]: time=2026-09-22T10:47:08.852Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.585568] niks3-server[1010]: 2026/09/22 10:47:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 131.595155] niks3-server[1010]: 2026/09/22 10:47:08 INFO Signed narinfos id=17 count=1 lb # [ 131.602038] niks3-server[1010]: 2026/09/22 10:47:08 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 131.608655] niks3-server[1010]: 2026/09/22 10:47:08 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 131.617123] niks3-server[1010]: 2026/09/22 10:47:08 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo lb # [ 131.620631] niks3-server[1010]: 2026/09/22 10:47:08 INFO Completed upload id=17 worker2 # [ 131.621128] nix-grpc-daemon[997]: time=2026-09-22T10:47:08.889Z level=INFO msg="Upload complete. (644ms)" worker2 # [ 131.628709] nix-grpc-daemon[867]: ts=2026-09-22T10:47:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 131.636595] nix-grpc-daemon[867]: ts=2026-09-22T10:47:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 131.651185] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 131.653723] nix-daemon[947]: reaped child process 1166, status = succeeded worker2 # [ 131.668002] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 133.114079] nix-daemon[947]: reaped child process 1180, status = succeeded worker2 # [ 133.318150] nix-daemon[947]: accepted connection from pid 1196, user nix-grpc-daemon (trusted) worker2 # [ 133.359394] nix-daemon[947]: reaped child process 1199, status = succeeded lb # [ 133.369563] niks3-server[1010]: 2026/09/22 10:47:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.383957] nix-grpc-daemon[997]: time=2026-09-22T10:47:10.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 133.385745] nix-grpc-daemon[997]: time=2026-09-22T10:47:10.649Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 133.399241] niks3-server[1010]: 2026/09/22 10:47:10 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 133.410806] niks3-server[1010]: 2026/09/22 10:47:10 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker2 # [ 133.415519] nix-grpc-daemon[997]: time=2026-09-22T10:47:10.683Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.416636] niks3-server[1010]: 2026/09/22 10:47:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign lb # [ 133.418579] niks3-server[1010]: 2026/09/22 10:47:10 INFO Signed narinfos id=18 count=1 lb # [ 133.425461] niks3-server[1010]: 2026/09/22 10:47:10 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 133.432131] niks3-server[1010]: 2026/09/22 10:47:10 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 133.438957] niks3-server[1010]: 2026/09/22 10:47:10 INFO Completed upload id=18 worker2 # [ 133.439577] nix-grpc-daemon[997]: time=2026-09-22T10:47:10.707Z level=INFO msg="Upload complete. (354ms)" lb # [ 133.441014] niks3-server[1010]: 2026/09/22 10:47:10 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker2 # [ 133.448998] nix-grpc-daemon[867]: ts=2026-09-22T10:47:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=10 outputs=1 worker1 # [ 133.423381] nix-grpc-daemon[860]: ts=2026-09-22T10:47:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 133.468480] nix-daemon[947]: reaped child process 1178, status = succeeded worker1 # [ 133.441057] nix-grpc-daemon[860]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 133.525562] nix-grpc-daemon[867]: ts=2026-09-22T10:47:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 133.529651] nix-grpc-daemon[867]: ts=2026-09-22T10:47:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 133.969913] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 133.940593] nix-grpc-daemon[860]: ts=2026-09-22T10:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 133.950904] nix-grpc-daemon[933]: time=2026-09-22T10:47:11.233Z level=INFO msg="All 2 paths already cached" worker2 # [ 134.010369] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=want client=12 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker2 # [ 134.019176] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 134.030205] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 134.045473] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 134.193051] nix-grpc-daemon[997]: time=2026-09-22T10:47:11.460Z level=INFO msg="All 2 paths already cached" worker2 # [ 134.208238] nix-grpc-daemon[867]: ts=2026-09-22T10:47:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 134.225635] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 134.483212] nix-daemon[947]: accepted connection from pid 1205, user nix-grpc-daemon (trusted) worker2 # [ 134.521790] nix-daemon[947]: reaped child process 1208, status = succeeded worker2 # [ 134.715247] nix-daemon[947]: accepted connection from pid 1210, user nix-grpc-daemon (trusted) lb # [ 134.741235] niks3-server[1010]: 2026/09/22 10:47:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.743918] nix-daemon[947]: reaped child process 1213, status = succeeded worker2 # [ 134.758449] nix-grpc-daemon[997]: time=2026-09-22T10:47:12.025Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 134.759888] nix-grpc-daemon[997]: time=2026-09-22T10:47:12.025Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 134.783968] niks3-server[1010]: 2026/09/22 10:47:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 134.791237] niks3-server[1010]: 2026/09/22 10:47:12 INFO Signed narinfos id=19 count=1 worker2 # [ 134.793291] nix-grpc-daemon[997]: time=2026-09-22T10:47:12.061Z level=INFO msg="Uploading 1 narinfos" lb # [ 134.794955] niks3-server[1010]: 2026/09/22 10:47:12 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 134.802190] niks3-server[1010]: 2026/09/22 10:47:12 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 134.822919] niks3-server[1010]: 2026/09/22 10:47:12 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 134.867928] niks3-server[1010]: 2026/09/22 10:47:12 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker2 # [ 134.874006] nix-grpc-daemon[997]: time=2026-09-22T10:47:12.142Z level=INFO msg="Upload complete. (627ms)" lb # [ 134.875567] niks3-server[1010]: 2026/09/22 10:47:12 INFO Completed upload id=19 worker2 # [ 134.880763] nix-grpc-daemon[867]: ts=2026-09-22T10:47:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 134.889874] nix-grpc-daemon[867]: ts=2026-09-22T10:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 134.900657] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 134.904695] nix-daemon[947]: reaped child process 1203, status = succeeded worker2 # [ 134.917072] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 136.161624] nix-daemon[947]: reaped child process 1217, status = succeeded worker2 # [ 136.349835] nix-daemon[947]: accepted connection from pid 1233, user nix-grpc-daemon (trusted) worker2 # [ 136.373666] nix-daemon[947]: reaped child process 1236, status = succeeded lb # [ 136.391459] niks3-server[1010]: 2026/09/22 10:47:13 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 136.404547] nix-grpc-daemon[997]: time=2026-09-22T10:47:13.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 136.406139] nix-grpc-daemon[997]: time=2026-09-22T10:47:13.670Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 136.443034] niks3-server[1010]: 2026/09/22 10:47:13 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst worker2 # [ 136.455374] nix-grpc-daemon[997]: time=2026-09-22T10:47:13.723Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.454569] niks3-server[1010]: 2026/09/22 10:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 136.462870] niks3-server[1010]: 2026/09/22 10:47:13 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 136.469908] niks3-server[1010]: 2026/09/22 10:47:13 INFO Signed narinfos id=20 count=1 lb # [ 136.472962] niks3-server[1010]: 2026/09/22 10:47:13 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker2 # [ 136.482460] nix-grpc-daemon[997]: time=2026-09-22T10:47:13.750Z level=INFO msg="Upload complete. (348ms)" lb # [ 136.482687] niks3-server[1010]: 2026/09/22 10:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 136.486875] niks3-server[1010]: 2026/09/22 10:47:13 INFO Completed upload id=20 lb # [ 136.490387] niks3-server[1010]: 2026/09/22 10:47:13 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker2 # [ 136.496807] nix-grpc-daemon[867]: ts=2026-09-22T10:47:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=11 outputs=1 worker2 # [ 136.513733] nix-grpc-daemon[867]: ts=2026-09-22T10:47:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 136.519563] nix-daemon[947]: reaped child process 1215, status = succeeded worker2 # [ 136.539343] nix-grpc-daemon[867]: ts=2026-09-22T10:47:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 136.543961] nix-grpc-daemon[867]: ts=2026-09-22T10:47:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=0 paths=1 nar_bytes_out=232 client # [ 136.641294] nix-daemon[1051]: reaped child process 1053, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 9.62 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 9.76 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.53 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.24 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.31 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.05 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 138.108448] nix-daemon[1051]: accepted connection from pid 1134, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 138.469066] nix-grpc-daemon[867]: ts=2026-09-22T10:47:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 138.480674] nix-grpc-daemon[867]: ts=2026-09-22T10:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 138.464785] nix-grpc-daemon[860]: ts=2026-09-22T10:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 138.482625] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 138.714063] nix-daemon[921]: accepted connection from pid 1215, user nix-grpc-daemon (trusted) worker1 # [ 138.737429] nix-daemon[921]: reaped child process 1218, status = succeeded lb # [ 138.787576] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 138.765443] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.048Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 138.767054] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.048Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 138.822672] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 138.839849] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 138.811509] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.094Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.852366] niks3-server[1010]: 2026/09/22 10:47:16 INFO Signed narinfos id=21 count=1 lb # [ 138.857377] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 138.867222] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 138.875280] niks3-server[1010]: 2026/09/22 10:47:16 INFO Completed upload id=21 worker1 # [ 138.841276] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.124Z level=INFO msg="Upload complete. (343ms)" lb # [ 138.878143] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 138.852540] nix-grpc-daemon[860]: ts=2026-09-22T10:47:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 138.867814] nix-daemon[921]: reaped child process 1213, status = succeeded worker2 # [ 138.905441] nix-grpc-daemon[867]: ts=2026-09-22T10:47:16Z level=debug event=want client=13 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker2 # [ 138.910052] nix-grpc-daemon[867]: ts=2026-09-22T10:47:16Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 lb # [ 138.913827] envoy[961]: conn peer=[2001:db8:1::1]:54458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=33 lb # [ 138.919299] envoy[961]: conn peer=[2001:db8:1::1]:54442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3444 tx=1873 ms=2522 lb # [ 138.925239] envoy[961]: conn peer=[2001:db8:1::1]:54462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3903 tx=2088 ms=2945 lb # [ 138.930527] envoy[961]: conn peer=[2001:db8:1::1]:56884 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=73 lb # [ 138.934632] envoy[961]: conn peer=[2001:db8:1::1]:56910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=63 lb # [ 138.939989] envoy[961]: conn peer=[2001:db8:1::1]:56896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4356 tx=2206 ms=2635 worker1 # [ 138.920414] nix-grpc-daemon[860]: ts=2026-09-22T10:47:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 138.943344] nix-grpc-daemon[860]: ts=2026-09-22T10:47:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 138.969079] nix-grpc-daemon[860]: ts=2026-09-22T10:47:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 138.987180] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 139.237562] nix-daemon[921]: accepted connection from pid 1222, user nix-grpc-daemon (trusted) worker1 # [ 139.263181] nix-daemon[921]: reaped child process 1225, status = succeeded worker1 # [ 139.469128] nix-daemon[921]: accepted connection from pid 1227, user nix-grpc-daemon (trusted) worker1 # [ 139.493554] nix-daemon[921]: reaped child process 1230, status = succeeded lb # [ 139.537467] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 139.548481] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 139.524993] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.807Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 139.527023] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.807Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker1 # [ 139.528927] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.807Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 139.601360] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 139.611355] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 139.621202] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 139.631966] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 139.636335] niks3-server[1010]: 2026/09/22 10:47:16 INFO Signed narinfos id=23 count=2 lb # [ 139.639300] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker1 # [ 139.606209] nix-grpc-daemon[933]: time=2026-09-22T10:47:16.888Z level=INFO msg="Uploading 3 narinfos" lb # [ 139.642061] niks3-server[1010]: 2026/09/22 10:47:16 INFO Signed narinfos id=22 count=1 lb # [ 139.645823] niks3-server[1010]: 2026/09/22 10:47:16 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 139.666033] niks3-server[1010]: 2026/09/22 10:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 139.794692] niks3-server[1010]: 2026/09/22 10:47:17 INFO Completed upload id=22 lb # [ 139.802024] niks3-server[1010]: 2026/09/22 10:47:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 139.808325] niks3-server[1010]: 2026/09/22 10:47:17 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete worker1 # [ 139.779502] nix-grpc-daemon[933]: time=2026-09-22T10:47:17.058Z level=INFO msg="Upload complete. (754ms)" lb # [ 139.817994] niks3-server[1010]: 2026/09/22 10:47:17 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 139.825291] niks3-server[1010]: 2026/09/22 10:47:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 139.830853] niks3-server[1010]: 2026/09/22 10:47:17 INFO Completed upload id=23 worker1 # [ 139.794365] nix-grpc-daemon[860]: ts=2026-09-22T10:47:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=2 nar_bytes_in=984 worker1 # [ 139.817391] nix-grpc-daemon[860]: ts=2026-09-22T10:47:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 139.827618] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 139.839993] nix-daemon[921]: reaped child process 1220, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 139.864183] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 140.696245] nix-daemon[921]: reaped child process 1234, status = succeeded worker1 # [ 140.872528] nix-daemon[921]: accepted connection from pid 1250, user nix-grpc-daemon (trusted) worker1 # [ 140.894974] nix-daemon[921]: reaped child process 1253, status = succeeded lb # [ 140.945749] niks3-server[1010]: 2026/09/22 10:47:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 140.924561] nix-grpc-daemon[933]: time=2026-09-22T10:47:18.207Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 140.987997] niks3-server[1010]: 2026/09/22 10:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign worker1 # [ 140.963891] nix-grpc-daemon[933]: time=2026-09-22T10:47:18.246Z level=INFO msg="Uploading 1 narinfos" lb # [ 140.999539] niks3-server[1010]: 2026/09/22 10:47:18 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 141.011438] niks3-server[1010]: 2026/09/22 10:47:18 INFO Signed narinfos id=24 count=1 lb # [ 141.023120] niks3-server[1010]: 2026/09/22 10:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete worker1 # [ 140.996674] nix-grpc-daemon[933]: time=2026-09-22T10:47:18.279Z level=INFO msg="Upload complete. (317ms)" lb # [ 141.031468] niks3-server[1010]: 2026/09/22 10:47:18 INFO Completed upload id=24 lb # [ 141.036717] niks3-server[1010]: 2026/09/22 10:47:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker1 # [ 141.004637] nix-grpc-daemon[860]: ts=2026-09-22T10:47:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=12 outputs=1 lb # [ 141.042570] niks3-server[1010]: 2026/09/22 10:47:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls client # copying 1 paths... worker1 # [ 141.013832] nix-daemon[921]: reaped child process 1232, status = succeeded worker2 # [ 141.049578] nix-grpc-daemon[867]: ts=2026-09-22T10:47:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 141.068283] nix-grpc-daemon[867]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker1 # [ 141.092149] nix-grpc-daemon[860]: ts=2026-09-22T10:47:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 141.100168] nix-grpc-daemon[860]: ts=2026-09-22T10:47:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_out=120 client # [ 141.186335] nix-daemon[1051]: reaped child process 1137, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.28 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 141.430847] nix-daemon[1051]: accepted connection from pid 1156, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 141.689949] nix-grpc-daemon[860]: ts=2026-09-22T10:47:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34910 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 141.698336] nix-grpc-daemon[860]: ts=2026-09-22T10:47:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 141.705949] nix-grpc-daemon[933]: time=2026-09-22T10:47:18.989Z level=INFO msg="All 1 paths already cached" worker2 # [ 141.761868] nix-grpc-daemon[867]: ts=2026-09-22T10:47:19Z level=debug event=want client=14 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker2 # [ 141.769561] nix-grpc-daemon[867]: ts=2026-09-22T10:47:19Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 141.773324] nix-grpc-daemon[867]: ts=2026-09-22T10:47:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 141.798322] nix-grpc-daemon[867]: ts=2026-09-22T10:47:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40376 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 141.827405] nix-grpc-daemon[867]: ts=2026-09-22T10:47:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 141.853498] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 141.876525] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5) worker2 # [ 141.954429] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5) worker2 # [ 142.128855] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 3/5) worker2 # [ 142.199146] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 4/5) worker2 # [ 142.231914] nix-grpc-daemon[867]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 142.238316] nix-grpc-daemon[867]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 142.250306] nix-grpc-daemon[867]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 142.501328] nix-daemon[947]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) worker2 # [ 142.524786] nix-daemon[947]: reaped child process 1256, status = succeeded worker2 # [ 142.714392] nix-daemon[947]: accepted connection from pid 1258, user nix-grpc-daemon (trusted) worker2 # [ 142.748318] nix-daemon[947]: reaped child process 1261, status = succeeded lb # [ 142.755785] niks3-server[1010]: 2026/09/22 10:47:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 142.770065] nix-grpc-daemon[997]: time=2026-09-22T10:47:20.037Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 142.771681] nix-grpc-daemon[997]: time=2026-09-22T10:47:20.037Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 142.796365] niks3-server[1010]: 2026/09/22 10:47:20 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 142.818149] niks3-server[1010]: 2026/09/22 10:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker2 # [ 142.824942] nix-grpc-daemon[997]: time=2026-09-22T10:47:20.092Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.831396] niks3-server[1010]: 2026/09/22 10:47:20 INFO Signed narinfos id=25 count=1 lb # [ 142.837956] niks3-server[1010]: 2026/09/22 10:47:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 142.849655] niks3-server[1010]: 2026/09/22 10:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 142.857103] niks3-server[1010]: 2026/09/22 10:47:20 INFO Completed upload id=25 worker2 # [ 142.857739] nix-grpc-daemon[997]: time=2026-09-22T10:47:20.125Z level=INFO msg="Upload complete. (578ms)" lb # [ 142.859948] niks3-server[1010]: 2026/09/22 10:47:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker2 # [ 142.869250] nix-grpc-daemon[867]: ts=2026-09-22T10:47:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=1 paths=1 nar_bytes_in=568 worker2 # [ 142.883872] nix-grpc-daemon[867]: ts=2026-09-22T10:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 worker2 # [ 142.902248] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 142.906740] nix-daemon[947]: reaped child process 1246, status = succeeded worker2 # [ 142.931665] nix-grpc-daemon[867]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 142.961007] nix-daemon[947]: accepted connection from pid 867, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 144.229145] nix-daemon[947]: reaped child process 1266, status = succeeded worker2 # [ 144.416740] nix-daemon[947]: accepted connection from pid 1282, user nix-grpc-daemon (trusted) worker2 # [ 144.439164] nix-daemon[947]: reaped child process 1285, status = succeeded lb # [ 144.451323] niks3-server[1010]: 2026/09/22 10:47:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.480706] nix-grpc-daemon[997]: time=2026-09-22T10:47:21.745Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 144.502687] niks3-server[1010]: 2026/09/22 10:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker2 # [ 144.512450] nix-grpc-daemon[997]: time=2026-09-22T10:47:21.780Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.514934] niks3-server[1010]: 2026/09/22 10:47:21 INFO Signed narinfos id=26 count=1 lb # [ 144.523206] niks3-server[1010]: 2026/09/22 10:47:21 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 144.530083] niks3-server[1010]: 2026/09/22 10:47:21 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 144.538113] niks3-server[1010]: 2026/09/22 10:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 144.546567] niks3-server[1010]: 2026/09/22 10:47:21 INFO Completed upload id=26 lb # [ 144.548002] niks3-server[1010]: 2026/09/22 10:47:21 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo client # copying 1 paths... worker2 # [ 144.546707] nix-grpc-daemon[997]: time=2026-09-22T10:47:21.814Z level=INFO msg="Upload complete. (360ms)" worker2 # [ 144.561393] nix-grpc-daemon[867]: ts=2026-09-22T10:47:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40376 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=13 outputs=1 worker1 # [ 144.529534] nix-grpc-daemon[860]: ts=2026-09-22T10:47:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34910 worker2 # [ 144.575440] nix-daemon[947]: reaped child process 1263, status = succeeded worker1 # [ 144.544604] nix-grpc-daemon[860]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 144.603609] nix-grpc-daemon[860]: ts=2026-09-22T10:47:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 144.606355] nix-grpc-daemon[860]: ts=2026-09-22T10:47:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_out=120 client # [ 144.740578] nix-daemon[1051]: reaped child process 1159, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.55 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 144.962001] nix-daemon[1051]: accepted connection from pid 1178, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 145.256817] nix-grpc-daemon[860]: ts=2026-09-22T10:47:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 145.267958] nix-grpc-daemon[860]: ts=2026-09-22T10:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 145.278838] nix-grpc-daemon[933]: time=2026-09-22T10:47:22.559Z level=INFO msg="All 1 paths already cached" worker2 # [ 145.332521] nix-grpc-daemon[867]: ts=2026-09-22T10:47:22Z level=debug event=want client=15 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 145.340130] nix-grpc-daemon[867]: ts=2026-09-22T10:47:22Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 145.368349] nix-grpc-daemon[860]: ts=2026-09-22T10:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 145.384683] nix-grpc-daemon[860]: ts=2026-09-22T10:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 145.400641] nix-grpc-daemon[933]: time=2026-09-22T10:47:22.683Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 145.409614] nix-grpc-daemon[860]: ts=2026-09-22T10:47:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 145.431008] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 145.614963] nix-daemon[921]: accepted connection from pid 1258, user nix-grpc-daemon (trusted) worker1 # [ 145.641856] nix-daemon[921]: reaped child process 1261, status = succeeded worker1 # [ 145.832859] nix-daemon[921]: accepted connection from pid 1263, user nix-grpc-daemon (trusted) worker1 # [ 145.857834] nix-daemon[921]: reaped child process 1266, status = succeeded lb # [ 145.906551] niks3-server[1010]: 2026/09/22 10:47:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 145.888064] nix-grpc-daemon[933]: time=2026-09-22T10:47:23.170Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 145.889718] nix-grpc-daemon[933]: time=2026-09-22T10:47:23.170Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 145.945825] niks3-server[1010]: 2026/09/22 10:47:23 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker1 # [ 145.928604] nix-grpc-daemon[933]: time=2026-09-22T10:47:23.211Z level=INFO msg="Uploading 1 narinfos" lb # [ 145.966922] niks3-server[1010]: 2026/09/22 10:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 145.976275] niks3-server[1010]: 2026/09/22 10:47:23 INFO Signed narinfos id=27 count=1 lb # [ 145.980679] niks3-server[1010]: 2026/09/22 10:47:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 145.982222] niks3-server[1010]: 2026/09/22 10:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete worker1 # [ 145.946551] nix-grpc-daemon[933]: time=2026-09-22T10:47:23.229Z level=INFO msg="Upload complete. (503ms)" lb # [ 145.986811] niks3-server[1010]: 2026/09/22 10:47:23 INFO Completed upload id=27 worker1 # [ 145.952751] nix-grpc-daemon[860]: ts=2026-09-22T10:47:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 145.990011] niks3-server[1010]: 2026/09/22 10:47:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker1 # [ 145.963924] nix-grpc-daemon[860]: ts=2026-09-22T10:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 145.972864] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 145.976933] nix-daemon[921]: reaped child process 1256, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 145.992260] nix-daemon[921]: accepted connection from pid 860, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 147.247251] nix-daemon[921]: reaped child process 1270, status = succeeded worker1 # [ 147.454320] nix-daemon[921]: accepted connection from pid 1286, user nix-grpc-daemon (trusted) worker1 # [ 147.479843] nix-daemon[921]: reaped child process 1289, status = succeeded lb # [ 147.529569] niks3-server[1010]: 2026/09/22 10:47:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 147.504968] nix-grpc-daemon[933]: time=2026-09-22T10:47:24.787Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 147.562839] niks3-server[1010]: 2026/09/22 10:47:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign worker1 # [ 147.540481] nix-grpc-daemon[933]: time=2026-09-22T10:47:24.823Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.579056] niks3-server[1010]: 2026/09/22 10:47:24 INFO Signed narinfos id=28 count=1 lb # [ 147.586491] niks3-server[1010]: 2026/09/22 10:47:24 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 147.595304] niks3-server[1010]: 2026/09/22 10:47:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 147.603880] niks3-server[1010]: 2026/09/22 10:47:24 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 147.608967] niks3-server[1010]: 2026/09/22 10:47:24 INFO Completed upload id=28 worker1 # [ 147.574239] nix-grpc-daemon[933]: time=2026-09-22T10:47:24.857Z level=INFO msg="Upload complete. (367ms)" lb # [ 147.611795] niks3-server[1010]: 2026/09/22 10:47:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker1 # [ 147.583301] nix-grpc-daemon[860]: ts=2026-09-22T10:47:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=14 outputs=1 client # copying 1 paths... worker2 # [ 147.627049] nix-grpc-daemon[867]: ts=2026-09-22T10:47:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40376 worker1 # [ 147.598898] nix-daemon[921]: reaped child process 1268, status = succeeded worker2 # [ 147.642144] nix-grpc-daemon[867]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 147.656721] nix-grpc-daemon[860]: ts=2026-09-22T10:47:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 worker1 # [ 147.663736] nix-grpc-daemon[860]: ts=2026-09-22T10:47:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34910 duration_s=0 paths=1 nar_bytes_out=120 client # [ 147.773385] nix-daemon[1051]: reaped child process 1181, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.04 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 11.00 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.56 seconds) client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.52 seconds) lb # [ 148.913925] envoy[961]: conn peer=[2001:db8:1::1]:56926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4232 tx=1963 ms=2689 client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure lb # [ 148.919118] envoy[961]: conn peer=[2001:db8:1::1]:41698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=34 lb # [ 148.924188] envoy[961]: conn peer=[2001:db8:1::1]:41700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2015 ms=2978 lb # [ 148.930145] envoy[961]: conn peer=[2001:db8:1::1]:41706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=67 lb # [ 148.935018] envoy[961]: conn peer=[2001:db8:1::1]:41722 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3784 tx=2005 ms=2427 lb # [ 148.939976] envoy[961]: conn peer=[2001:db8:1::1]:41730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=30 client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.23 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds) worker2 # [ 149.479209] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 149.791009] nix-grpc-daemon[867]: ts=2026-09-22T10:47:27Z level=info event=signal_exit lb # [ 149.809247] niks3-server[1010]: 2026/09/22 10:47:27 INFO lead: released remote=192.168.1.4:48870 worker1 # [ 150.776999] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 150.815653] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=scheduler_restarting worker2 # [ 150.821433] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 150.829708] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50082 worker2 # [ 150.835230] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 150.839837] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50082 worker2 # [ 150.845249] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 150.884653] nix-daemon[947]: reaped child process 949, status = succeeded worker1 # [ 150.978745] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 150.985217] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 151.051994] nix-grpc-daemon[867]: ts=2026-09-22T10:47:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 151.183475] nix-grpc-daemon[867]: E0000 00:00:1790074048.451101 912 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker2 # [ 151.191953] nix-grpc-daemon[867]: E0000 00:00:1790074048.451457 912 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 151.186577] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.193120] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 151.394364] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.400946] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 151.602079] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.606383] nix-grpc-daemon[860]: ts=2026-09-22T10:47:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 151.807954] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.811363] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 151.878103] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 151.883416] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 151.893300] systemd[1]: nix-grpc-daemon.service: Consumed 8.417s CPU time over 1min 39.092s wall clock time, 59.2M memory peak, 72.9K incoming IP traffic, 173.2K outgoing IP traffic. worker2 # [ 151.906429] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 151.915003] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 151.918291] systemd[1]: nix-grpc-daemon.socket: Consumed 875us CPU time over 2min 14.952s wall clock time, 256K memory peak, 102.1K incoming IP traffic, 80.7K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 152.012732] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.018449] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 152.219277] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.227226] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 152.285261] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34910 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker1 # [ 152.295823] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34910 client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.37 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds) worker1 # [ 152.428614] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.436156] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 152.508148] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 152.591030] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 152.637204] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.641288] nix-grpc-daemon[860]: ts=2026-09-22T10:47:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 152.816814] systemd[1]: Starting scheduler-token.service... worker1 # [ 152.845100] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.851831] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 152.909612] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=signal_exit worker2 # [ 152.949525] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.954558] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.960412] systemd[1]: scheduler-token.service: Consumed 72ms CPU time over 137ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.990176] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 153.054487] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 153.070485] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 153.274835] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 153.281079] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 153.485937] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 153.489531] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 153.494593] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 153.504118] nix-grpc-daemon[1312]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 153.506359] nix-grpc-daemon[1312]: E0000 00:00:1790074050.769610 1312 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 153.483041] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 153.522068] nix-grpc-daemon[1312]: E0000 00:00:1790074050.772019 1312 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 153.490983] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 153.531126] niks3-server[1010]: 2026/09/22 10:47:30 INFO lead: acquired remote=192.168.1.4:59914 worker2 # [ 153.529902] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 153.535807] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 153.537212] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=scheduler_take_over worker2 # [ 153.544121] nix-grpc-daemon[1312]: E0000 00:00:1790074050.808354 1351 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 153.568345] nix-grpc-daemon[1312]: E0000 00:00:1790074050.826646 1351 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 153.579438] nix-grpc-daemon[1312]: E0000 00:00:1790074050.826909 1351 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 153.582872] nix-grpc-daemon[1312]: E0000 00:00:1790074050.846212 1351 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 153.599756] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 153.603039] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 153.691735] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 153.694749] nix-grpc-daemon[860]: ts=2026-09-22T10:47:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 153.804175] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 153.807243] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 153.895945] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 153.899094] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 154.008761] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 154.012377] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 154.099727] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 154.102747] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 154.213287] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 154.216391] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 154.303822] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 154.309531] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 154.375580] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=scheduler_restarting worker2 # [ 154.417342] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 154.419833] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 154.420657] nix-daemon[921]: reaped child process 923, status = succeeded worker1 # [ 154.527455] nix-grpc-daemon[860]: ts=2026-09-22T10:47:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 154.621921] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 154.682214] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 154.683604] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:42000 worker2 # [ 154.685641] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 155.167525] nix-grpc-daemon[860]: E0000 00:00:1790074052.448564 906 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true worker1 # [ 155.177303] nix-grpc-daemon[860]: E0000 00:00:1790074052.449524 906 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 155.699908] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 155.703055] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 155.703960] systemd[1]: nix-grpc-daemon.service: Consumed 8.502s CPU time over 1min 43.077s wall clock time, 59.5M memory peak, 92.5K incoming IP traffic, 161K outgoing IP traffic. worker1 # [ 155.715858] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 155.718959] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 155.721859] systemd[1]: nix-grpc-daemon.socket: Consumed 817us CPU time over 2min 18.214s wall clock time, 256K memory peak, 93.2K incoming IP traffic, 75.8K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.06 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 156.109259] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 156.120253] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 156.139899] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5) worker2 # [ 156.155652] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5) worker2 # [ 156.161017] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 3/5) worker2 # [ 156.444616] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 649 ms (attempt 4/5) worker2 # [ 157.108131] nix-grpc-daemon[1312]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 157.108462] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 157.144350] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 157.154869] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 157.186401] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:34Z level=info event=niks3_push_started pid=1372 worker2 # [ 157.469886] nix-grpc-daemon[1372]: time=2026-09-22T10:47:34.735Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 158.249835] nix-daemon[947]: accepted connection from pid 1385, user nix-grpc-daemon (trusted) worker2 # [ 158.306265] nix-daemon[947]: reaped child process 1388, status = succeeded lb # [ 158.387003] niks3-server[1010]: 2026/09/22 10:47:35 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 158.470406] niks3-server[1010]: 2026/09/22 10:47:35 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 159.001088] envoy[961]: conn peer=[2001:db8:1::4]:35954 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=31922 tx=2141 ms=97638 worker2 # [ 158.982811] nix-grpc-daemon[1372]: time=2026-09-22T10:47:36.191Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 159.130074] nix-grpc-daemon[1372]: time=2026-09-22T10:47:36.213Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 159.360151] envoy[961]: conn peer=[2001:db8:1::1]:37446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=28 lb # [ 159.473323] envoy[961]: conn peer=[2001:db8:1::3]:59170 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9199 tx=2831 ms=101227 worker2 # [ 159.288675] nix-grpc-daemon[1372]: time=2026-09-22T10:47:36.472Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 159.961327] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 159.971732] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 159.980305] niks3-server[1010]: 2026/09/22 10:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 159.982406] nix-grpc-daemon[1372]: time=2026-09-22T10:47:37.248Z level=INFO msg="Uploading 3 narinfos" lb # [ 159.986157] niks3-server[1010]: 2026/09/22 10:47:37 INFO Signed narinfos id=30 count=2 lb # [ 159.992263] niks3-server[1010]: 2026/09/22 10:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 159.997147] niks3-server[1010]: 2026/09/22 10:47:37 INFO Signed narinfos id=29 count=1 lb # [ 159.998363] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 160.020883] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 160.033357] niks3-server[1010]: 2026/09/22 10:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 160.037787] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 160.042970] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 160.046031] niks3-server[1010]: 2026/09/22 10:47:37 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 160.049360] niks3-server[1010]: 2026/09/22 10:47:37 INFO Completed upload id=30 lb # [ 160.051834] niks3-server[1010]: 2026/09/22 10:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 160.054263] niks3-server[1010]: 2026/09/22 10:47:37 INFO Completed upload id=29 worker2 # [ 160.054602] nix-grpc-daemon[1372]: time=2026-09-22T10:47:37.322Z level=INFO msg="Upload complete. (2.585s)" worker2 # [ 160.065299] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=2 paths=2 nar_bytes_in=312 worker2 # [ 160.069414] nix-daemon[947]: reaped child process 1370, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 4.25 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.02 seconds) worker1 # [ 160.148814] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 160.154064] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 160.353912] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 160.356982] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 160.361828] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 160.376127] nix-grpc-daemon[1312]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 160.377778] nix-grpc-daemon[1312]: E0000 00:00:1790074057.656476 1312 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 160.398715] nix-grpc-daemon[1312]: E0000 00:00:1790074057.659059 1312 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 160.405442] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 160.446031] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 160.447913] nix-grpc-daemon[1312]: E0000 00:00:1790074057.689636 1354 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 160.451671] nix-grpc-daemon[1312]: E0000 00:00:1790074057.708728 1354 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 160.458817] nix-grpc-daemon[1312]: E0000 00:00:1790074057.709125 1354 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 160.464998] nix-grpc-daemon[1312]: E0000 00:00:1790074057.728038 1354 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 160.468768] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 160.500674] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:42000 worker2 # [ 160.507572] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.22 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.22 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.86 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb # [ 160.837715] systemd[1]: Stopping niks3 server... lb # [ 160.838042] niks3-server[1010]: 2026/09/22 10:47:38 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 160.843936] niks3-server[1010]: 2026/09/22 10:47:38 INFO lead: released remote=192.168.1.4:59914 worker1 # [ 160.808413] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:38Z level=info event=scheduler_lock_lost status=200 worker2 # [ 160.843252] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:38Z level=info event=scheduler_lock_lost status=200 lb # [ 165.144350] systemd[1]: niks3.service: Deactivated successfully. lb # [ 165.146888] systemd[1]: Stopped niks3 server. lb # [ 165.150780] systemd[1]: niks3.service: Consumed 2.410s CPU time over 1min 52.023s wall clock time, 22.6M memory peak, 77.5K incoming IP traffic, 112.8K outgoing IP traffic. lb # [ 165.173309] systemd[1]: Starting niks3 server... lb # [ 165.260649] niks3-server[1133]: 2026/09/22 10:47:42 goose: no migrations to run. current version: 20260920000000 lb # [ 165.268959] niks3-server[1133]: 2026/09/22 10:47:42 OK 1_commit_pending_closure.sql (9.02ms) lb # [ 165.273416] niks3-server[1133]: 2026/09/22 10:47:42 OK 2_object_stats_trigger.sql (2.5ms) lb # [ 165.274857] niks3-server[1133]: 2026/09/22 10:47:42 goose: up to current file version: 2 lb # [ 165.276885] niks3-server[1133]: 2026/09/22 10:47:42 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key lb # [ 165.292918] niks3-server[1133]: 2026/09/22 10:47:42 INFO Read proxy enabled — serving cache objects from S3 lb # [ 165.294433] niks3-server[1133]: 2026/09/22 10:47:42 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 165.302797] niks3-server[1133]: 2026/09/22 10:47:42 INFO systemd watchdog enabled interval=15s lb # [ 165.305693] niks3-server[1133]: 2026/09/22 10:47:42 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 165.307178] niks3-server[1133]: 2026/09/22 10:47:42 INFO lead: acquired remote=192.168.1.4:34284 lb # [ 165.308574] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 165.945075] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 165.965813] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 166.032560] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 166.094563] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 166.105140] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 166.143754] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 166.169109] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42006 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 166.192260] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 166.213407] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 166.444717] nix-daemon[947]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 166.469392] nix-daemon[947]: reaped child process 1410, status = succeeded worker2 # [ 166.653930] nix-daemon[947]: accepted connection from pid 1412, user nix-grpc-daemon (trusted) worker2 # [ 166.686446] nix-daemon[947]: reaped child process 1415, status = succeeded lb # [ 166.701017] niks3-server[1133]: 2026/09/22 10:47:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 166.729879] nix-grpc-daemon[1372]: time=2026-09-22T10:47:43.997Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 166.732445] nix-grpc-daemon[1372]: time=2026-09-22T10:47:43.997Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 166.758020] niks3-server[1133]: 2026/09/22 10:47:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 166.775507] niks3-server[1133]: 2026/09/22 10:47:44 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 166.793890] niks3-server[1133]: 2026/09/22 10:47:44 INFO Signed narinfos id=31 count=1 worker2 # [ 166.798278] nix-grpc-daemon[1372]: time=2026-09-22T10:47:44.066Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.808314] niks3-server[1133]: 2026/09/22 10:47:44 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 166.815701] niks3-server[1133]: 2026/09/22 10:47:44 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 166.824315] niks3-server[1133]: 2026/09/22 10:47:44 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo lb # [ 166.834249] niks3-server[1133]: 2026/09/22 10:47:44 INFO Completed upload id=31 worker2 # [ 166.835047] nix-grpc-daemon[1372]: time=2026-09-22T10:47:44.103Z level=INFO msg="Upload complete. (601ms)" worker2 # [ 166.840384] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 166.845256] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 166.858484] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 166.863330] nix-daemon[947]: reaped child process 1405, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 166.878965] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 167.737838] nix-daemon[947]: reaped child process 1419, status = succeeded worker2 # [ 167.995238] nix-daemon[947]: accepted connection from pid 1435, user nix-grpc-daemon (trusted) worker2 # [ 168.021344] nix-daemon[947]: reaped child process 1438, status = succeeded lb # [ 168.034519] niks3-server[1133]: 2026/09/22 10:47:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.055559] nix-grpc-daemon[1372]: time=2026-09-22T10:47:45.320Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 168.059219] nix-grpc-daemon[1372]: time=2026-09-22T10:47:45.320Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 168.082483] niks3-server[1133]: 2026/09/22 10:47:45 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 168.088779] niks3-server[1133]: 2026/09/22 10:47:45 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 168.094741] niks3-server[1133]: 2026/09/22 10:47:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker2 # [ 168.097228] nix-grpc-daemon[1372]: time=2026-09-22T10:47:45.364Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.100573] niks3-server[1133]: 2026/09/22 10:47:45 INFO Signed narinfos id=32 count=1 lb # [ 168.104615] niks3-server[1133]: 2026/09/22 10:47:45 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 168.110257] niks3-server[1133]: 2026/09/22 10:47:45 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 168.121230] niks3-server[1133]: 2026/09/22 10:47:45 INFO Completed upload id=32 worker2 # [ 168.121825] nix-grpc-daemon[1372]: time=2026-09-22T10:47:45.389Z level=INFO msg="Upload complete. (423ms)" lb # [ 168.126406] niks3-server[1133]: 2026/09/22 10:47:45 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 168.132272] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 168.139523] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 168.144322] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 168.160343] nix-daemon[947]: reaped child process 1417, status = succeeded worker2 # [ 168.184391] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 168.195364] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 168.211099] nix-grpc-daemon[1372]: time=2026-09-22T10:47:45.479Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 168.220824] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 168.227929] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 168.415496] nix-daemon[947]: accepted connection from pid 1444, user nix-grpc-daemon (trusted) worker2 # [ 168.461074] nix-daemon[947]: reaped child process 1447, status = succeeded worker2 # [ 168.669337] nix-daemon[947]: accepted connection from pid 1449, user nix-grpc-daemon (trusted) worker2 # [ 168.694057] nix-daemon[947]: reaped child process 1452, status = succeeded lb # [ 168.741046] niks3-server[1133]: 2026/09/22 10:47:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.761431] nix-grpc-daemon[1372]: time=2026-09-22T10:47:46.029Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 168.763766] nix-grpc-daemon[1372]: time=2026-09-22T10:47:46.029Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 168.775953] niks3-server[1133]: 2026/09/22 10:47:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker2 # [ 168.783294] nix-grpc-daemon[1372]: time=2026-09-22T10:47:46.051Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.785000] niks3-server[1133]: 2026/09/22 10:47:46 INFO Signed narinfos id=33 count=1 lb # [ 168.790815] niks3-server[1133]: 2026/09/22 10:47:46 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 168.800239] niks3-server[1133]: 2026/09/22 10:47:46 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 168.806276] niks3-server[1133]: 2026/09/22 10:47:46 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 168.811505] niks3-server[1133]: 2026/09/22 10:47:46 INFO Completed upload id=33 worker2 # [ 168.812896] nix-grpc-daemon[1372]: time=2026-09-22T10:47:46.081Z level=INFO msg="Upload complete. (575ms)" lb # [ 168.820367] niks3-server[1133]: 2026/09/22 10:47:46 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 168.820402] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 168.828592] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 168.842069] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 168.846185] nix-daemon[947]: reaped child process 1442, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 168.856181] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) lb # [ 168.948882] envoy[961]: conn peer=[2001:db8:1::1]:37460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=305 ms=3971 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 170.300939] nix-daemon[947]: reaped child process 1456, status = succeeded worker2 # [ 170.685676] nix-daemon[947]: accepted connection from pid 1472, user nix-grpc-daemon (trusted) worker2 # [ 170.720734] nix-daemon[947]: reaped child process 1475, status = succeeded lb # [ 170.972743] niks3-server[1133]: 2026/09/22 10:47:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.992705] nix-grpc-daemon[1372]: time=2026-09-22T10:47:48.259Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 170.994816] nix-grpc-daemon[1372]: time=2026-09-22T10:47:48.259Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 171.037544] niks3-server[1133]: 2026/09/22 10:47:48 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 171.050513] niks3-server[1133]: 2026/09/22 10:47:48 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst worker2 # [ 171.052316] nix-grpc-daemon[1372]: time=2026-09-22T10:47:48.320Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.055503] niks3-server[1133]: 2026/09/22 10:47:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 171.070049] niks3-server[1133]: 2026/09/22 10:47:48 INFO Signed narinfos id=34 count=1 lb # [ 171.071807] niks3-server[1133]: 2026/09/22 10:47:48 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 171.080311] niks3-server[1133]: 2026/09/22 10:47:48 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 171.089471] niks3-server[1133]: 2026/09/22 10:47:48 INFO Completed upload id=34 worker2 # [ 171.090647] nix-grpc-daemon[1372]: time=2026-09-22T10:47:48.358Z level=INFO msg="Upload complete. (818ms)" lb # [ 171.094399] niks3-server[1133]: 2026/09/22 10:47:48 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 171.100636] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=2 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 171.109820] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 171.112541] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 171.125034] nix-daemon[947]: reaped child process 1454, status = succeeded worker2 # [ 171.158577] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 171.176729] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 171.202825] nix-grpc-daemon[1372]: time=2026-09-22T10:47:48.470Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 171.211989] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 171.228500] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 171.853391] nix-daemon[947]: accepted connection from pid 1481, user nix-grpc-daemon (trusted) worker2 # [ 171.891608] nix-daemon[947]: reaped child process 1484, status = succeeded worker2 # [ 172.421870] nix-daemon[947]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker2 # [ 172.459130] nix-daemon[947]: reaped child process 1489, status = succeeded lb # [ 172.488343] niks3-server[1133]: 2026/09/22 10:47:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 172.506311] nix-grpc-daemon[1372]: time=2026-09-22T10:47:49.773Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 172.509644] nix-grpc-daemon[1372]: time=2026-09-22T10:47:49.777Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 172.526929] niks3-server[1133]: 2026/09/22 10:47:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 172.534237] niks3-server[1133]: 2026/09/22 10:47:49 INFO Signed narinfos id=35 count=1 worker2 # [ 172.540484] nix-grpc-daemon[1372]: time=2026-09-22T10:47:49.808Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.541434] niks3-server[1133]: 2026/09/22 10:47:49 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 172.547890] niks3-server[1133]: 2026/09/22 10:47:49 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 172.561038] niks3-server[1133]: 2026/09/22 10:47:49 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 172.574048] niks3-server[1133]: 2026/09/22 10:47:49 INFO Completed upload id=35 worker2 # [ 172.575186] nix-grpc-daemon[1372]: time=2026-09-22T10:47:49.843Z level=INFO msg="Upload complete. (1.332s)" worker2 # [ 172.580826] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=1 paths=1 nar_bytes_in=784 lb # [ 172.581626] niks3-server[1133]: 2026/09/22 10:47:49 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 172.588837] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 worker2 # [ 172.605056] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker2 # [ 172.607845] nix-daemon[947]: reaped child process 1479, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 172.621109] nix-daemon[947]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 175.077190] nix-daemon[947]: reaped child process 1493, status = succeeded worker2 # [ 175.709091] nix-daemon[947]: accepted connection from pid 1509, user nix-grpc-daemon (trusted) worker2 # [ 175.753920] nix-daemon[947]: reaped child process 1512, status = succeeded lb # [ 175.797631] niks3-server[1133]: 2026/09/22 10:47:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 175.812976] nix-grpc-daemon[1372]: time=2026-09-22T10:47:53.080Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 175.817065] nix-grpc-daemon[1372]: time=2026-09-22T10:47:53.082Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 175.843246] niks3-server[1133]: 2026/09/22 10:47:53 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 175.863779] niks3-server[1133]: 2026/09/22 10:47:53 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker2 # [ 175.871484] nix-grpc-daemon[1372]: time=2026-09-22T10:47:53.138Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.874106] niks3-server[1133]: 2026/09/22 10:47:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 175.881136] niks3-server[1133]: 2026/09/22 10:47:53 INFO Signed narinfos id=36 count=1 lb # [ 175.887547] niks3-server[1133]: 2026/09/22 10:47:53 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 175.894551] niks3-server[1133]: 2026/09/22 10:47:53 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 175.901164] niks3-server[1133]: 2026/09/22 10:47:53 INFO Completed upload id=36 worker2 # [ 175.903399] nix-grpc-daemon[1372]: time=2026-09-22T10:47:53.171Z level=INFO msg="Upload complete. (855ms)" lb # [ 175.906369] niks3-server[1133]: 2026/09/22 10:47:53 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 175.908753] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42006 duration_s=3 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 175.933081] nix-daemon[947]: reaped child process 1491, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 10.28 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.36 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 15.63 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 176.623637] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 177.546750] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:54Z level=info event=signal_exit lb # [ 178.082902] niks3-server[1133]: 2026/09/22 10:47:55 INFO lead: released remote=192.168.1.4:34284 lb # [ 178.949264] envoy[961]: conn peer=[2001:db8:1::1]:42784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9882 tx=4183 ms=9971 worker2 # [ 179.075238] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_restarting worker1 # [ 179.044559] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 179.083986] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 179.093322] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:42000 worker2 # [ 179.098084] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 179.100887] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:42000 worker2 # [ 179.103712] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 179.144802] nix-daemon[947]: reaped child process 1364, status = succeeded worker2 # [ 179.192928] nix-grpc-daemon[1312]: E0000 00:00:1790074076.460916 1357 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker2 # [ 179.238928] nix-grpc-daemon[1312]: E0000 00:00:1790074076.506934 1357 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 179.244948] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 179.248004] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 179.307395] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 179.448979] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 179.454865] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 179.541956] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 179.548391] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 179.550000] systemd[1]: nix-grpc-daemon.service: Consumed 2.980s CPU time over 26.551s wall clock time, 36.1M memory peak, 49.8K incoming IP traffic, 81.8K outgoing IP traffic. worker2 # [ 179.557926] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 179.561197] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 179.562135] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 27.068s wall clock time, 256K memory peak, 44.5K incoming IP traffic, 33.9K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 179.655848] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 179.658702] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 179.862316] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 179.867300] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 180.069123] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 180.071560] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 180.266199] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_take_over lb # [ 180.300820] niks3-server[1133]: 2026/09/22 10:47:57 INFO lead: acquired remote=192.168.1.3:55974 worker1 # [ 180.272264] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 180.275391] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 180.476505] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 180.484848] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:37692 worker1 # [ 180.486777] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.13 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2 worker1 # [ 181.014195] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52008 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 181.026924] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 181.045159] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5) worker1 # [ 181.134508] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5) worker1 # [ 181.248899] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 3/5) worker1 # [ 181.393286] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 4/5) worker1 # [ 181.572900] nix-grpc-daemon[1312]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 181.574392] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 181.609857] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 181.614519] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.660942] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 181.679981] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52008 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 181.702161] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 181.719968] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 181.744402] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:59Z level=info event=niks3_push_started pid=1391 worker1 # [ 181.964608] nix-grpc-daemon[1391]: time=2026-09-22T10:47:59.246Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 182.157813] nix-daemon[921]: accepted connection from pid 1404, user nix-grpc-daemon (trusted) worker1 # [ 182.182915] nix-daemon[921]: reaped child process 1407, status = succeeded worker1 # [ 182.383184] nix-daemon[921]: accepted connection from pid 1409, user nix-grpc-daemon (trusted) worker1 # [ 182.405978] nix-daemon[921]: reaped child process 1412, status = succeeded lb # [ 182.457368] niks3-server[1133]: 2026/09/22 10:47:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 182.442380] nix-grpc-daemon[1391]: time=2026-09-22T10:47:59.723Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 182.444754] nix-grpc-daemon[1391]: time=2026-09-22T10:47:59.723Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 182.508477] niks3-server[1133]: 2026/09/22 10:47:59 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst worker1 # [ 182.502506] nix-grpc-daemon[1391]: time=2026-09-22T10:47:59.784Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.529986] niks3-server[1133]: 2026/09/22 10:47:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 182.540880] niks3-server[1133]: 2026/09/22 10:47:59 INFO Signed narinfos id=37 count=1 lb # [ 182.553477] niks3-server[1133]: 2026/09/22 10:47:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 182.563387] niks3-server[1133]: 2026/09/22 10:47:59 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 182.572384] niks3-server[1133]: 2026/09/22 10:47:59 INFO Completed upload id=37 worker1 # [ 182.536821] nix-grpc-daemon[1391]: time=2026-09-22T10:47:59.819Z level=INFO msg="Upload complete. (568ms)" lb # [ 182.575390] niks3-server[1133]: 2026/09/22 10:47:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 182.549462] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 182.561113] nix-grpc-daemon[1312]: ts=2026-09-22T10:47:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 182.572850] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 182.578307] nix-daemon[921]: reaped child process 1389, status = succeeded worker1 # [ 182.591615] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 183.625825] nix-daemon[921]: reaped child process 1416, status = succeeded worker1 # [ 183.799035] nix-daemon[921]: accepted connection from pid 1432, user nix-grpc-daemon (trusted) worker1 # [ 183.825231] nix-daemon[921]: reaped child process 1435, status = succeeded lb # [ 183.874390] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.851075] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.133Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 183.859436] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.136Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 183.919966] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 183.934548] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 183.951013] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker1 # [ 183.924263] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.206Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.962293] niks3-server[1133]: 2026/09/22 10:48:01 INFO Signed narinfos id=38 count=1 lb # [ 183.968274] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 183.978967] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete worker1 # [ 183.951120] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.234Z level=INFO msg="Upload complete. (342ms)" lb # [ 183.987845] niks3-server[1133]: 2026/09/22 10:48:01 INFO Completed upload id=38 lb # [ 183.990267] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 183.974655] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 183.995422] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 183.999248] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 184.003564] nix-daemon[921]: reaped child process 1414, status = succeeded worker1 # [ 184.035206] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 184.051387] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 184.064190] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.347Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 184.074070] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 184.087264] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 184.299129] nix-daemon[921]: accepted connection from pid 1441, user nix-grpc-daemon (trusted) worker1 # [ 184.325935] nix-daemon[921]: reaped child process 1444, status = succeeded worker1 # [ 184.520098] nix-daemon[921]: accepted connection from pid 1446, user nix-grpc-daemon (trusted) worker1 # [ 184.541514] nix-daemon[921]: reaped child process 1449, status = succeeded lb # [ 184.591036] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.568994] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.851Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 184.570687] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.851Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 184.626707] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker1 # [ 184.603976] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.886Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.647261] niks3-server[1133]: 2026/09/22 10:48:01 INFO Signed narinfos id=39 count=1 lb # [ 184.654877] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 184.677032] niks3-server[1133]: 2026/09/22 10:48:01 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete worker1 # [ 184.656400] nix-grpc-daemon[1391]: time=2026-09-22T10:48:01.939Z level=INFO msg="Upload complete. (554ms)" lb # [ 184.690374] niks3-server[1133]: 2026/09/22 10:48:01 INFO Completed upload id=39 lb # [ 184.694997] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo lb # [ 184.703120] niks3-server[1133]: 2026/09/22 10:48:01 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls worker1 # [ 184.667561] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 184.680494] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 184.693966] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 184.700742] nix-daemon[921]: reaped child process 1439, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 184.719656] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 186.154850] nix-daemon[921]: reaped child process 1453, status = succeeded worker1 # [ 186.348186] nix-daemon[921]: accepted connection from pid 1469, user nix-grpc-daemon (trusted) worker1 # [ 186.369901] nix-daemon[921]: reaped child process 1472, status = succeeded lb # [ 186.422632] niks3-server[1133]: 2026/09/22 10:48:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.405612] nix-grpc-daemon[1391]: time=2026-09-22T10:48:03.687Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 186.407253] nix-grpc-daemon[1391]: time=2026-09-22T10:48:03.687Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 186.471649] niks3-server[1133]: 2026/09/22 10:48:03 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst worker1 # [ 186.462229] nix-grpc-daemon[1391]: time=2026-09-22T10:48:03.745Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.495712] niks3-server[1133]: 2026/09/22 10:48:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 186.508727] niks3-server[1133]: 2026/09/22 10:48:03 INFO Signed narinfos id=40 count=1 lb # [ 186.514770] niks3-server[1133]: 2026/09/22 10:48:03 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 186.524372] niks3-server[1133]: 2026/09/22 10:48:03 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 186.533451] niks3-server[1133]: 2026/09/22 10:48:03 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 186.538588] niks3-server[1133]: 2026/09/22 10:48:03 INFO Completed upload id=40 worker1 # [ 186.504132] nix-grpc-daemon[1391]: time=2026-09-22T10:48:03.787Z level=INFO msg="Upload complete. (371ms)" lb # [ 186.541244] niks3-server[1133]: 2026/09/22 10:48:03 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 186.524583] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 186.540300] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 186.543984] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 186.549250] nix-daemon[921]: reaped child process 1451, status = succeeded worker1 # [ 186.567065] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 186.581249] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 186.594195] nix-grpc-daemon[1391]: time=2026-09-22T10:48:03.877Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 186.604544] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 186.614450] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 186.805500] nix-daemon[921]: accepted connection from pid 1478, user nix-grpc-daemon (trusted) worker1 # [ 186.832161] nix-daemon[921]: reaped child process 1481, status = succeeded worker1 # [ 187.024597] nix-daemon[921]: accepted connection from pid 1483, user nix-grpc-daemon (trusted) worker1 # [ 187.050579] nix-daemon[921]: reaped child process 1486, status = succeeded lb # [ 187.097621] niks3-server[1133]: 2026/09/22 10:48:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.075404] nix-grpc-daemon[1391]: time=2026-09-22T10:48:04.358Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 187.077820] nix-grpc-daemon[1391]: time=2026-09-22T10:48:04.360Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" worker1 # [ 187.112592] nix-grpc-daemon[1391]: time=2026-09-22T10:48:04.395Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.138143] niks3-server[1133]: 2026/09/22 10:48:04 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 187.151014] niks3-server[1133]: 2026/09/22 10:48:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 187.163320] niks3-server[1133]: 2026/09/22 10:48:04 INFO Signed narinfos id=41 count=1 lb # [ 187.168467] niks3-server[1133]: 2026/09/22 10:48:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 187.178228] niks3-server[1133]: 2026/09/22 10:48:04 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker1 # [ 187.148366] nix-grpc-daemon[1391]: time=2026-09-22T10:48:04.431Z level=INFO msg="Upload complete. (525ms)" lb # [ 187.183517] niks3-server[1133]: 2026/09/22 10:48:04 INFO Completed upload id=41 lb # [ 187.186627] niks3-server[1133]: 2026/09/22 10:48:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 187.156206] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 187.168462] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 187.183503] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 187.189622] nix-daemon[921]: reaped child process 1476, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 187.206093] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 188.632833] nix-daemon[921]: reaped child process 1490, status = succeeded worker1 # [ 188.808787] nix-daemon[921]: accepted connection from pid 1506, user nix-grpc-daemon (trusted) worker1 # [ 188.830162] nix-daemon[921]: reaped child process 1509, status = succeeded lb # [ 188.881060] niks3-server[1133]: 2026/09/22 10:48:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.859498] nix-grpc-daemon[1391]: time=2026-09-22T10:48:06.140Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 188.861292] nix-grpc-daemon[1391]: time=2026-09-22T10:48:06.140Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 188.919923] niks3-server[1133]: 2026/09/22 10:48:06 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 188.935448] niks3-server[1133]: 2026/09/22 10:48:06 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 188.916962] nix-grpc-daemon[1391]: time=2026-09-22T10:48:06.199Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.952970] niks3-server[1133]: 2026/09/22 10:48:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 188.965665] niks3-server[1133]: 2026/09/22 10:48:06 INFO Signed narinfos id=42 count=1 lb # [ 188.971460] envoy[961]: conn peer=[2001:db8:1::4]:39732 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=25705 worker1 # [ 188.946757] nix-grpc-daemon[1391]: time=2026-09-22T10:48:06.230Z level=INFO msg="Upload complete. (329ms)" lb # [ 188.981128] niks3-server[1133]: 2026/09/22 10:48:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker1 # [ 188.953034] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 lb # [ 188.986694] niks3-server[1133]: 2026/09/22 10:48:06 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 188.990993] niks3-server[1133]: 2026/09/22 10:48:06 INFO Completed upload id=42 lb # [ 188.992877] niks3-server[1133]: 2026/09/22 10:48:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 188.964614] nix-daemon[921]: reaped child process 1488, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.30 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.28 seconds) worker2 # [ 189.433095] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 189.443878] systemd[1]: Starting scheduler-token.service... worker2 # [ 189.557268] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 189.563364] systemd[1]: Finished scheduler-token.service. worker2 # [ 189.565417] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 114ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 189.570016] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 189.761143] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 189.766015] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 189.768089] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 189.775297] nix-grpc-daemon[1533]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 189.791653] nix-grpc-daemon[1533]: E0000 00:00:1790074087.042768 1533 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 189.802331] nix-grpc-daemon[1533]: E0000 00:00:1790074087.046957 1533 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 189.806822] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 189.808806] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 189.812181] nix-grpc-daemon[1533]: E0000 00:00:1790074087.078592 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 189.840149] nix-grpc-daemon[1533]: E0000 00:00:1790074087.097353 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 189.842675] nix-grpc-daemon[1533]: E0000 00:00:1790074087.097630 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 189.851011] nix-grpc-daemon[1533]: E0000 00:00:1790074087.117897 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 189.865697] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 189.841978] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 189.842493] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:37692 worker1 # [ 189.852437] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.90 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs client # [ 190.253465] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: bcb79a721d324c4786407517f09e26bd client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds) worker2 # [ 190.587525] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58182 worker2 # [ 190.597038] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) client # [ 190.571351] nix[1304]: don't know how to build these paths: client # [ 190.572795] nix[1304]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 190.593304] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 190.600446] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 190.648218] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 190.665574] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52008 client # [ 190.666616] nix[1304]: copying 1 paths to 'lb:50051'... client # [ 190.667862] nix[1304]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 190.678110] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 190.694373] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 190.913114] nix-daemon[921]: accepted connection from pid 1522, user nix-grpc-daemon (trusted) worker1 # [ 190.938486] nix-daemon[921]: reaped child process 1525, status = succeeded worker1 # [ 191.116697] nix-daemon[921]: accepted connection from pid 1527, user nix-grpc-daemon (trusted) worker1 # [ 191.139291] nix-daemon[921]: reaped child process 1530, status = succeeded lb # [ 191.194464] niks3-server[1133]: 2026/09/22 10:48:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.171762] nix-grpc-daemon[1391]: time=2026-09-22T10:48:08.454Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.173541] nix-grpc-daemon[1391]: time=2026-09-22T10:48:08.454Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 191.228545] niks3-server[1133]: 2026/09/22 10:48:08 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 191.241478] niks3-server[1133]: 2026/09/22 10:48:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 191.213749] nix-grpc-daemon[1391]: time=2026-09-22T10:48:08.496Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.252226] niks3-server[1133]: 2026/09/22 10:48:08 INFO Signed narinfos id=43 count=1 lb # [ 191.254886] niks3-server[1133]: 2026/09/22 10:48:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 191.265243] niks3-server[1133]: 2026/09/22 10:48:08 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 191.276214] niks3-server[1133]: 2026/09/22 10:48:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo lb # [ 191.281465] niks3-server[1133]: 2026/09/22 10:48:08 INFO Completed upload id=43 worker1 # [ 191.247373] nix-grpc-daemon[1391]: time=2026-09-22T10:48:08.530Z level=INFO msg="Upload complete. (535ms)" worker1 # [ 191.257486] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 191.275692] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 worker1 # [ 191.282860] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) worker1 # [ 191.288921] nix-daemon[921]: reaped child process 1520, status = succeeded client # [ 191.290357] nix[1304]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 191.304284] nix-daemon[921]: accepted connection from pid 1312, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 193.232725] systemd-journald[281]: Received client request to rotate journal, rotating. worker2 # [ 193.250690] systemd-journald[281]: Vacuuming done, freed 0B of archived journals from /var/log/journal/877443b90b5641cfba3c4c17d61524ec. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 193.273440] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 193.413628] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:10Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.06 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 198.765487] nix-daemon[921]: reaped child process 1534, status = succeeded lb # [ 198.952661] envoy[961]: conn peer=[2001:db8:1::1]:51200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9679 tx=4162 ms=7973 worker1 # [ 198.957822] nix-daemon[921]: accepted connection from pid 1575, user nix-grpc-daemon (trusted) worker1 # [ 198.979584] nix-daemon[921]: reaped child process 1578, status = succeeded lb # [ 199.027041] niks3-server[1133]: 2026/09/22 10:48:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 199.008234] nix-grpc-daemon[1391]: time=2026-09-22T10:48:16.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 199.009876] nix-grpc-daemon[1391]: time=2026-09-22T10:48:16.288Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 199.073725] niks3-server[1133]: 2026/09/22 10:48:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 199.090283] niks3-server[1133]: 2026/09/22 10:48:16 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 199.100500] niks3-server[1133]: 2026/09/22 10:48:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign worker1 # [ 199.071099] nix-grpc-daemon[1391]: time=2026-09-22T10:48:16.353Z level=INFO msg="Uploading 1 narinfos" lb # [ 199.108731] niks3-server[1133]: 2026/09/22 10:48:16 INFO Signed narinfos id=44 count=1 lb # [ 199.111273] niks3-server[1133]: 2026/09/22 10:48:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 199.123364] niks3-server[1133]: 2026/09/22 10:48:16 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 199.131150] niks3-server[1133]: 2026/09/22 10:48:16 INFO Completed upload id=44 worker1 # [ 199.095347] nix-grpc-daemon[1391]: time=2026-09-22T10:48:16.378Z level=INFO msg="Upload complete. (371ms)" lb # [ 199.134114] niks3-server[1133]: 2026/09/22 10:48:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 199.112635] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52008 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 199.125725] nix-daemon[921]: reaped child process 1532, status = succeeded client # [ 199.145631] systemd[1]: rs.service: Deactivated successfully. client # [ 199.150140] systemd[1]: rs.service: Consumed 205ms CPU time over 8.895s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 199.414393] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:16Z level=info event=signal_exit lb # [ 200.121020] niks3-server[1133]: 2026/09/22 10:48:17 INFO lead: released remote=192.168.1.3:55974 worker2 # [ 201.116872] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 201.086425] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=scheduler_restarting worker1 # [ 201.091463] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:37692 worker1 # [ 201.097510] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 201.102271] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 201.107378] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:37692 worker1 # [ 201.112392] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 201.138028] nix-daemon[921]: reaped child process 1366, status = succeeded worker2 # [ 201.317156] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 201.322990] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 201.308005] nix-grpc-daemon[1312]: ts=2026-09-22T10:48:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 201.524378] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 201.529951] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 201.515062] nix-grpc-daemon[1312]: E0000 00:00:1790074098.798130 1357 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 201.540426] nix-grpc-daemon[1312]: E0000 00:00:1790074098.823538 1357 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 201.731422] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 201.737615] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 201.939115] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 201.946854] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.31 seconds) client: waiting for success: ! systemctl is-active rs worker1 # [ 202.007309] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 202.015135] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 202.019570] systemd[1]: nix-grpc-daemon.service: Consumed 2.066s CPU time over 41.850s wall clock time, 37.1M memory peak, 46.5K incoming IP traffic, 85.3K outgoing IP traffic. worker1 # [ 202.035404] systemd[1]: Starting Nix worker-protocol over gRPC... client: (finished: waiting for success: ! systemctl is-active rs, in 0.08 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' worker2 # [ 202.148500] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 202.153345] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 202.237563] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 202.240276] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 202.244584] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 202.251383] nix-grpc-daemon[1585]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 202.259835] nix-grpc-daemon[1585]: E0000 00:00:1790074099.534078 1585 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 202.264228] nix-grpc-daemon[1585]: E0000 00:00:1790074099.537987 1585 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 202.267142] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 202.300714] niks3-server[1133]: 2026/09/22 10:48:19 INFO lead: acquired remote=192.168.1.3:41398 worker1 # [ 202.269845] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 202.270756] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=scheduler_take_over worker1 # [ 202.276089] nix-grpc-daemon[1585]: E0000 00:00:1790074099.556054 1627 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 202.304099] nix-grpc-daemon[1585]: E0000 00:00:1790074099.574518 1627 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 202.310385] nix-grpc-daemon[1585]: E0000 00:00:1790074099.574771 1627 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 202.313594] nix-grpc-daemon[1585]: E0000 00:00:1790074099.592334 1627 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 202.355071] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 202.324318] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 202.326478] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 202.361233] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 202.527704] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 202.562500] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 202.533456] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 202.568245] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 202.734551] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 202.769823] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 202.740942] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 202.775702] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 202.976535] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 202.942125] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 202.980295] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 202.947883] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 203.183496] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 203.149384] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 203.161444] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:34886 worker1 # [ 203.168317] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 203.179031] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 203.183101] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:34886 worker1 # [ 203.187765] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.20 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid client # [ 203.455032] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: d61e0b4503414186a25b99e7d4f4d308 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds) worker1 # [ 203.754326] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34888 worker1 # [ 203.771114] nix-daemon[921]: accepted connection from pid 1585, user nix-grpc-daemon (trusted) client # [ 203.779458] nix[1340]: don't know how to build these paths: client # [ 203.781484] nix[1340]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker1 # [ 203.805759] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:21Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 203.813421] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:21Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 203.855980] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34888 worker1 # [ 203.877566] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5) worker1 # [ 203.968794] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5) worker1 # [ 204.091516] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 220 ms (attempt 3/5) worker1 # [ 204.320262] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 553 ms (attempt 4/5) worker1 # [ 204.898751] nix-grpc-daemon[1585]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 204.905385] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 205.844247] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34888 client # [ 206.025017] nix[1340]: copying 1 paths to 'lb:50051'... client # [ 206.025240] nix[1340]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 206.039773] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34888 worker1 # [ 206.065661] nix-daemon[921]: accepted connection from pid 1585, user nix-grpc-daemon (trusted) worker1 # [ 206.136545] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:23Z level=info event=niks3_push_started pid=1662 worker1 # [ 206.339155] nix-grpc-daemon[1662]: time=2026-09-22T10:48:23.621Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 206.533471] nix-daemon[921]: accepted connection from pid 1675, user nix-grpc-daemon (trusted) worker1 # [ 206.555300] nix-daemon[921]: reaped child process 1678, status = succeeded worker1 # [ 206.742870] nix-daemon[921]: accepted connection from pid 1680, user nix-grpc-daemon (trusted) worker1 # [ 206.762571] nix-daemon[921]: reaped child process 1683, status = succeeded lb # [ 206.814455] niks3-server[1133]: 2026/09/22 10:48:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 206.790613] nix-grpc-daemon[1662]: time=2026-09-22T10:48:24.072Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 206.792504] nix-grpc-daemon[1662]: time=2026-09-22T10:48:24.072Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 206.857229] niks3-server[1133]: 2026/09/22 10:48:24 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 206.871901] niks3-server[1133]: 2026/09/22 10:48:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 206.843852] nix-grpc-daemon[1662]: time=2026-09-22T10:48:24.126Z level=INFO msg="Uploading 1 narinfos" lb # [ 206.883938] niks3-server[1133]: 2026/09/22 10:48:24 INFO Signed narinfos id=45 count=1 lb # [ 206.891531] niks3-server[1133]: 2026/09/22 10:48:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 206.905616] niks3-server[1133]: 2026/09/22 10:48:24 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete worker1 # [ 206.877607] nix-grpc-daemon[1662]: time=2026-09-22T10:48:24.160Z level=INFO msg="Upload complete. (534ms)" lb # [ 206.912920] niks3-server[1133]: 2026/09/22 10:48:24 INFO Completed upload id=45 lb # [ 206.917928] niks3-server[1133]: 2026/09/22 10:48:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker1 # [ 206.888432] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34888 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 206.904563] nix-grpc-daemon[1585]: ts=2026-09-22T10:48:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34888 worker1 # [ 206.912321] nix-daemon[921]: accepted connection from pid 1585, user nix-grpc-daemon (trusted) worker1 # [ 206.920315] nix-daemon[921]: reaped child process 1660, status = succeeded client # [ 206.918828] nix[1340]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 206.933649] nix-daemon[921]: accepted connection from pid 1585, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker1 # [ 208.824766] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1585 (nix-grpc-daemon) on client request. worker1 # [ 208.826564] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 208.873627] niks3-server[1133]: 2026/09/22 10:48:26 INFO lead: released remote=192.168.1.3:41398 client # [ 208.836787] nix[1340]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 208.881980] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:26Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.09 seconds) client: must succeed: sleep 5.0 client # [ 208.844356] nix[1340]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker1 # [ 208.872189] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 208.873933] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker1 # [ 208.875212] systemd[1]: nix-grpc-daemon.service: Consumed 741ms CPU time over 6.833s wall clock time, 34.3M memory peak, 14.4K incoming IP traffic, 18.7K outgoing IP traffic. worker1 # [ 208.890728] nix-daemon[921]: reaped child process 1685, status = succeeded worker1 # [ 208.891970] nix-daemon[921]: reaped child process 1644, status = succeeded worker1 # [ 208.900606] nix-daemon[1687]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 208.909493] nix-daemon[921]: reaped child process 1687, status = failed with exit code 1 lb # [ 208.956235] envoy[961]: conn peer=[2001:db8:1::1]:51210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1428 ms=8571 lb # [ 208.957941] envoy[961]: conn peer=[2001:db8:1::3]:50696 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2265 tx=831 ms=40836 lb # [ 208.959527] envoy[961]: conn peer=[2001:db8:1::3]:42314 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=411 ms=6530 worker2 # [ 209.082588] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 209.208851] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1. worker1 # [ 209.218018] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 209.444468] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 209.447325] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 209.451635] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 209.460145] nix-grpc-daemon[1724]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 209.461746] nix-grpc-daemon[1724]: E0000 00:00:1790074106.741857 1724 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 209.477070] nix-grpc-daemon[1724]: E0000 00:00:1790074106.748657 1724 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 209.479711] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 209.483521] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 209.485406] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=scheduler_take_over lb # [ 209.517392] niks3-server[1133]: 2026/09/22 10:48:26 INFO lead: acquired remote=192.168.1.3:41450 worker1 # [ 209.490506] nix-grpc-daemon[1724]: E0000 00:00:1790074106.771849 1766 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 209.548178] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58182 worker1 # [ 209.520246] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 209.521716] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:34918 worker1 # [ 209.523394] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 209.535466] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 209.537958] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 209.568175] nix-grpc-daemon[1533]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5) worker1 # [ 209.540066] nix-grpc-daemon[1724]: E0000 00:00:1790074106.797680 1766 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 209.542335] nix-grpc-daemon[1724]: E0000 00:00:1790074106.797887 1766 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 209.547670] nix-grpc-daemon[1724]: E0000 00:00:1790074106.817812 1766 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 209.553123] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 209.554632] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:34918 worker1 # [ 209.556195] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 209.610378] nix-grpc-daemon[1533]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 2/5) worker2 # [ 209.655564] nix-grpc-daemon[1533]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 246 ms (attempt 3/5) worker2 # [ 209.906016] nix-grpc-daemon[1533]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 408 ms (attempt 4/5) worker2 # [ 210.322777] nix-grpc-daemon[1533]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 210.328878] nix-grpc-daemon[1533]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 210.345222] nix-grpc-daemon[1533]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'... worker2 # [ 210.371722] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) worker2 # [ 210.381782] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) client # [ 210.337400] nix[1340]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.08 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 214.419174] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:31Z level=info event=niks3_push_started pid=1699 worker2 # [ 214.468598] nix-daemon[947]: reaped child process 1674, status = succeeded worker2 # [ 214.661960] nix-grpc-daemon[1699]: time=2026-09-22T10:48:31.929Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 214.898724] nix-daemon[947]: accepted connection from pid 1712, user nix-grpc-daemon (trusted) worker2 # [ 214.921526] nix-daemon[947]: reaped child process 1715, status = succeeded lb # [ 214.939811] niks3-server[1133]: 2026/09/22 10:48:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 214.955014] nix-grpc-daemon[1699]: time=2026-09-22T10:48:32.220Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 214.958230] nix-grpc-daemon[1699]: time=2026-09-22T10:48:32.220Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 215.001615] niks3-server[1133]: 2026/09/22 10:48:32 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst worker2 # [ 215.012230] nix-grpc-daemon[1699]: time=2026-09-22T10:48:32.279Z level=INFO msg="Uploading 1 narinfos" lb # [ 215.010756] niks3-server[1133]: 2026/09/22 10:48:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 215.018134] niks3-server[1133]: 2026/09/22 10:48:32 INFO Signed narinfos id=46 count=1 lb # [ 215.022042] niks3-server[1133]: 2026/09/22 10:48:32 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 215.028983] niks3-server[1133]: 2026/09/22 10:48:32 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 215.036358] niks3-server[1133]: 2026/09/22 10:48:32 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 215.043275] niks3-server[1133]: 2026/09/22 10:48:32 INFO Completed upload id=46 worker2 # [ 215.044103] nix-grpc-daemon[1699]: time=2026-09-22T10:48:32.312Z level=INFO msg="Upload complete. (378ms)" worker2 # [ 215.052058] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58182 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker2 # [ 215.063605] nix-daemon[947]: reaped child process 1672, status = succeeded client # [ 215.049743] systemd[1]: mid.service: Deactivated successfully. client # [ 215.052746] systemd[1]: mid.service: Consumed 335ms CPU time over 11.597s wall clock time, 41.2M memory peak, 9.2K incoming IP traffic, 10.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.20 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 12.16 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr client # [ 215.574605] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 4f0889ac341a49fb8240153b97afe428 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 215.877744] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58182 client # [ 215.842150] nix[1376]: don't know how to build these paths: client # [ 215.843225] nix[1376]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 215.865483] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:33Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 215.865824] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:33Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 215.919806] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 215.938055] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 215.969412] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5) worker1 # [ 216.011299] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 102 ms (attempt 2/5) worker1 # [ 216.121615] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 307 ms (attempt 3/5) worker1 # [ 216.436764] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 4/5) worker1 # [ 216.740377] nix-grpc-daemon[1724]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 216.746151] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 216.768195] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 client # [ 216.776536] nix[1376]: copying 1 paths to 'lb:50051'... worker1 # [ 216.787838] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 client # [ 216.780828] nix[1376]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 216.808752] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 216.849531] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:34Z level=info event=niks3_push_started pid=1804 worker1 # [ 217.071442] nix-grpc-daemon[1804]: time=2026-09-22T10:48:34.353Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 217.253586] nix-daemon[921]: accepted connection from pid 1817, user nix-grpc-daemon (trusted) worker1 # [ 217.276123] nix-daemon[921]: reaped child process 1820, status = succeeded worker1 # [ 217.455721] nix-daemon[921]: accepted connection from pid 1822, user nix-grpc-daemon (trusted) worker1 # [ 217.475890] nix-daemon[921]: reaped child process 1825, status = succeeded lb # [ 217.526841] niks3-server[1133]: 2026/09/22 10:48:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 217.505857] nix-grpc-daemon[1804]: time=2026-09-22T10:48:34.789Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 217.509551] nix-grpc-daemon[1804]: time=2026-09-22T10:48:34.791Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 217.566806] niks3-server[1133]: 2026/09/22 10:48:34 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker1 # [ 217.538374] nix-grpc-daemon[1804]: time=2026-09-22T10:48:34.821Z level=INFO msg="Uploading 1 narinfos" lb # [ 217.574522] niks3-server[1133]: 2026/09/22 10:48:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 217.579429] niks3-server[1133]: 2026/09/22 10:48:34 INFO Signed narinfos id=47 count=1 lb # [ 217.582647] niks3-server[1133]: 2026/09/22 10:48:34 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 217.589496] niks3-server[1133]: 2026/09/22 10:48:34 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 217.593872] niks3-server[1133]: 2026/09/22 10:48:34 INFO Completed upload id=47 worker1 # [ 217.559696] nix-grpc-daemon[1804]: time=2026-09-22T10:48:34.842Z level=INFO msg="Upload complete. (484ms)" lb # [ 217.595731] niks3-server[1133]: 2026/09/22 10:48:34 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker1 # [ 217.568236] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 217.571438] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 217.585811] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 217.590264] nix-daemon[921]: reaped child process 1801, status = succeeded worker1 # [ 217.601907] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) client # [ 217.591700] nix[1376]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... lb # [ 218.952405] envoy[961]: conn peer=[2001:db8:1::1]:50802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1877 ms=11287 client: must succeed: systemctl kill -s INT intr client # [ 219.363092] systemd[1]: intr.service: Sent signal SIGINT to main process 1376 (nix) on client request. worker1 # [ 219.383573] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:36Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 219.372487] nix[1376]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 219.383497] nix[1376]: error: interrupted by the user client # [ 219.412663] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 219.414463] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 219.417369] systemd[1]: intr.service: Consumed 185ms CPU time over 3.837s wall clock time, 41.2M memory peak, 6.3K incoming IP traffic, 8.2K outgoing IP traffic. worker1 # [ 219.441799] nix-daemon[1829]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 219.446300] nix-daemon[921]: reaped child process 1827, status = succeeded worker1 # [ 219.454796] nix-daemon[921]: reaped child process 1829, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.01 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw client # [ 219.610048] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 20645317a9d94cfaaa908dc92456cddc client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds) worker1 # [ 219.894868] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37536 client # [ 219.893689] nix[1397]: don't know how to build these paths: client # [ 219.894776] nix[1397]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 219.915043] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:37Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 219.921464] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:37Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 219.997795] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58182 worker2 # [ 220.010323] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58182 client # [ 219.972448] nix[1397]: copying 1 paths to 'lb:50051'... worker2 # [ 220.019691] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58182 client # [ 219.975396] nix[1397]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 220.027160] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) worker2 # [ 220.207644] nix-daemon[947]: accepted connection from pid 1748, user nix-grpc-daemon (trusted) worker2 # [ 220.226818] nix-daemon[947]: reaped child process 1751, status = succeeded worker2 # [ 220.405948] nix-daemon[947]: accepted connection from pid 1753, user nix-grpc-daemon (trusted) worker2 # [ 220.427585] nix-daemon[947]: reaped child process 1756, status = succeeded lb # [ 220.440827] niks3-server[1133]: 2026/09/22 10:48:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 220.452438] nix-grpc-daemon[1699]: time=2026-09-22T10:48:37.719Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 220.454078] nix-grpc-daemon[1699]: time=2026-09-22T10:48:37.719Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 220.476425] niks3-server[1133]: 2026/09/22 10:48:37 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker2 # [ 220.487403] nix-grpc-daemon[1699]: time=2026-09-22T10:48:37.755Z level=INFO msg="Uploading 1 narinfos" lb # [ 220.490529] niks3-server[1133]: 2026/09/22 10:48:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 220.503494] niks3-server[1133]: 2026/09/22 10:48:37 INFO Signed narinfos id=48 count=1 lb # [ 220.509683] niks3-server[1133]: 2026/09/22 10:48:37 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 220.518460] niks3-server[1133]: 2026/09/22 10:48:37 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete worker2 # [ 220.523698] nix-grpc-daemon[1699]: time=2026-09-22T10:48:37.791Z level=INFO msg="Upload complete. (488ms)" lb # [ 220.524393] niks3-server[1133]: 2026/09/22 10:48:37 INFO Completed upload id=48 lb # [ 220.528710] niks3-server[1133]: 2026/09/22 10:48:37 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 220.532294] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58182 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 220.547162] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58182 worker2 # [ 220.554044] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) worker2 # [ 220.558947] nix-daemon[947]: reaped child process 1746, status = succeeded client # [ 220.524547] nix[1397]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2 # [ 220.571212] nix-daemon[947]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # [ 223.184624] nixos[1792]: switching to system configuration /nix/store/53zsn476qs62sf3s60b4si9wblfzz0vj-nixos-system-worker2-test worker2 # activating the configuration... worker2 # [ 223.188132] systemd[1]: Stopped target Remote File Systems. worker2 # [ 223.189000] systemd[1]: Stopped target Local File Systems. worker2 # setting up /etc... worker2 # [ 223.970155] systemd[1]: Reload requested from client PID 1792 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 223.982147] systemd[1]: Reloading... worker2 # [ 224.837349] systemd[1]: Reloading finished in 863 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 224.920239] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # [ 225.082252] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # [ 225.083468] systemd[1]: Reached target Reactivate sysinit units. worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service worker2 # [ 225.091374] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 225.092977] dbus-broker-launch[589]: Caught SIGHUP, trigger reload. worker2 # [ 225.097137] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 225.129766] dbus-broker-launch[589]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 225.139115] dbus-broker-launch[589]: NSS returned no entry for 'systemd-timesync' worker2 # [ 225.141974] dbus-broker-launch[589]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 225.155000] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 225.181586] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 225.187740] systemd[1]: Reached target Local File Systems. worker2 # [ 225.192306] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 225.196163] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 225.197014] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 225.198188] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 225.217155] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 225.218773] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 225.220001] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 225.228607] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 225.232163] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 225.236075] systemd[1]: Reached target Remote File Systems. worker2 # [ 225.714730] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 225.722318] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 225.725164] systemd[1]: Starting linger-users.service... worker2 # [ 225.786884] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 225.792095] systemd[1]: Finished linger-users.service. worker2 # [ 225.910730] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:43Z level=info event=draining worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 226.044824] nixos[1792]: finished switching to system configuration /nix/store/53zsn476qs62sf3s60b4si9wblfzz0vj-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.72 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2 worker1 # [ 226.373422] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37536 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 226.385005] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 226.413987] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 226.421125] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 226.468594] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 226.489007] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 226.506376] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 226.523940] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 226.752444] nix-daemon[921]: accepted connection from pid 1880, user nix-grpc-daemon (trusted) worker1 # [ 226.775480] nix-daemon[921]: reaped child process 1883, status = succeeded worker1 # [ 226.943622] nix-daemon[921]: accepted connection from pid 1885, user nix-grpc-daemon (trusted) worker1 # [ 226.963854] nix-daemon[921]: reaped child process 1888, status = succeeded lb # [ 227.017716] niks3-server[1133]: 2026/09/22 10:48:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 226.995237] nix-grpc-daemon[1804]: time=2026-09-22T10:48:44.277Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 226.996944] nix-grpc-daemon[1804]: time=2026-09-22T10:48:44.277Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" worker1 # [ 227.029379] nix-grpc-daemon[1804]: time=2026-09-22T10:48:44.312Z level=INFO msg="Uploading 1 narinfos" lb # [ 227.055944] niks3-server[1133]: 2026/09/22 10:48:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 227.073966] niks3-server[1133]: 2026/09/22 10:48:44 INFO Signed narinfos id=49 count=1 lb # [ 227.081787] niks3-server[1133]: 2026/09/22 10:48:44 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 227.098058] niks3-server[1133]: 2026/09/22 10:48:44 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 227.107547] niks3-server[1133]: 2026/09/22 10:48:44 INFO Completed upload id=49 worker1 # [ 227.069773] nix-grpc-daemon[1804]: time=2026-09-22T10:48:44.352Z level=INFO msg="Upload complete. (525ms)" worker1 # [ 227.080551] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=528 lb # [ 227.116831] niks3-server[1133]: 2026/09/22 10:48:44 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 227.095054] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 lb # [ 227.130373] niks3-server[1133]: 2026/09/22 10:48:44 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker1 # [ 227.101915] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 227.107122] nix-daemon[921]: reaped child process 1878, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker1 # [ 227.119504] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 228.147778] nix-daemon[921]: reaped child process 1892, status = succeeded worker1 # [ 228.309127] nix-daemon[921]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker1 # [ 228.329593] nix-daemon[921]: reaped child process 1911, status = succeeded lb # [ 228.375786] niks3-server[1133]: 2026/09/22 10:48:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 228.356495] nix-grpc-daemon[1804]: time=2026-09-22T10:48:45.638Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 228.358070] nix-grpc-daemon[1804]: time=2026-09-22T10:48:45.638Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 228.443157] niks3-server[1133]: 2026/09/22 10:48:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 228.418141] nix-grpc-daemon[1804]: time=2026-09-22T10:48:45.700Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.454079] niks3-server[1133]: 2026/09/22 10:48:45 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 228.463413] niks3-server[1133]: 2026/09/22 10:48:45 INFO Signed narinfos id=50 count=1 worker1 # [ 228.440302] nix-grpc-daemon[1804]: time=2026-09-22T10:48:45.723Z level=INFO msg="Upload complete. (307ms)" lb # [ 228.468058] niks3-server[1133]: 2026/09/22 10:48:45 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 228.478523] niks3-server[1133]: 2026/09/22 10:48:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 228.483096] niks3-server[1133]: 2026/09/22 10:48:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 228.487007] niks3-server[1133]: 2026/09/22 10:48:45 INFO Completed upload id=50 lb # [ 228.489014] niks3-server[1133]: 2026/09/22 10:48:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 228.455486] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 228.470043] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 228.477596] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 228.482016] nix-daemon[921]: reaped child process 1890, status = succeeded worker1 # [ 228.513736] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 228.524434] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 228.536251] nix-grpc-daemon[1804]: time=2026-09-22T10:48:45.819Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 228.544666] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 228.561544] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 228.775930] nix-daemon[921]: accepted connection from pid 1917, user nix-grpc-daemon (trusted) worker1 # [ 228.797801] nix-daemon[921]: reaped child process 1920, status = succeeded lb # [ 228.953905] envoy[961]: conn peer=[2001:db8:1::1]:39564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=851 ms=3562 worker1 # [ 228.987321] nix-daemon[921]: accepted connection from pid 1922, user nix-grpc-daemon (trusted) worker1 # [ 229.007137] nix-daemon[921]: reaped child process 1925, status = succeeded lb # [ 229.056578] niks3-server[1133]: 2026/09/22 10:48:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 229.036448] nix-grpc-daemon[1804]: time=2026-09-22T10:48:46.319Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 229.038048] nix-grpc-daemon[1804]: time=2026-09-22T10:48:46.319Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 229.091827] niks3-server[1133]: 2026/09/22 10:48:46 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 229.102637] niks3-server[1133]: 2026/09/22 10:48:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker1 # [ 229.074272] nix-grpc-daemon[1804]: time=2026-09-22T10:48:46.357Z level=INFO msg="Uploading 1 narinfos" lb # [ 229.110910] niks3-server[1133]: 2026/09/22 10:48:46 INFO Signed narinfos id=51 count=1 lb # [ 229.115633] niks3-server[1133]: 2026/09/22 10:48:46 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 229.123891] niks3-server[1133]: 2026/09/22 10:48:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker1 # [ 229.095549] nix-grpc-daemon[1804]: time=2026-09-22T10:48:46.378Z level=INFO msg="Upload complete. (518ms)" lb # [ 229.130334] niks3-server[1133]: 2026/09/22 10:48:46 INFO Completed upload id=51 lb # [ 229.132917] niks3-server[1133]: 2026/09/22 10:48:46 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker1 # [ 229.102334] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 229.112403] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 229.126668] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 229.127975] nix-daemon[921]: reaped child process 1915, status = succeeded worker1 # [ 229.137195] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 230.365637] nix-daemon[921]: reaped child process 1929, status = succeeded worker1 # [ 230.539425] nix-daemon[921]: accepted connection from pid 1945, user nix-grpc-daemon (trusted) worker1 # [ 230.563150] nix-daemon[921]: reaped child process 1948, status = succeeded lb # [ 230.610626] niks3-server[1133]: 2026/09/22 10:48:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 230.587093] nix-grpc-daemon[1804]: time=2026-09-22T10:48:47.869Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 230.588742] nix-grpc-daemon[1804]: time=2026-09-22T10:48:47.869Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 230.649999] niks3-server[1133]: 2026/09/22 10:48:47 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 230.661152] niks3-server[1133]: 2026/09/22 10:48:47 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 230.672113] niks3-server[1133]: 2026/09/22 10:48:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign worker1 # [ 230.643687] nix-grpc-daemon[1804]: time=2026-09-22T10:48:47.926Z level=INFO msg="Uploading 1 narinfos" lb # [ 230.681028] niks3-server[1133]: 2026/09/22 10:48:47 INFO Signed narinfos id=52 count=1 lb # [ 230.687135] niks3-server[1133]: 2026/09/22 10:48:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 230.696832] niks3-server[1133]: 2026/09/22 10:48:47 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 230.705227] niks3-server[1133]: 2026/09/22 10:48:47 INFO Completed upload id=52 worker1 # [ 230.670719] nix-grpc-daemon[1804]: time=2026-09-22T10:48:47.953Z level=INFO msg="Upload complete. (322ms)" lb # [ 230.708627] niks3-server[1133]: 2026/09/22 10:48:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 230.685679] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 230.704559] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:47Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 230.708956] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:47Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 230.713148] nix-daemon[921]: reaped child process 1927, status = succeeded worker1 # [ 230.741962] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 230.753277] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 230.764827] nix-grpc-daemon[1804]: time=2026-09-22T10:48:48.048Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 230.773582] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 230.788532] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 231.000378] nix-daemon[921]: accepted connection from pid 1954, user nix-grpc-daemon (trusted) worker1 # [ 231.018917] nix-daemon[921]: reaped child process 1957, status = succeeded worker1 # [ 231.205670] nix-daemon[921]: accepted connection from pid 1959, user nix-grpc-daemon (trusted) worker1 # [ 231.225193] nix-daemon[921]: reaped child process 1962, status = succeeded lb # [ 231.271200] niks3-server[1133]: 2026/09/22 10:48:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 231.247076] nix-grpc-daemon[1804]: time=2026-09-22T10:48:48.529Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 231.248830] nix-grpc-daemon[1804]: time=2026-09-22T10:48:48.530Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 231.301054] niks3-server[1133]: 2026/09/22 10:48:48 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst worker1 # [ 231.280219] nix-grpc-daemon[1804]: time=2026-09-22T10:48:48.563Z level=INFO msg="Uploading 1 narinfos" lb # [ 231.317853] niks3-server[1133]: 2026/09/22 10:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 231.328058] niks3-server[1133]: 2026/09/22 10:48:48 INFO Signed narinfos id=53 count=1 lb # [ 231.330925] niks3-server[1133]: 2026/09/22 10:48:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 231.339991] niks3-server[1133]: 2026/09/22 10:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete worker1 # [ 231.312530] nix-grpc-daemon[1804]: time=2026-09-22T10:48:48.595Z level=INFO msg="Upload complete. (505ms)" lb # [ 231.346928] niks3-server[1133]: 2026/09/22 10:48:48 INFO Completed upload id=53 lb # [ 231.350553] niks3-server[1133]: 2026/09/22 10:48:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker1 # [ 231.320127] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 231.324313] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 231.335550] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 231.337208] nix-daemon[921]: reaped child process 1952, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 231.350123] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 232.574034] nix-daemon[921]: reaped child process 1966, status = succeeded worker1 # [ 232.756920] nix-daemon[921]: accepted connection from pid 1982, user nix-grpc-daemon (trusted) worker1 # [ 232.782031] nix-daemon[921]: reaped child process 1985, status = succeeded lb # [ 232.833736] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 232.810756] nix-grpc-daemon[1804]: time=2026-09-22T10:48:50.091Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 232.812377] nix-grpc-daemon[1804]: time=2026-09-22T10:48:50.091Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 232.866812] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 232.886642] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 232.859007] nix-grpc-daemon[1804]: time=2026-09-22T10:48:50.141Z level=INFO msg="Uploading 1 narinfos" lb # [ 232.900149] niks3-server[1133]: 2026/09/22 10:48:50 INFO Signed narinfos id=54 count=1 lb # [ 232.906901] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 232.914037] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 232.923859] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete worker1 # [ 232.894909] nix-grpc-daemon[1804]: time=2026-09-22T10:48:50.178Z level=INFO msg="Upload complete. (334ms)" lb # [ 232.929796] niks3-server[1133]: 2026/09/22 10:48:50 INFO Completed upload id=54 lb # [ 232.934846] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker1 # [ 232.905035] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker1 # [ 232.917377] nix-daemon[921]: reaped child process 1964, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.89 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.25 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.07 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 233.423120] nix-daemon[947]: reaped child process 1760, status = succeeded worker2 # [ 233.598764] nix-daemon[947]: accepted connection from pid 1932, user nix-grpc-daemon (trusted) worker2 # [ 233.621366] nix-daemon[947]: reaped child process 1935, status = succeeded lb # [ 233.635029] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 233.649016] nix-grpc-daemon[1699]: time=2026-09-22T10:48:50.916Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 233.650581] nix-grpc-daemon[1699]: time=2026-09-22T10:48:50.916Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 233.681806] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 233.692459] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst worker2 # [ 233.700176] nix-grpc-daemon[1699]: time=2026-09-22T10:48:50.967Z level=INFO msg="Uploading 1 narinfos" lb # [ 233.702484] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 233.708191] niks3-server[1133]: 2026/09/22 10:48:50 INFO Signed narinfos id=55 count=1 lb # [ 233.712403] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 233.723388] niks3-server[1133]: 2026/09/22 10:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 233.729211] niks3-server[1133]: 2026/09/22 10:48:50 INFO Completed upload id=55 worker2 # [ 233.729802] nix-grpc-daemon[1699]: time=2026-09-22T10:48:50.997Z level=INFO msg="Upload complete. (327ms)" lb # [ 233.732195] niks3-server[1133]: 2026/09/22 10:48:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 233.745192] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58182 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker2 # [ 233.759527] nix-daemon[947]: reaped child process 1758, status = succeeded client # [ 233.742325] systemd[1]: sw.service: Deactivated successfully. client # [ 233.747225] systemd[1]: sw.service: Consumed 200ms CPU time over 14.136s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 233.912675] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:51Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 234.609619] systemd[1]: Starting scheduler-token.service... worker2 # [ 234.701069] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 234.705292] systemd[1]: Finished scheduler-token.service. worker2 # [ 234.706760] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 235.062530] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:52Z level=info event=scheduler_restarting worker1 # [ 235.037682] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:34918 worker1 # [ 235.043100] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:52Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 235.094968] nix-grpc-daemon[1533]: ts=2026-09-22T10:48:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 235.121344] nix-daemon[947]: reaped child process 1592, status = succeeded worker2 # [ 235.993318] nix-grpc-daemon[1533]: E0000 00:00:1790074133.261050 1578 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true worker2 # [ 235.993813] nix-grpc-daemon[1533]: E0000 00:00:1790074133.261340 1578 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 236.316276] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 236.319407] systemd[1]: nix-grpc-daemon.service: Consumed 1.139s CPU time over 46.743s wall clock time, 33.8M memory peak, 23.6K incoming IP traffic, 40.6K outgoing IP traffic. worker2 # [ 236.343745] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 236.538715] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 236.541471] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 236.545926] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 236.555389] nix-grpc-daemon[1952]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 236.566230] nix-grpc-daemon[1952]: E0000 00:00:1790074133.821014 1952 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 236.570340] nix-grpc-daemon[1952]: E0000 00:00:1790074133.823351 1952 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 236.588047] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 236.589942] systemd[1]: Started Nix worker-protocol over gRPC. worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.08 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 236.590786] nix-grpc-daemon[1952]: E0000 00:00:1790074133.843059 1991 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 236.614446] nix-grpc-daemon[1952]: E0000 00:00:1790074133.866341 1991 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 236.617574] nix-grpc-daemon[1952]: E0000 00:00:1790074133.866576 1991 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 236.623242] nix-grpc-daemon[1952]: E0000 00:00:1790074133.881265 1991 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 236.628190] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 236.595568] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:34918 worker1 # [ 236.599578] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.31 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 238.182172] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 238.196544] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 238.217870] nix-grpc-daemon[1952]: ts=2026-09-22T10:48:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 238.238132] nix-grpc-daemon[1952]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5) worker2 # [ 238.250839] nix-grpc-daemon[1952]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 2/5) worker2 # [ 238.403407] nix-grpc-daemon[1952]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 3/5) worker2 # [ 238.511929] nix-grpc-daemon[1952]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 681 ms (attempt 4/5) lb # [ 238.954753] envoy[961]: conn peer=[2001:db8:1::1]:60518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4078 ms=6555 lb # [ 238.960036] envoy[961]: conn peer=[2001:db8:1::1]:39570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1430 ms=13822 lb # [ 238.964876] envoy[961]: conn peer=[2001:db8:1::4]:42086 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9904 tx=834 ms=45217 worker2 # [ 239.194330] nix-grpc-daemon[1952]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 239.198158] nix-grpc-daemon[1952]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 239.218037] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 239.218569] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 239.218701] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 239.218811] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 239.264504] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 239.283557] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 239.300876] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 239.317142] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 239.532521] nix-daemon[921]: accepted connection from pid 1994, user nix-grpc-daemon (trusted) worker1 # [ 239.554129] nix-daemon[921]: reaped child process 1997, status = succeeded worker1 # [ 239.742489] nix-daemon[921]: accepted connection from pid 1999, user nix-grpc-daemon (trusted) worker1 # [ 239.763830] nix-daemon[921]: reaped child process 2002, status = succeeded lb # [ 239.814088] niks3-server[1133]: 2026/09/22 10:48:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 239.789355] nix-grpc-daemon[1804]: time=2026-09-22T10:48:57.071Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 239.790880] nix-grpc-daemon[1804]: time=2026-09-22T10:48:57.071Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 239.845787] niks3-server[1133]: 2026/09/22 10:48:57 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 239.827133] nix-grpc-daemon[1804]: time=2026-09-22T10:48:57.110Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.863555] niks3-server[1133]: 2026/09/22 10:48:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 239.874718] niks3-server[1133]: 2026/09/22 10:48:57 INFO Signed narinfos id=56 count=1 lb # [ 239.885335] niks3-server[1133]: 2026/09/22 10:48:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 239.891246] niks3-server[1133]: 2026/09/22 10:48:57 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 239.897041] niks3-server[1133]: 2026/09/22 10:48:57 INFO Completed upload id=56 worker1 # [ 239.861946] nix-grpc-daemon[1804]: time=2026-09-22T10:48:57.145Z level=INFO msg="Upload complete. (528ms)" lb # [ 239.900314] niks3-server[1133]: 2026/09/22 10:48:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 239.872337] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 239.887828] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 239.894757] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 239.899974] nix-daemon[921]: reaped child process 1992, status = succeeded worker1 # [ 239.915888] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 240.570213] nix-daemon[921]: reaped child process 2006, status = succeeded worker1 # [ 240.746907] nix-daemon[921]: accepted connection from pid 2022, user nix-grpc-daemon (trusted) worker1 # [ 240.769266] nix-daemon[921]: reaped child process 2025, status = succeeded lb # [ 240.816890] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 240.792448] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.075Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 240.794015] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.075Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 240.855900] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 240.872990] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker1 # [ 240.844939] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.127Z level=INFO msg="Uploading 1 narinfos" lb # [ 240.885148] niks3-server[1133]: 2026/09/22 10:48:58 INFO Signed narinfos id=57 count=1 lb # [ 240.887507] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 240.895379] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker1 # [ 240.868929] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.152Z level=INFO msg="Upload complete. (338ms)" lb # [ 240.903241] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 240.906902] niks3-server[1133]: 2026/09/22 10:48:58 INFO Completed upload id=57 lb # [ 240.909893] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 240.885247] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=1 worker1 # [ 240.898296] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 240.909238] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 240.914882] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 240.918635] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 240.922984] nix-daemon[921]: reaped child process 2004, status = succeeded worker1 # [ 240.927270] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 240.937091] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 241.125310] nix-daemon[921]: accepted connection from pid 2031, user nix-grpc-daemon (trusted) worker1 # [ 241.145737] nix-daemon[921]: reaped child process 2034, status = succeeded worker1 # [ 241.329387] nix-daemon[921]: accepted connection from pid 2036, user nix-grpc-daemon (trusted) worker1 # [ 241.349811] nix-daemon[921]: reaped child process 2039, status = succeeded lb # [ 241.395308] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 241.370685] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.653Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 241.372341] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.653Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 241.424569] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst lb # [ 241.437472] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker1 # [ 241.409381] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.692Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.447772] niks3-server[1133]: 2026/09/22 10:48:58 INFO Signed narinfos id=58 count=1 lb # [ 241.454448] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 241.463066] niks3-server[1133]: 2026/09/22 10:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 241.470097] niks3-server[1133]: 2026/09/22 10:48:58 INFO Completed upload id=58 worker1 # [ 241.435333] nix-grpc-daemon[1804]: time=2026-09-22T10:48:58.718Z level=INFO msg="Upload complete. (488ms)" lb # [ 241.472843] niks3-server[1133]: 2026/09/22 10:48:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 241.444305] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 241.458642] nix-grpc-daemon[1724]: ts=2026-09-22T10:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 241.465997] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 241.471346] nix-daemon[921]: reaped child process 2029, status = succeeded worker1 # [ 241.483458] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 242.730369] nix-daemon[921]: reaped child process 2043, status = succeeded worker1 # [ 242.928174] nix-daemon[921]: accepted connection from pid 2059, user nix-grpc-daemon (trusted) worker1 # [ 242.948786] nix-daemon[921]: reaped child process 2062, status = succeeded lb # [ 242.997303] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 242.976527] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 242.978309] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.255Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 243.046248] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 243.022689] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.305Z level=INFO msg="Uploading 1 narinfos" lb # [ 243.060786] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 243.069205] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 243.073976] niks3-server[1133]: 2026/09/22 10:49:00 INFO Signed narinfos id=59 count=1 lb # [ 243.080579] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 243.084127] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 243.087653] niks3-server[1133]: 2026/09/22 10:49:00 INFO Completed upload id=59 worker1 # [ 243.053705] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.336Z level=INFO msg="Upload complete. (355ms)" lb # [ 243.089948] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 243.073478] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=1 worker1 # [ 243.096442] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 243.101249] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 243.107664] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 243.113155] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 243.116991] nix-daemon[921]: reaped child process 2041, status = succeeded worker1 # [ 243.121600] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 243.131724] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 243.332583] nix-daemon[921]: accepted connection from pid 2068, user nix-grpc-daemon (trusted) worker1 # [ 243.355839] nix-daemon[921]: reaped child process 2071, status = succeeded worker1 # [ 243.540137] nix-daemon[921]: accepted connection from pid 2073, user nix-grpc-daemon (trusted) worker1 # [ 243.559305] nix-daemon[921]: reaped child process 2076, status = succeeded lb # [ 243.606449] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 243.582117] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 243.583860] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.865Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 243.635359] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 243.652126] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker1 # [ 243.623358] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.906Z level=INFO msg="Uploading 1 narinfos" lb # [ 243.664409] niks3-server[1133]: 2026/09/22 10:49:00 INFO Signed narinfos id=60 count=1 lb # [ 243.672519] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 243.680216] niks3-server[1133]: 2026/09/22 10:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 243.687224] niks3-server[1133]: 2026/09/22 10:49:00 INFO Completed upload id=60 worker1 # [ 243.652597] nix-grpc-daemon[1804]: time=2026-09-22T10:49:00.935Z level=INFO msg="Upload complete. (509ms)" lb # [ 243.690584] niks3-server[1133]: 2026/09/22 10:49:00 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 243.660395] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 243.676381] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 243.683582] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 243.689591] nix-daemon[921]: reaped child process 2066, status = succeeded worker1 # [ 243.706796] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 244.753563] nix-daemon[921]: reaped child process 2080, status = succeeded worker1 # [ 244.935418] nix-daemon[921]: accepted connection from pid 2096, user nix-grpc-daemon (trusted) worker1 # [ 244.957748] nix-daemon[921]: reaped child process 2099, status = succeeded lb # [ 245.005192] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 244.982428] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.265Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 244.985477] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.267Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 245.045479] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 245.063302] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker1 # [ 245.034153] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.317Z level=INFO msg="Uploading 1 narinfos" lb # [ 245.071985] niks3-server[1133]: 2026/09/22 10:49:02 INFO Signed narinfos id=61 count=1 lb # [ 245.077953] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 245.086063] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 245.094586] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 245.098990] niks3-server[1133]: 2026/09/22 10:49:02 INFO Completed upload id=61 worker1 # [ 245.064193] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.347Z level=INFO msg="Upload complete. (344ms)" lb # [ 245.102161] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 245.084939] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=1 worker1 # [ 245.100945] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 245.108226] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 245.111338] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 245.117659] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 245.122013] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.404Z level=INFO msg="All 1 paths already cached" worker1 # [ 245.124551] nix-daemon[921]: reaped child process 2078, status = succeeded worker1 # [ 245.127861] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 245.138643] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 245.338587] nix-daemon[921]: accepted connection from pid 2105, user nix-grpc-daemon (trusted) worker1 # [ 245.362010] nix-daemon[921]: reaped child process 2108, status = succeeded worker1 # [ 245.543436] nix-daemon[921]: accepted connection from pid 2110, user nix-grpc-daemon (trusted) worker1 # [ 245.562735] nix-daemon[921]: reaped child process 2113, status = succeeded lb # [ 245.610079] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 245.581645] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 245.583190] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.864Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 245.628908] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst lb # [ 245.634944] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 245.603793] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.886Z level=INFO msg="Uploading 1 narinfos" lb # [ 245.639543] niks3-server[1133]: 2026/09/22 10:49:02 INFO Signed narinfos id=62 count=1 lb # [ 245.643038] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 245.649314] niks3-server[1133]: 2026/09/22 10:49:02 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 245.654560] niks3-server[1133]: 2026/09/22 10:49:02 INFO Completed upload id=62 worker1 # [ 245.619944] nix-grpc-daemon[1804]: time=2026-09-22T10:49:02.903Z level=INFO msg="Upload complete. (469ms)" lb # [ 245.656458] niks3-server[1133]: 2026/09/22 10:49:02 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 245.624100] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 245.635685] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 245.641739] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 245.645670] nix-daemon[921]: reaped child process 2103, status = succeeded worker1 # [ 245.654330] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 246.906477] nix-daemon[921]: reaped child process 2117, status = succeeded worker1 # [ 247.086048] nix-daemon[921]: accepted connection from pid 2133, user nix-grpc-daemon (trusted) worker1 # [ 247.108217] nix-daemon[921]: reaped child process 2136, status = succeeded lb # [ 247.157857] niks3-server[1133]: 2026/09/22 10:49:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 247.131992] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.414Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 247.137541] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.417Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 247.196182] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 247.215808] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 247.224516] niks3-server[1133]: 2026/09/22 10:49:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker1 # [ 247.195262] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.478Z level=INFO msg="Uploading 1 narinfos" lb # [ 247.232634] niks3-server[1133]: 2026/09/22 10:49:04 INFO Signed narinfos id=63 count=1 lb # [ 247.237367] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 247.244887] niks3-server[1133]: 2026/09/22 10:49:04 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete worker1 # [ 247.216381] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.499Z level=INFO msg="Upload complete. (350ms)" lb # [ 247.251066] niks3-server[1133]: 2026/09/22 10:49:04 INFO Completed upload id=63 lb # [ 247.254583] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 247.225633] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=1 worker1 # [ 247.232215] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 247.240609] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 247.242426] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 247.245419] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 247.247436] nix-daemon[921]: reaped child process 2115, status = succeeded worker1 # [ 247.252796] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.536Z level=INFO msg="All 1 paths already cached" worker1 # [ 247.258678] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 247.269020] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 247.445092] nix-daemon[921]: accepted connection from pid 2142, user nix-grpc-daemon (trusted) worker1 # [ 247.465746] nix-daemon[921]: reaped child process 2145, status = succeeded worker1 # [ 247.639526] nix-daemon[921]: accepted connection from pid 2147, user nix-grpc-daemon (trusted) worker1 # [ 247.662604] nix-daemon[921]: reaped child process 2150, status = succeeded lb # [ 247.705214] niks3-server[1133]: 2026/09/22 10:49:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 247.681958] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.964Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 247.683516] nix-grpc-daemon[1804]: time=2026-09-22T10:49:04.964Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 247.738665] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 247.717223] nix-grpc-daemon[1804]: time=2026-09-22T10:49:05.000Z level=INFO msg="Uploading 1 narinfos" lb # [ 247.755229] niks3-server[1133]: 2026/09/22 10:49:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 247.765236] niks3-server[1133]: 2026/09/22 10:49:04 INFO Signed narinfos id=64 count=1 lb # [ 247.771144] niks3-server[1133]: 2026/09/22 10:49:04 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 247.745963] nix-grpc-daemon[1804]: time=2026-09-22T10:49:05.029Z level=INFO msg="Upload complete. (468ms)" lb # [ 247.780592] niks3-server[1133]: 2026/09/22 10:49:05 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete worker1 # [ 247.750515] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 247.785665] niks3-server[1133]: 2026/09/22 10:49:05 INFO Completed upload id=64 worker1 # [ 247.755207] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 lb # [ 247.789030] niks3-server[1133]: 2026/09/22 10:49:05 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 247.764186] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 247.767210] nix-daemon[921]: reaped child process 2140, status = succeeded worker1 # [ 247.774889] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 248.606739] nix-daemon[921]: reaped child process 2154, status = succeeded worker1 # [ 248.789594] nix-daemon[921]: accepted connection from pid 2170, user nix-grpc-daemon (trusted) worker1 # [ 248.812710] nix-daemon[921]: reaped child process 2173, status = succeeded lb # [ 248.861390] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 248.840164] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.121Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 248.841814] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.122Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 248.914459] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 248.891523] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.174Z level=INFO msg="Uploading 1 narinfos" lb # [ 248.930405] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 248.939000] niks3-server[1133]: 2026/09/22 10:49:06 INFO Signed narinfos id=65 count=1 lb # [ 248.944094] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 248.947484] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 248.955856] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 248.962476] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 248.931935] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.215Z level=INFO msg="Upload complete. (345ms)" lb # [ 248.966495] niks3-server[1133]: 2026/09/22 10:49:06 INFO Completed upload id=65 worker1 # [ 248.940521] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=1 worker1 # [ 248.943150] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 248.952779] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 248.955498] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 248.961626] nix-daemon[921]: reaped child process 2152, status = succeeded worker1 # [ 248.964505] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 248.974935] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.258Z level=INFO msg="All 1 paths already cached" worker1 # [ 248.980194] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 248.989661] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 249.204243] nix-daemon[921]: accepted connection from pid 2179, user nix-grpc-daemon (trusted) worker1 # [ 249.232813] nix-daemon[921]: reaped child process 2182, status = succeeded worker1 # [ 249.432065] nix-daemon[921]: accepted connection from pid 2184, user nix-grpc-daemon (trusted) worker1 # [ 249.454695] nix-daemon[921]: reaped child process 2187, status = succeeded lb # [ 249.501720] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 249.478624] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.761Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 249.480540] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.761Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 249.543347] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 249.523959] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.806Z level=INFO msg="Uploading 1 narinfos" lb # [ 249.563193] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 249.574336] niks3-server[1133]: 2026/09/22 10:49:06 INFO Signed narinfos id=66 count=1 lb # [ 249.580098] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 249.592190] niks3-server[1133]: 2026/09/22 10:49:06 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker1 # [ 249.568394] nix-grpc-daemon[1804]: time=2026-09-22T10:49:06.851Z level=INFO msg="Upload complete. (568ms)" lb # [ 249.602920] niks3-server[1133]: 2026/09/22 10:49:06 INFO Completed upload id=66 lb # [ 249.607123] niks3-server[1133]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 249.576128] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 249.579416] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 249.588522] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 249.593729] nix-daemon[921]: reaped child process 2177, status = succeeded worker1 # [ 249.604151] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 250.836760] nix-daemon[921]: reaped child process 2191, status = succeeded worker1 # [ 251.013693] nix-daemon[921]: accepted connection from pid 2207, user nix-grpc-daemon (trusted) worker1 # [ 251.034770] nix-daemon[921]: reaped child process 2210, status = succeeded lb # [ 251.081553] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 251.058735] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.340Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 251.061098] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.340Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 251.117032] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 251.132123] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 251.143463] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 251.114457] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.397Z level=INFO msg="Uploading 1 narinfos" lb # [ 251.150513] niks3-server[1133]: 2026/09/22 10:49:08 INFO Signed narinfos id=67 count=1 lb # [ 251.156275] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 251.167252] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 251.173742] niks3-server[1133]: 2026/09/22 10:49:08 INFO Completed upload id=67 worker1 # [ 251.138895] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.422Z level=INFO msg="Upload complete. (325ms)" lb # [ 251.176966] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 251.159429] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=1 worker1 # [ 251.176627] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 251.188376] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 251.195691] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 251.200928] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 251.205702] nix-daemon[921]: reaped child process 2189, status = succeeded worker1 # [ 251.209236] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.483Z level=INFO msg="All 2 paths already cached" worker1 # [ 251.218911] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 251.226811] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 251.414210] nix-daemon[921]: accepted connection from pid 2216, user nix-grpc-daemon (trusted) worker1 # [ 251.435598] nix-daemon[921]: reaped child process 2219, status = succeeded worker1 # [ 251.621414] nix-daemon[921]: accepted connection from pid 2221, user nix-grpc-daemon (trusted) worker1 # [ 251.641789] nix-daemon[921]: reaped child process 2224, status = succeeded lb # [ 251.690157] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 251.666186] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 251.668215] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.951Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 251.722382] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst lb # [ 251.734656] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 251.706521] nix-grpc-daemon[1804]: time=2026-09-22T10:49:08.989Z level=INFO msg="Uploading 1 narinfos" lb # [ 251.747229] niks3-server[1133]: 2026/09/22 10:49:08 INFO Signed narinfos id=68 count=1 lb # [ 251.754467] niks3-server[1133]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 251.765523] niks3-server[1133]: 2026/09/22 10:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 251.775491] niks3-server[1133]: 2026/09/22 10:49:09 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo lb # [ 251.780738] niks3-server[1133]: 2026/09/22 10:49:09 INFO Completed upload id=68 worker1 # [ 251.746713] nix-grpc-daemon[1804]: time=2026-09-22T10:49:09.029Z level=INFO msg="Upload complete. (513ms)" worker1 # [ 251.757202] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 251.772337] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 251.786626] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 251.795377] nix-daemon[921]: reaped child process 2214, status = succeeded worker1 # [ 251.802672] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 253.058419] nix-daemon[921]: reaped child process 2228, status = succeeded worker1 # [ 253.243372] nix-daemon[921]: accepted connection from pid 2244, user nix-grpc-daemon (trusted) worker1 # [ 253.264931] nix-daemon[921]: reaped child process 2247, status = succeeded lb # [ 253.313933] niks3-server[1133]: 2026/09/22 10:49:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 253.288451] nix-grpc-daemon[1804]: time=2026-09-22T10:49:10.570Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 253.290088] nix-grpc-daemon[1804]: time=2026-09-22T10:49:10.570Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 253.351027] niks3-server[1133]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 253.365769] niks3-server[1133]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 253.376339] niks3-server[1133]: 2026/09/22 10:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker1 # [ 253.347778] nix-grpc-daemon[1804]: time=2026-09-22T10:49:10.630Z level=INFO msg="Uploading 1 narinfos" lb # [ 253.383591] niks3-server[1133]: 2026/09/22 10:49:10 INFO Signed narinfos id=69 count=1 lb # [ 253.389515] niks3-server[1133]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 253.399664] niks3-server[1133]: 2026/09/22 10:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete worker1 # [ 253.370885] nix-grpc-daemon[1804]: time=2026-09-22T10:49:10.654Z level=INFO msg="Upload complete. (353ms)" lb # [ 253.407088] niks3-server[1133]: 2026/09/22 10:49:10 INFO Completed upload id=69 lb # [ 253.409803] niks3-server[1133]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 253.385231] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=1 worker1 # [ 253.403006] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:10Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 253.408283] nix-daemon[921]: reaped child process 2226, status = succeeded worker1 # [ 253.426357] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 253.442206] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 253.457532] nix-grpc-daemon[1804]: time=2026-09-22T10:49:10.740Z level=INFO msg="All 2 paths already cached" worker1 # [ 253.467214] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 253.481077] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 253.681069] nix-daemon[921]: accepted connection from pid 2251, user nix-grpc-daemon (trusted) worker1 # [ 253.701859] nix-daemon[921]: reaped child process 2254, status = succeeded worker1 # [ 253.884343] nix-daemon[921]: accepted connection from pid 2256, user nix-grpc-daemon (trusted) worker1 # [ 253.904142] nix-daemon[921]: reaped child process 2259, status = succeeded lb # [ 253.952048] niks3-server[1133]: 2026/09/22 10:49:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 253.927212] nix-grpc-daemon[1804]: time=2026-09-22T10:49:11.210Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 253.928896] nix-grpc-daemon[1804]: time=2026-09-22T10:49:11.210Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 253.981012] niks3-server[1133]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst lb # [ 253.997779] niks3-server[1133]: 2026/09/22 10:49:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker1 # [ 253.969424] nix-grpc-daemon[1804]: time=2026-09-22T10:49:11.252Z level=INFO msg="Uploading 1 narinfos" lb # [ 254.009544] niks3-server[1133]: 2026/09/22 10:49:11 INFO Signed narinfos id=70 count=1 lb # [ 254.015830] niks3-server[1133]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 253.989038] nix-grpc-daemon[1804]: time=2026-09-22T10:49:11.272Z level=INFO msg="Upload complete. (492ms)" lb # [ 254.024892] niks3-server[1133]: 2026/09/22 10:49:11 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 254.030081] niks3-server[1133]: 2026/09/22 10:49:11 INFO Completed upload id=70 lb # [ 254.033358] niks3-server[1133]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 253.999811] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 254.010418] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 254.016964] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 254.021805] nix-daemon[921]: reaped child process 2249, status = succeeded worker1 # [ 254.034743] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 255.472853] nix-daemon[921]: reaped child process 2263, status = succeeded worker1 # [ 255.652684] nix-daemon[921]: accepted connection from pid 2279, user nix-grpc-daemon (trusted) worker1 # [ 255.673243] nix-daemon[921]: reaped child process 2282, status = succeeded lb # [ 255.722093] niks3-server[1133]: 2026/09/22 10:49:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 255.697693] nix-grpc-daemon[1804]: time=2026-09-22T10:49:12.978Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 255.699267] nix-grpc-daemon[1804]: time=2026-09-22T10:49:12.978Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 255.771163] niks3-server[1133]: 2026/09/22 10:49:12 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 255.779183] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker1 # [ 255.752585] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.035Z level=INFO msg="Uploading 1 narinfos" lb # [ 255.789795] niks3-server[1133]: 2026/09/22 10:49:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 255.800239] niks3-server[1133]: 2026/09/22 10:49:13 INFO Signed narinfos id=71 count=1 lb # [ 255.803284] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 255.815446] niks3-server[1133]: 2026/09/22 10:49:13 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 255.821451] niks3-server[1133]: 2026/09/22 10:49:13 INFO Completed upload id=71 worker1 # [ 255.786554] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.069Z level=INFO msg="Upload complete. (341ms)" lb # [ 255.824814] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 255.801322] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=1 worker1 # [ 255.818562] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 255.823376] nix-daemon[921]: reaped child process 2261, status = succeeded worker1 # [ 255.843751] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 255.860122] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 255.875906] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.159Z level=INFO msg="All 2 paths already cached" worker1 # [ 255.882355] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 255.897968] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 256.113729] nix-daemon[921]: accepted connection from pid 2286, user nix-grpc-daemon (trusted) worker1 # [ 256.134686] nix-daemon[921]: reaped child process 2289, status = succeeded worker1 # [ 256.320119] nix-daemon[921]: accepted connection from pid 2291, user nix-grpc-daemon (trusted) worker1 # [ 256.339461] nix-daemon[921]: reaped child process 2294, status = succeeded lb # [ 256.391204] niks3-server[1133]: 2026/09/22 10:49:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 256.368154] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.651Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 256.369746] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.651Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 256.426854] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 256.443066] niks3-server[1133]: 2026/09/22 10:49:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 256.415936] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.698Z level=INFO msg="Uploading 1 narinfos" lb # [ 256.456224] niks3-server[1133]: 2026/09/22 10:49:13 INFO Signed narinfos id=72 count=1 lb # [ 256.458848] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 256.468353] niks3-server[1133]: 2026/09/22 10:49:13 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 256.439936] nix-grpc-daemon[1804]: time=2026-09-22T10:49:13.723Z level=INFO msg="Upload complete. (524ms)" lb # [ 256.475866] niks3-server[1133]: 2026/09/22 10:49:13 INFO Completed upload id=72 worker1 # [ 256.444125] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 256.479588] niks3-server[1133]: 2026/09/22 10:49:13 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 256.449838] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 256.458521] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 256.461866] nix-daemon[921]: reaped child process 2284, status = succeeded worker1 # [ 256.471591] nix-daemon[921]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 257.090211] nix-daemon[921]: reaped child process 2298, status = succeeded worker1 # [ 257.249867] nix-daemon[921]: accepted connection from pid 2314, user nix-grpc-daemon (trusted) worker1 # [ 257.271623] nix-daemon[921]: reaped child process 2317, status = succeeded lb # [ 257.321011] niks3-server[1133]: 2026/09/22 10:49:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 257.298418] nix-grpc-daemon[1804]: time=2026-09-22T10:49:14.580Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 257.299947] nix-grpc-daemon[1804]: time=2026-09-22T10:49:14.580Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 257.360992] niks3-server[1133]: 2026/09/22 10:49:14 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 257.374709] niks3-server[1133]: 2026/09/22 10:49:14 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 257.387467] niks3-server[1133]: 2026/09/22 10:49:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 257.358008] nix-grpc-daemon[1804]: time=2026-09-22T10:49:14.640Z level=INFO msg="Uploading 1 narinfos" lb # [ 257.394152] niks3-server[1133]: 2026/09/22 10:49:14 INFO Signed narinfos id=73 count=1 lb # [ 257.399183] niks3-server[1133]: 2026/09/22 10:49:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 257.408133] niks3-server[1133]: 2026/09/22 10:49:14 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 257.413354] niks3-server[1133]: 2026/09/22 10:49:14 INFO Completed upload id=73 worker1 # [ 257.378892] nix-grpc-daemon[1804]: time=2026-09-22T10:49:14.662Z level=INFO msg="Upload complete. (300ms)" lb # [ 257.415724] niks3-server[1133]: 2026/09/22 10:49:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 257.393182] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37536 duration_s=0 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=1 worker1 # [ 257.409427] nix-daemon[921]: reaped child process 2296, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.61 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 257.615116] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 1a612397f5b542828755542f9eac78e8 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker2 # [ 257.926453] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39760 client # [ 257.889387] nix[1477]: don't know how to build these paths: client # [ 257.890528] nix[1477]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker1 # [ 257.905053] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 client # [ 257.919221] nix[1477]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker worker1 # [ 257.929028] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:15Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb # [ 258.965459] envoy[961]: conn peer=[2001:db8:1::1]:54250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27105 tx=11406 ms=19274 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code client # [ 287.922068] nix[1477]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu} client # [ 287.935173] nix[1477]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 287.998811] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 288.006398] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 288.012440] systemd[1]: gpu.service: Consumed 199ms CPU time over 30.389s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.33 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.12 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.92 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill lb # [ 288.969279] envoy[961]: conn peer=[2001:db8:1::1]:57014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=30075 worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.27 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 290.533400] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:47Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.14 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2 worker1 # [ 291.468898] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37536 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 291.479358] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37536 worker1 # [ 291.513734] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:48Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 291.519505] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:48Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 291.601613] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 291.620436] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39760 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 291.636101] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 291.657906] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 291.691292] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:48Z level=info event=niks3_push_started pid=2012 worker2 # [ 291.913013] nix-grpc-daemon[2012]: time=2026-09-22T10:49:49.180Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 292.088534] nix-daemon[947]: accepted connection from pid 2025, user nix-grpc-daemon (trusted) worker2 # [ 292.107271] nix-daemon[947]: reaped child process 2028, status = succeeded worker2 # [ 292.283561] nix-daemon[947]: accepted connection from pid 2030, user nix-grpc-daemon (trusted) worker2 # [ 292.301449] nix-daemon[947]: reaped child process 2033, status = succeeded lb # [ 292.317093] niks3-server[1133]: 2026/09/22 10:49:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 292.333102] nix-grpc-daemon[2012]: time=2026-09-22T10:49:49.599Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 292.334658] nix-grpc-daemon[2012]: time=2026-09-22T10:49:49.599Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 292.365504] niks3-server[1133]: 2026/09/22 10:49:49 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 292.383029] nix-grpc-daemon[2012]: time=2026-09-22T10:49:49.650Z level=INFO msg="Uploading 1 narinfos" lb # [ 292.381850] niks3-server[1133]: 2026/09/22 10:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 292.390600] niks3-server[1133]: 2026/09/22 10:49:49 INFO Signed narinfos id=74 count=1 lb # [ 292.397993] niks3-server[1133]: 2026/09/22 10:49:49 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 292.407909] niks3-server[1133]: 2026/09/22 10:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 292.421300] niks3-server[1133]: 2026/09/22 10:49:49 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo lb # [ 292.426484] niks3-server[1133]: 2026/09/22 10:49:49 INFO Completed upload id=74 worker2 # [ 292.427066] nix-grpc-daemon[2012]: time=2026-09-22T10:49:49.695Z level=INFO msg="Upload complete. (509ms)" worker2 # [ 292.438475] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 292.459476] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 292.471377] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 292.481251] nix-daemon[947]: reaped child process 2010, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 292.490753] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 293.738375] nix-daemon[947]: reaped child process 2037, status = succeeded worker2 # [ 293.916579] nix-daemon[947]: accepted connection from pid 2053, user nix-grpc-daemon (trusted) worker2 # [ 293.935777] nix-daemon[947]: reaped child process 2056, status = succeeded lb # [ 293.948904] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 293.960949] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 293.966790] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.231Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 293.986990] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 294.010634] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.277Z level=INFO msg="Uploading 1 narinfos" lb # [ 294.005822] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 294.015943] niks3-server[1133]: 2026/09/22 10:49:51 INFO Signed narinfos id=75 count=1 lb # [ 294.019834] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 294.028706] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 294.031158] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 294.037280] niks3-server[1133]: 2026/09/22 10:49:51 INFO Completed upload id=75 worker2 # [ 294.036469] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.304Z level=INFO msg="Upload complete. (333ms)" lb # [ 294.039345] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 294.048873] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=1 worker1 # [ 294.018400] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:51Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 294.024517] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:51Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 294.064911] nix-daemon[947]: reaped child process 2035, status = succeeded worker2 # [ 294.096388] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 294.112446] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 294.127287] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.395Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 294.137949] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 294.152132] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 294.363807] nix-daemon[947]: accepted connection from pid 2060, user nix-grpc-daemon (trusted) worker2 # [ 294.382283] nix-daemon[947]: reaped child process 2063, status = succeeded worker2 # [ 294.558681] nix-daemon[947]: accepted connection from pid 2065, user nix-grpc-daemon (trusted) worker2 # [ 294.575975] nix-daemon[947]: reaped child process 2068, status = succeeded lb # [ 294.587358] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 294.597267] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.865Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 294.598804] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.865Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 294.613058] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst lb # [ 294.627547] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker2 # [ 294.633662] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.901Z level=INFO msg="Uploading 1 narinfos" lb # [ 294.640390] niks3-server[1133]: 2026/09/22 10:49:51 INFO Signed narinfos id=76 count=1 lb # [ 294.647103] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 294.656822] niks3-server[1133]: 2026/09/22 10:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 294.661350] niks3-server[1133]: 2026/09/22 10:49:51 INFO Completed upload id=76 worker2 # [ 294.661721] nix-grpc-daemon[2012]: time=2026-09-22T10:49:51.929Z level=INFO msg="Upload complete. (493ms)" lb # [ 294.663944] niks3-server[1133]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 294.668273] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 294.682062] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 294.689160] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 294.693823] nix-daemon[947]: reaped child process 2058, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker2 # [ 294.707843] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 295.938121] nix-daemon[947]: reaped child process 2072, status = succeeded worker2 # [ 296.111814] nix-daemon[947]: accepted connection from pid 2088, user nix-grpc-daemon (trusted) worker2 # [ 296.130600] nix-daemon[947]: reaped child process 2091, status = succeeded lb # [ 296.145022] niks3-server[1133]: 2026/09/22 10:49:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 296.155872] nix-grpc-daemon[2012]: time=2026-09-22T10:49:53.423Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 296.157532] nix-grpc-daemon[2012]: time=2026-09-22T10:49:53.423Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 296.177729] niks3-server[1133]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 296.192223] niks3-server[1133]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 296.200193] niks3-server[1133]: 2026/09/22 10:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker2 # [ 296.206105] nix-grpc-daemon[2012]: time=2026-09-22T10:49:53.473Z level=INFO msg="Uploading 1 narinfos" lb # [ 296.209157] niks3-server[1133]: 2026/09/22 10:49:53 INFO Signed narinfos id=77 count=1 lb # [ 296.215247] niks3-server[1133]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 296.225195] niks3-server[1133]: 2026/09/22 10:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 296.230641] niks3-server[1133]: 2026/09/22 10:49:53 INFO Completed upload id=77 lb # [ 296.233896] niks3-server[1133]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 296.230692] nix-grpc-daemon[2012]: time=2026-09-22T10:49:53.498Z level=INFO msg="Upload complete. (312ms)" worker1 # [ 296.214816] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:53Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 296.245327] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=1 worker1 # [ 296.220963] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:53Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 296.261814] nix-daemon[947]: reaped child process 2070, status = succeeded worker2 # [ 296.297687] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 296.313390] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39760 client # copying 1 paths to 'lb:50051'... worker2 # [ 296.328051] nix-grpc-daemon[2012]: time=2026-09-22T10:49:53.596Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 296.334520] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 296.349867] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 296.548770] nix-daemon[947]: accepted connection from pid 2095, user nix-grpc-daemon (trusted) worker2 # [ 296.568934] nix-daemon[947]: reaped child process 2098, status = succeeded worker2 # [ 296.739222] nix-daemon[947]: accepted connection from pid 2100, user nix-grpc-daemon (trusted) worker2 # [ 296.758614] nix-daemon[947]: reaped child process 2103, status = succeeded lb # [ 296.770719] niks3-server[1133]: 2026/09/22 10:49:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 296.781131] nix-grpc-daemon[2012]: time=2026-09-22T10:49:54.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 296.782634] nix-grpc-daemon[2012]: time=2026-09-22T10:49:54.049Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 296.800689] niks3-server[1133]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 296.812947] nix-grpc-daemon[2012]: time=2026-09-22T10:49:54.080Z level=INFO msg="Uploading 1 narinfos" lb # [ 296.816971] niks3-server[1133]: 2026/09/22 10:49:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 296.826442] niks3-server[1133]: 2026/09/22 10:49:54 INFO Signed narinfos id=78 count=1 lb # [ 296.830066] niks3-server[1133]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 296.839659] nix-grpc-daemon[2012]: time=2026-09-22T10:49:54.107Z level=INFO msg="Upload complete. (472ms)" lb # [ 296.839482] niks3-server[1133]: 2026/09/22 10:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 296.844761] niks3-server[1133]: 2026/09/22 10:49:54 INFO Completed upload id=78 lb # [ 296.848130] niks3-server[1133]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 296.848999] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 296.859767] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 worker2 # [ 296.865656] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 296.870320] nix-daemon[947]: reaped child process 2093, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 296.882941] nix-daemon[947]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 297.711708] nix-daemon[947]: reaped child process 2107, status = succeeded worker2 # [ 297.880171] nix-daemon[947]: accepted connection from pid 2123, user nix-grpc-daemon (trusted) worker2 # [ 297.899169] nix-daemon[947]: reaped child process 2126, status = succeeded lb # [ 297.913513] niks3-server[1133]: 2026/09/22 10:49:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 297.924402] nix-grpc-daemon[2012]: time=2026-09-22T10:49:55.191Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 297.925894] nix-grpc-daemon[2012]: time=2026-09-22T10:49:55.191Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 297.951234] niks3-server[1133]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 297.965145] nix-grpc-daemon[2012]: time=2026-09-22T10:49:55.233Z level=INFO msg="Uploading 1 narinfos" lb # [ 297.967773] niks3-server[1133]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 297.976609] niks3-server[1133]: 2026/09/22 10:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 297.981662] niks3-server[1133]: 2026/09/22 10:49:55 INFO Signed narinfos id=79 count=1 lb # [ 297.987646] niks3-server[1133]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 297.991103] niks3-server[1133]: 2026/09/22 10:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 297.995131] niks3-server[1133]: 2026/09/22 10:49:55 INFO Completed upload id=79 worker2 # [ 297.995128] nix-grpc-daemon[2012]: time=2026-09-22T10:49:55.263Z level=INFO msg="Upload complete. (303ms)" lb # [ 297.997706] niks3-server[1133]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 298.004615] nix-grpc-daemon[1952]: ts=2026-09-22T10:49:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39760 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=1 worker2 # [ 298.021958] nix-daemon[947]: reaped child process 2105, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.84 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 298.534613] nix-grpc-daemon[1724]: ts=2026-09-22T10:49:55Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 0.05 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.79 seconds) (finished: run the VM test script, in 300.76 seconds) test script finished in 301.07s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) lb # [ 298.970719] envoy[961]: conn peer=[2001:db8:1::1]:44050 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=4107 ms=6518 kill QemuMachine (pid 51) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) kill QemuMachine (pid 50) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) (finished: cleanup, in 1.24 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh