tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm lb # Disk image does not exist, creating the virtualisation disk image... lb # Formatting '/build/vm-state-lb/tmp.nOfWybEVGv', fmt=raw size=1073741824 client # Disk image does not exist, creating the virtualisation disk image... worker1 # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) client # Formatting '/build/vm-state-client/tmp.FSib7LfjrR', fmt=raw size=1073741824 lb # mke2fs 1.47.4 (6-Mar-2025) lb: QEMU running (pid 50) worker2: QEMU running (pid 52) client # mke2fs 1.47.4 (6-Mar-2025) lb # Discarding device blocks: 0/262144 done worker2 # Disk image does not exist, creating the virtualisation disk image... client # Discarding device blocks: 0/262144 done lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Formatting '/build/vm-state-worker2/tmp.mZYubaQsHZ', fmt=raw size=1073741824 client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Filesystem UUID: ecc6f207-dc8e-431d-b120-fb330412fd42 worker2 # mke2fs 1.47.4 (6-Mar-2025) client # Filesystem UUID: 46f38149-c1f1-4a74-96ca-737ecde35846 lb # Superblock backups stored on blocks: worker2 # Discarding device blocks: 0/262144 done client # Superblock backups stored on blocks: lb # 32768, 98304, 163840, 229376 worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # 32768, 98304, 163840, 229376 lb # worker2 # Filesystem UUID: 7967d1d0-0434-48a8-b218-64dc68f647f4 client # lb # Allocating group tables: 0/8 done worker2 # Superblock backups stored on blocks: client # Allocating group tables: 0/8 done lb # Writing inode tables: 0/8 done worker2 # 32768, 98304, 163840, 229376 client # Writing inode tables: 0/8 done lb # Creating journal (8192 blocks): done worker2 # client # Creating journal (8192 blocks): done lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Allocating group tables: 0/8 done client # Writing superblocks and filesystem accounting information: 0/8 done lb # worker2 # Writing inode tables: 0/8 done client # lb # Virtualisation disk image created. worker2 # Creating journal (8192 blocks): done client # Virtualisation disk image created. lb # Starting virtiofs daemons... worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Starting virtiofs daemons... lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # Virtualisation disk image created. client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # Starting virtiofs daemons... client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # Formatting '/build/vm-state-worker1/tmp.wIlLy48BwW', fmt=raw size=1073741824 worker1: QEMU running (pid 51) worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # Discarding device blocks: 0/262144 done worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes (finished: start all VMs, in 0.51 seconds) worker1 # Filesystem UUID: 60eb5a0c-8c20-428a-b205-a289251928cb client: waiting for unit network-addresses-eth1.service worker1 # Superblock backups stored on blocks: client: waiting for the VM to finish booting worker1 # 32768, 98304, 163840, 229376 worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # worker1 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker1 # Virtualisation disk image created. worker1 # Starting virtiofs daemons... worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test/init regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration", will be passed to user space. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.000000] Policy zone: DMA client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] random: crng init done lb # [ 0.000000] KASLR enabled worker1 # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] random: crng init done client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] Zone ranges: lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] Zone ranges: worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] DMA32 empty lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Normal empty client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Device empty lb # [ 0.000000] Normal empty worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] Device empty worker1 # [ 0.000000] Early memory node ranges client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] Movable zone start for each node lb # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000034] arm-pv: using stolen time PV worker1 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000645] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000818] Console: colour dummy device 80x25 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000826] printk: legacy console [tty0] enabled worker1 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] alternatives: applying boot alternatives client # [ 0.001017] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.001024] pid_max: default: 32768 minimum: 301 client # [ 0.001099] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.001260] landlock: Up and running. client # [ 0.001262] Yama: becoming mindful. client # [ 0.001918] LSM support for eBPF active worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test/init regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.002093] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qvl0xxkr124pdkzdvanphvcpy7197xra-nixos-system-lb-test/init regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.002127] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.004064] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] KASLR enabled client # [ 0.004069] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] random: crng init done client # [ 0.005349] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.005441] EFI services will not be available. worker2 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.005530] smp: Bringing up secondary CPUs ... worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.005538] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.005541] SMP: Total of 1 processors activated. worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.005546] CPU: All CPU(s) started at EL1 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] Zone ranges: client # [ 0.005557] CPU features: detected: Branch Target Identification lb # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.005562] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Normal empty client # [ 0.005565] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.000000] Device empty worker1 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Movable zone start for each node lb # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] Early memory node ranges client # [ 0.005568] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005572] CPU features: detected: Common not Private translations lb # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.005575] CPU features: detected: CRC32 instructions worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.005578] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.005581] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.005585] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.005588] CPU features: detected: E0PD worker2 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.005590] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.005593] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.005597] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.005600] CPU features: detected: Fine Grained Traps worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.005603] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.005608] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.005611] CPU features: detected: LSE atomic instructions worker2 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.005629] CPU features: detected: Privileged Access Never client # [ 0.005632] CPU features: detected: PMUv3 lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005635] CPU features: detected: RAS Extension Support worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005638] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005640] CPU features: detected: Random Number Generator lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.005643] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.005646] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.005649] CPU features: detected: TLB range maintenance instructions lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.005654] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test/init regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.005695] alternatives: applying system-wide alternatives worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.008615] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.008828] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) client # [ 0.009243] devtmpfs: initialized worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.011094] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.011117] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.011322] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.011326] 0 pages in range for non-PLT usage worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.011327] 508272 pages in range for PLT usage lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.011451] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.012232] DMI not present or invalid. lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.016072] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.018320] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000030] arm-pv: using stolen time PV lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.018507] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000030] arm-pv: using stolen time PV worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.018707] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.000426] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.018730] audit: initializing netlink subsys (disabled) lb # [ 0.000440] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000599] Console: colour dummy device 80x25 worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000606] printk: legacy console [tty0] enabled lb # [ 0.000624] Console: colour dummy device 80x25 client # [ 0.019268] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000631] printk: legacy console [tty0] enabled client # [ 0.019270] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.000792] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.019274] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000826] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000799] pid_max: default: 32768 minimum: 301 client # [ 0.019277] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.000833] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000873] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000997] landlock: Up and running. worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.019281] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.000915] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000999] Yama: becoming mindful. lb # [ 0.001045] landlock: Up and running. worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.001465] LSM support for eBPF active client # [ 0.019308] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.001047] Yama: becoming mindful. worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.001507] LSM support for eBPF active client # [ 0.019316] cpuidle: using governor ladder worker1 # [ 0.001605] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.019322] cpuidle: using governor menu worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.001654] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.001643] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.019513] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.001686] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.003506] rcu: Hierarchical SRCU implementation. client # [ 0.019528] ASID allocator initialised with 65536 entries lb # [ 0.003595] rcu: Hierarchical SRCU implementation. worker1 # [ 0.003511] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.020829] Serial: AMBA PL011 UART driver worker1 # [ 0.004766] fsl-mc MSI: its@8080000 domain created lb # [ 0.003600] rcu: Max phase no-delay instances is 1000. client # [ 0.026246] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.004844] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.004857] EFI services will not be available. client # [ 0.026406] printk: console [ttyAMA0] enabled lb # [ 0.004944] EFI services will not be available. worker1 # [ 0.004921] smp: Bringing up secondary CPUs ... worker1 # [ 0.004929] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.005058] smp: Bringing up secondary CPUs ... worker1 # [ 0.004932] SMP: Total of 1 processors activated. lb # [ 0.005066] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.004937] CPU: All CPU(s) started at EL1 lb # [ 0.005069] SMP: Total of 1 processors activated. worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.005074] CPU: All CPU(s) started at EL1 worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.004948] CPU features: detected: Branch Target Identification worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.005085] CPU features: detected: Branch Target Identification worker1 # [ 0.004953] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.005089] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.004956] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.005093] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.004960] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.005096] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.004964] CPU features: detected: Common not Private translations worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.005100] CPU features: detected: Common not Private translations worker1 # [ 0.004967] CPU features: detected: CRC32 instructions lb # [ 0.005103] CPU features: detected: CRC32 instructions worker1 # [ 0.004970] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.005106] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.144405] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.004973] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.144421] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.005110] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.004976] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.004979] CPU features: detected: E0PD client # [ 0.144427] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.005113] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.005116] CPU features: detected: E0PD worker1 # [ 0.004982] CPU features: detected: Enhanced Counter Virtualization client # [ 0.144432] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.005118] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.004985] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.144436] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.004989] CPU features: detected: Enhanced Virtualization Traps client # [ 0.144440] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.005125] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.004992] CPU features: detected: Fine Grained Traps client # [ 0.144445] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.005128] CPU features: detected: Fine Grained Traps worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.004996] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.144449] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005131] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.005000] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000030] arm-pv: using stolen time PV lb # [ 0.005136] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.005004] CPU features: detected: LSE atomic instructions worker1 # [ 0.005006] CPU features: detected: Privileged Access Never lb # [ 0.005139] CPU features: detected: LSE atomic instructions worker2 # [ 0.000400] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.005009] CPU features: detected: PMUv3 client # [ 0.151631] fbcon: Taking over console lb # [ 0.005142] CPU features: detected: Privileged Access Never client # [ 0.151644] ACPI: Interpreter disabled. worker2 # [ 0.000542] Console: colour dummy device 80x25 worker1 # [ 0.005012] CPU features: detected: RAS Extension Support lb # [ 0.005145] CPU features: detected: PMUv3 worker2 # [ 0.000550] printk: legacy console [tty0] enabled lb # [ 0.005147] CPU features: detected: RAS Extension Support worker1 # [ 0.005015] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.005017] CPU features: detected: Random Number Generator lb # [ 0.005150] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.000743] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000750] pid_max: default: 32768 minimum: 301 lb # [ 0.005153] CPU features: detected: Random Number Generator worker2 # [ 0.000822] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.005155] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.000966] landlock: Up and running. lb # [ 0.005158] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.000969] Yama: becoming mindful. worker2 # [ 0.001436] LSM support for eBPF active lb # [ 0.005161] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.001572] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.005166] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.005203] alternatives: applying system-wide alternatives worker2 # [ 0.001609] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.003466] rcu: Hierarchical SRCU implementation. lb # [ 0.008094] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.003471] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.004724] fsl-mc MSI: its@8080000 domain created client # [ 0.160654] iommu: Default domain type: Translated worker2 # [ 0.004814] EFI services will not be available. lb # [ 0.008256] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) client # [ 0.160664] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.008631] devtmpfs: initialized worker2 # [ 0.004892] smp: Bringing up secondary CPUs ... client # [ 0.161006] SCSI subsystem initialized worker2 # [ 0.004900] smp: Brought up 1 node, 1 CPU lb # [ 0.010412] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.004903] SMP: Total of 1 processors activated. worker1 # [ 0.005020] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.004908] CPU: All CPU(s) started at EL1 client # [ 0.162961] usbcore: registered new interface driver usbfs worker1 # [ 0.005023] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.010440] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.004919] CPU features: detected: Branch Target Identification client # [ 0.162993] usbcore: registered new interface driver hub worker1 # [ 0.005025] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.004923] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.163016] usbcore: registered new device driver usb worker1 # [ 0.005030] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.163295] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.004927] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.005067] alternatives: applying system-wide alternatives client # [ 0.163301] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.008013] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.004931] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.163311] PTP clock support registered client # [ 0.163366] EDAC MC: Ver: 3.0.0 worker2 # [ 0.004934] CPU features: detected: Common not Private translations client # [ 0.167986] scmi_core: SCMI protocol bus registered worker2 # [ 0.004938] CPU features: detected: CRC32 instructions worker1 # [ 0.008255] Memory: 1405628K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker1 # [ 0.008619] devtmpfs: initialized worker2 # [ 0.004940] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.168943] FPGA manager framework client # [ 0.169871] vgaarb: loaded worker1 # [ 0.010401] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.004944] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.170491] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.010430] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.004947] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004950] CPU features: detected: E0PD worker1 # [ 0.010677] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.004953] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.010684] 0 pages in range for non-PLT usage lb # [ 0.010648] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.010685] 508272 pages in range for PLT usage worker2 # [ 0.004956] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.010652] 0 pages in range for non-PLT usage worker1 # [ 0.010801] pinctrl core: initialized pinctrl subsystem lb # [ 0.010653] 508272 pages in range for PLT usage worker2 # [ 0.004959] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.011569] DMI not present or invalid. lb # [ 0.010742] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.004963] CPU features: detected: Fine Grained Traps client # [ 0.174177] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.014864] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.011489] DMI not present or invalid. client # [ 0.174214] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.017125] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.004966] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.014893] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.004971] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.017307] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.017093] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.004974] CPU features: detected: LSE atomic instructions worker1 # [ 0.017505] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.004977] CPU features: detected: Privileged Access Never lb # [ 0.017282] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.004979] CPU features: detected: PMUv3 worker1 # [ 0.017526] audit: initializing netlink subsys (disabled) client # [ 0.177958] netfs: FS-Cache loaded lb # [ 0.017482] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.004982] CPU features: detected: RAS Extension Support worker1 # [ 0.018074] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.178092] pnp: PnP ACPI: disabled lb # [ 0.017505] audit: initializing netlink subsys (disabled) worker2 # [ 0.004985] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.018076] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.018089] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.004988] CPU features: detected: Random Number Generator worker1 # [ 0.018080] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.018091] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.004990] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.018082] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.018094] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.004993] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.018087] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.018097] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.004996] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.018113] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018102] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.181944] NET: Registered PF_INET protocol family worker2 # [ 0.005000] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.018121] cpuidle: using governor ladder worker1 # [ 0.018127] cpuidle: using governor menu lb # [ 0.018145] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.182220] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.005038] alternatives: applying system-wide alternatives lb # [ 0.018154] cpuidle: using governor ladder worker1 # [ 0.018317] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018160] cpuidle: using governor menu worker2 # [ 0.007964] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.018332] ASID allocator initialised with 65536 entries worker1 # [ 0.019575] Serial: AMBA PL011 UART driver lb # [ 0.018344] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018358] ASID allocator initialised with 65536 entries worker2 # [ 0.008179] Memory: 1405708K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.024808] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.019567] Serial: AMBA PL011 UART driver worker2 # [ 0.008577] devtmpfs: initialized worker1 # [ 0.024971] printk: console [ttyAMA0] enabled worker2 # [ 0.010333] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.024943] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.025067] printk: console [ttyAMA0] enabled worker1 # [ 0.144264] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.010355] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.144285] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.010524] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.145057] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.144290] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.010528] 0 pages in range for non-PLT usage worker2 # [ 0.010529] 508272 pages in range for PLT usage lb # [ 0.145076] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.144294] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.010634] pinctrl core: initialized pinctrl subsystem lb # [ 0.145081] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.144299] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.011386] DMI not present or invalid. lb # [ 0.145085] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.144303] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.014617] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.145090] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.016818] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.144307] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.145094] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.144311] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.145098] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.145102] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.151597] fbcon: Taking over console worker1 # [ 0.151611] ACPI: Interpreter disabled. worker1 # [ 0.153471] iommu: Default domain type: Translated lb # [ 0.152594] fbcon: Taking over console worker1 # [ 0.153481] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.152610] ACPI: Interpreter disabled. lb # [ 0.154489] iommu: Default domain type: Translated worker1 # [ 0.155162] SCSI subsystem initialized lb # [ 0.154500] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.156244] SCSI subsystem initialized worker2 # [ 0.017013] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.017206] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.017228] audit: initializing netlink subsys (disabled) worker2 # [ 0.017749] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.017751] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.017754] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.017757] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.017762] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.163049] usbcore: registered new interface driver usbfs worker1 # [ 0.163080] usbcore: registered new interface driver hub worker2 # [ 0.017786] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.163371] usbcore: registered new interface driver usbfs worker1 # [ 0.163096] usbcore: registered new device driver usb worker2 # [ 0.017794] cpuidle: using governor ladder lb # [ 0.163404] usbcore: registered new interface driver hub worker2 # [ 0.017800] cpuidle: using governor menu worker1 # [ 0.163365] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.163420] usbcore: registered new device driver usb worker2 # [ 0.017984] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.163696] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.163371] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.017998] ASID allocator initialised with 65536 entries worker1 # [ 0.163381] PTP clock support registered worker2 # [ 0.019190] Serial: AMBA PL011 UART driver lb # [ 0.163702] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.163431] EDAC MC: Ver: 3.0.0 lb # [ 0.163713] PTP clock support registered lb # [ 0.163770] EDAC MC: Ver: 3.0.0 worker1 # [ 0.167911] scmi_core: SCMI protocol bus registered worker2 # [ 0.024452] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.024595] printk: console [ttyAMA0] enabled lb # [ 0.168320] scmi_core: SCMI protocol bus registered worker1 # [ 0.168862] FPGA manager framework worker1 # [ 0.169807] vgaarb: loaded lb # [ 0.169305] FPGA manager framework worker2 # [ 0.144519] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.170291] vgaarb: loaded worker1 # [ 0.170430] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.144538] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.170930] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.144544] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.171967] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.144548] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.171994] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.144552] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.172829] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.144557] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.172859] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.144561] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.144565] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.175650] netfs: FS-Cache loaded client # [ 0.210641] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.175747] pnp: PnP ACPI: disabled client # [ 0.210678] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.176553] netfs: FS-Cache loaded lb # [ 0.176666] pnp: PnP ACPI: disabled client # [ 0.210704] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.151946] fbcon: Taking over console worker2 # [ 0.151959] ACPI: Interpreter disabled. client # [ 0.210776] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.153830] iommu: Default domain type: Translated client # [ 0.210917] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.153840] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.210995] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.155573] SCSI subsystem initialized client # [ 0.211027] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.211062] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.179655] NET: Registered PF_INET protocol family client # [ 0.211163] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.211188] NET: Registered PF_XDP protocol family worker1 # [ 0.179910] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.180700] NET: Registered PF_INET protocol family client # [ 0.211205] PCI: CLS 0 bytes, default 64 client # [ 0.211439] Trying to unpack rootfs image as initramfs... lb # [ 0.180954] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.227916] kvm [1]: HYP mode not available worker2 # [ 0.163247] usbcore: registered new interface driver usbfs worker2 # [ 0.163279] usbcore: registered new interface driver hub worker2 # [ 0.163303] usbcore: registered new device driver usb worker2 # [ 0.163581] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.163587] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.163598] PTP clock support registered worker2 # [ 0.163638] EDAC MC: Ver: 3.0.0 worker2 # [ 0.168286] scmi_core: SCMI protocol bus registered worker2 # [ 0.169248] FPGA manager framework worker2 # [ 0.170231] vgaarb: loaded worker2 # [ 0.170868] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.172232] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.172262] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.173872] netfs: FS-Cache loaded worker2 # [ 0.173984] pnp: PnP ACPI: disabled worker2 # [ 0.179968] NET: Registered PF_INET protocol family worker2 # [ 0.180241] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.210803] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.210850] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.210878] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.209256] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.209308] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.209333] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.209403] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.209547] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.209633] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.213962] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.209668] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.214118] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.209703] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.214203] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.209794] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.214238] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.209821] NET: Registered PF_XDP protocol family worker1 # [ 0.209839] PCI: CLS 0 bytes, default 64 lb # [ 0.214274] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.210083] Trying to unpack rootfs image as initramfs... lb # [ 0.214377] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.214428] NET: Registered PF_XDP protocol family lb # [ 0.214446] PCI: CLS 0 bytes, default 64 lb # [ 0.214690] Trying to unpack rootfs image as initramfs... lb # [ 0.229643] kvm [1]: HYP mode not available worker1 # [ 0.228256] kvm [1]: HYP mode not available worker2 # [ 0.209124] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.209166] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.209191] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.209267] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.209423] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.209504] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.209539] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.209574] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.209705] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.209745] NET: Registered PF_XDP protocol family worker2 # [ 0.209762] PCI: CLS 0 bytes, default 64 worker2 # [ 0.209989] Trying to unpack rootfs image as initramfs... worker2 # [ 0.224644] kvm [1]: HYP mode not available client # [ 0.321533] Initialise system trusted keyrings client # [ 0.322310] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.323602] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.324354] 9p: Installing v9fs 9p2000 file system support client # [ 0.348382] Key type asymmetric registered client # [ 0.348410] Asymmetric key parser 'x509' registered client # [ 0.348486] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.350704] io scheduler mq-deadline registered client # [ 0.350713] io scheduler kyber registered lb # [ 0.323576] Initialise system trusted keyrings worker1 # [ 0.324008] Initialise system trusted keyrings worker1 # [ 0.324741] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.324301] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.325977] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.325579] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.326386] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.326797] 9p: Installing v9fs 9p2000 file system support client # [ 0.366657] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.367359] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.367763] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.367780] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.367793] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.367801] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.367823] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.367851] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.367927] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.367937] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.315424] Initialise system trusted keyrings client # [ 0.367942] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.367947] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.316151] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.367952] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.317434] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.368021] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.318199] 9p: Installing v9fs 9p2000 file system support client # [ 0.368462] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.368647] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.368664] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.368694] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.368710] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.369159] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.369342] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.369357] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.369387] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.369852] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.370031] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.370046] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.370075] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.370541] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.354853] Key type asymmetric registered client # [ 0.370724] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.354880] Asymmetric key parser 'x509' registered lb # [ 0.355113] Key type asymmetric registered client # [ 0.370739] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.355144] Asymmetric key parser 'x509' registered worker1 # [ 0.354953] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.370768] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.357030] io scheduler mq-deadline registered lb # [ 0.355219] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.370786] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.357041] io scheduler kyber registered lb # [ 0.357291] io scheduler mq-deadline registered lb # [ 0.357302] io scheduler kyber registered client # [ 0.371244] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.371425] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.371454] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.371900] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.372081] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.372110] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.372516] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.372692] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.372941] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.373125] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.373155] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.366612] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.373596] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.373792] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.367085] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.373822] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.374272] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.368071] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.374453] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.368485] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.368537] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.374482] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.346923] Key type asymmetric registered lb # [ 0.368961] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.368503] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.346952] Asymmetric key parser 'x509' registered lb # [ 0.368977] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.368528] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.347027] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.349128] io scheduler mq-deadline registered lb # [ 0.368990] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.368537] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.349139] io scheduler kyber registered worker1 # [ 0.368558] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.368999] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.369019] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.368584] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.368682] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.369044] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.368691] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.369116] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.368697] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.369126] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.368703] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.369132] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.368708] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.369138] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.368794] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.369143] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.369234] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.369227] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.369416] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.369663] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.369433] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.369848] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.369462] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.359019] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.369865] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.369478] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.423089] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.369895] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.369927] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.423405] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.369912] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.370107] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.423424] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.370123] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.370381] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.423453] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.360397] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.370564] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.370152] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.423921] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.360819] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.370580] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.424101] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.370609] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.360836] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.424117] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.424146] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.360849] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.390125] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.424720] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.360857] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.390312] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.424731] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.390329] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.360877] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.424737] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.390358] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.360902] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.424781] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.390806] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.360976] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.424828] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.360985] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.424875] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.360992] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.360997] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.424922] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.361002] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.424969] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.361059] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.425017] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.361517] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.425063] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.361703] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.425110] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.361720] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425156] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.361750] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.394706] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.361766] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.425224] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.394909] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.362219] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.425269] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.394926] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.362402] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.425291] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.394955] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.362418] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425312] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.395426] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.362448] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.425333] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.395610] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.425354] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.395626] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.399042] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.425375] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.395655] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.399072] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.381482] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.425397] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.395673] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.381670] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.399103] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.425423] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.396122] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.381685] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.399122] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.425444] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.396308] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.381715] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.399612] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.425466] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.396338] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.382181] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.399805] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425488] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.382365] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.396785] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.399835] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.425510] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.382380] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.396971] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.400289] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.425531] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.382410] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.397000] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.400477] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425552] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.382429] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.397384] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.425572] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.400507] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.397563] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.425593] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.400894] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.425623] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.397851] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.401076] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.398036] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425644] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.401359] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.425670] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.398066] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.401546] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.425679] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.398527] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.401576] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.425684] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.398718] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.426490] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.402024] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.398748] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.402220] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.399194] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.402250] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.399378] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.402700] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.399407] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.402885] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.399866] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.402915] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.400159] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.403394] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.400177] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.403656] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.400206] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.403674] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.400660] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.403704] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.400840] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.400856] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.404170] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.394951] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.404353] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.400886] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.395159] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.404369] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.401442] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.395190] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.404399] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.401453] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.395683] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.404958] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.395870] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.401458] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.404969] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.395899] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.401501] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.404975] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.396287] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.401548] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.405021] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.396469] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.401593] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.405067] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.396720] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.401637] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.396907] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.405114] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.401682] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.396937] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.405161] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.401727] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.397402] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.405207] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.397588] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.401773] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.405252] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.397618] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.401818] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.481572] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.405298] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.398078] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.401863] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.405344] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.398265] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.401932] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.398295] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.405391] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.401977] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.398749] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.405465] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.401998] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.399032] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.405512] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.402019] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.399048] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.405533] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.402040] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.399078] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.405555] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.402061] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.399544] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.405576] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.402082] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.399727] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.405598] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.402103] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.399742] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.405620] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.402126] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.399772] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.405641] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.402148] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.400340] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.405666] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.402169] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.400351] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.405688] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.402191] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.400357] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.405710] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.402213] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.400402] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.405732] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.402234] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.491848] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.405755] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.402254] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.400449] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.402275] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.405777] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.400496] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.402295] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.405798] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.400543] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.405819] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.402316] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.405841] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.402337] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.400590] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.402363] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.496078] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.400636] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.405862] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.402372] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.405884] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.497972] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.400682] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.405910] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.402377] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.405919] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.400728] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.405924] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.400774] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.406728] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.400843] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.400889] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.400910] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.400932] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.400953] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.400974] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.400996] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.463280] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.401017] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.401041] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.401064] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.401085] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.401108] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.401130] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.401151] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.401172] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.511835] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.401193] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.513802] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.401214] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.401235] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.471089] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 62855 usecs worker2 # [ 0.401257] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.401282] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.401291] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.517245] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.401296] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.402107] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.519934] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.488785] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.530662] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.492246] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.532547] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.494350] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.534835] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.495088] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.497093] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.478517] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.547797] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.550352] msm_serial: driver initialized lb # [ 0.505342] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.507039] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.551025] SuperH (H)SCI(F) driver initialized client # [ 0.551086] STM32 USART driver initialized worker1 # [ 0.509162] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.488341] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.515804] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.518046] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.521737] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.518614] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.520797] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.522625] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.524391] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.527990] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.526188] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.499423] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.530610] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.501280] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.503205] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.505240] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.540005] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.584681] loop: module loaded lb # [ 0.541838] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.584854] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.515030] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.585654] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.516794] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.518591] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.544622] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.546925] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.591048] megasas: 07.734.00.00-rc1 client # [ 0.591722] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.555690] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.528952] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.558247] msm_serial: driver initialized worker1 # [ 0.558410] SuperH (H)SCI(F) driver initialized lb # [ 0.564009] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.559551] STM32 USART driver initialized lb # [ 0.566572] msm_serial: driver initialized lb # [ 0.566731] SuperH (H)SCI(F) driver initialized client # [ 0.607559] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.566789] STM32 USART driver initialized client # [ 0.607610] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.609752] Using buffer write method worker2 # [ 0.539075] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.609820] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.551256] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.553871] msm_serial: driver initialized worker2 # [ 0.554010] SuperH (H)SCI(F) driver initialized worker2 # [ 0.554063] STM32 USART driver initialized client # [ 0.623241] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.623272] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.624922] Using buffer write method client # [ 0.624949] Concatenating MTD devices: client # [ 0.624953] (0): "0.flash" client # [ 0.624957] (1): "0.flash" client # [ 0.624968] into device "0.flash" worker1 # [ 0.589206] loop: module loaded worker1 # [ 0.589378] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.590124] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.595054] megasas: 07.734.00.00-rc1 worker1 # [ 0.595736] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.599957] loop: module loaded lb # [ 0.600147] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.600867] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.582410] loop: module loaded worker2 # [ 0.582603] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.607826] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.607891] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.609478] Using buffer write method worker1 # [ 0.609549] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.591589] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.611616] megasas: 07.734.00.00-rc1 lb # [ 0.612336] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.614231] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.614258] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.615896] Using buffer write method worker2 # [ 0.594737] megasas: 07.734.00.00-rc1 lb # [ 0.615971] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.595478] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.623226] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.618057] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.623273] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.624863] Using buffer write method lb # [ 0.618081] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.624890] Concatenating MTD devices: worker1 # [ 0.624894] (0): "0.flash" worker1 # [ 0.624898] (1): "0.flash" worker1 # [ 0.624907] into device "0.flash" worker2 # [ 0.606914] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.638422] Using buffer write method worker2 # [ 0.606942] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.638461] Concatenating MTD devices: lb # [ 0.638465] (0): "0.flash" worker2 # [ 0.609418] Using buffer write method lb # [ 0.638474] (1): "0.flash" lb # [ 0.638478] into device "0.flash" worker2 # [ 0.609504] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.611217] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.611239] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.612834] Using buffer write method worker2 # [ 0.612860] Concatenating MTD devices: worker2 # [ 0.612864] (0): "0.flash" worker2 # [ 0.612868] (1): "0.flash" worker2 # [ 0.612877] into device "0.flash" client # [ 0.867637] Freeing initrd memory: 26972K client # [ 0.873657] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.877375] thunder_xcv, ver 1.0 client # [ 0.877421] thunder_bgx, ver 1.0 client # [ 0.877442] nicpf, ver 1.0 client # [ 0.878066] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.878074] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.878100] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.878107] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.878135] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.878140] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.878163] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.878169] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.878299] sky2: driver version 1.30 client # [ 0.886927] usbcore: registered new interface driver usb-storage client # [ 0.886982] usbcore: registered new interface driver usbserial_generic client # [ 0.886998] usbserial: USB Serial support registered for generic client # [ 0.887609] hv_vmbus: registering driver hyperv_keyboard client # [ 0.891779] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.891806] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.892000] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.894470] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.895151] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305) client # [ 0.895490] i2c_dev: i2c /dev entries driver client # [ 0.900181] sdhci: Secure Digital Host Controller Interface driver client # [ 0.900193] sdhci: Copyright(c) Pierre Ossman client # [ 0.900458] Synopsys Designware Multimedia Card Interface Driver client # [ 0.900828] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.902334] hid: raw HID events driver (C) Jiri Kosina client # [ 0.902528] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.902835] hub 1-0:1.0: USB hub found client # [ 0.902855] hub 1-0:1.0: 6 ports detected client # [ 0.908248] usbcore: registered new interface driver usbhid client # [ 0.908258] usbhid: USB HID core driver client # [ 0.910566] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.912072] drop_monitor: Initializing network drop monitor service client # [ 0.912214] NET: Registered PF_INET6 protocol family client # [ 0.915234] Segment Routing with IPv6 client # [ 0.915251] In-situ OAM (IOAM) with IPv6 lb # [ 0.870694] Freeing initrd memory: 26948K client # [ 0.915293] NET: Registered PF_PACKET protocol family worker1 # [ 0.870675] Freeing initrd memory: 26976K client # [ 0.916905] 9pnet: Installing 9P2000 support client # [ 0.916957] Key type dns_resolver registered worker1 # [ 0.876542] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.876853] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.880183] thunder_xcv, ver 1.0 worker1 # [ 0.880227] thunder_bgx, ver 1.0 worker1 # [ 0.880250] nicpf, ver 1.0 lb # [ 0.880638] thunder_xcv, ver 1.0 worker1 # [ 0.880806] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.923528] registered taskstats version 1 lb # [ 0.880690] thunder_bgx, ver 1.0 lb # [ 0.880712] nicpf, ver 1.0 client # [ 0.923687] Loading compiled-in X.509 certificates worker1 # [ 0.880813] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.881294] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.880836] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.881301] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.880844] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.881326] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.880873] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.881333] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.880878] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.881365] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.880917] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.881371] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.880922] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.881395] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.881055] sky2: driver version 1.30 lb # [ 0.881401] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.932352] Demotion targets for Node 0: null lb # [ 0.881550] sky2: driver version 1.30 client # [ 0.932456] Key type .fscrypt registered worker1 # [ 0.890156] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.932462] Key type fscrypt-provisioning registered worker1 # [ 0.890192] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.932558] ima: No TPM chip found, activating TPM-bypass! client # [ 0.932577] ima: Allocated hash algorithm: sha1 worker1 # [ 0.890410] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.932597] ima: No architecture policies found worker2 # [ 0.867490] Freeing initrd memory: 26944K lb # [ 0.890296] usbcore: registered new interface driver usb-storage lb # [ 0.890362] usbcore: registered new interface driver usbserial_generic client # [ 0.936679] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.890384] usbserial: USB Serial support registered for generic worker1 # [ 0.893029] usbcore: registered new interface driver usb-storage lb # [ 0.891758] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.893130] usbcore: registered new interface driver usbserial_generic lb # [ 0.891789] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.893144] usbserial: USB Serial support registered for generic lb # [ 0.891992] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.893737] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.896690] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.897547] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.873522] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.898220] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.897575] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305) worker1 # [ 0.897876] i2c_dev: i2c /dev entries driver lb # [ 0.898248] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305) lb # [ 0.898548] i2c_dev: i2c /dev entries driver worker2 # [ 0.877325] thunder_xcv, ver 1.0 worker2 # [ 0.877370] thunder_bgx, ver 1.0 worker2 # [ 0.877409] nicpf, ver 1.0 worker2 # [ 0.877987] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.877995] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.878019] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.903179] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.878028] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.903190] sdhci: Copyright(c) Pierre Ossman lb # [ 0.903944] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.878056] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.903457] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.903956] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.903823] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.878062] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.904231] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.905318] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.878084] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.904603] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.905560] usbcore: registered new interface driver usbhid worker2 # [ 0.878090] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.906150] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.905566] usbhid: USB HID core driver worker2 # [ 0.878223] sky2: driver version 1.30 lb # [ 0.906394] usbcore: registered new interface driver usbhid lb # [ 0.906401] usbhid: USB HID core driver client # [ 0.954465] clk: Disabling unused clocks worker1 # [ 0.910839] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.954490] PM: genpd: Disabling unused power domains lb # [ 0.911331] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.911179] hub 1-0:1.0: USB hub found worker2 # [ 0.886856] usbcore: registered new interface driver usb-storage worker1 # [ 0.911199] hub 1-0:1.0: 6 ports detected lb # [ 0.912351] hub 1-0:1.0: USB hub found lb # [ 0.912840] hub 1-0:1.0: 6 ports detected worker2 # [ 0.888233] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.888264] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.913325] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.888520] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.914310] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.958616] Freeing unused kernel memory: 4736K client # [ 0.958811] Run /init as init process worker1 # [ 0.915992] drop_monitor: Initializing network drop monitor service worker1 # [ 0.916225] NET: Registered PF_INET6 protocol family worker2 # [ 0.891006] usbcore: registered new interface driver usbserial_generic lb # [ 0.917000] drop_monitor: Initializing network drop monitor service worker1 # [ 0.918153] Segment Routing with IPv6 worker2 # [ 0.891023] usbserial: USB Serial support registered for generic lb # [ 0.917174] NET: Registered PF_INET6 protocol family worker1 # [ 0.918172] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.891719] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.918200] NET: Registered PF_PACKET protocol family lb # [ 0.919081] Segment Routing with IPv6 lb # [ 0.919099] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.894743] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.919126] NET: Registered PF_PACKET protocol family worker2 # [ 0.894772] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305) worker1 # [ 0.919843] 9pnet: Installing 9P2000 support worker2 # [ 0.896875] i2c_dev: i2c /dev entries driver worker1 # [ 0.919893] Key type dns_resolver registered lb # [ 0.920762] 9pnet: Installing 9P2000 support lb # [ 0.920812] Key type dns_resolver registered worker2 # [ 0.900626] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.900638] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.900904] Synopsys Designware Multimedia Card Interface Driver client # [ 0.972681] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.901275] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.926800] registered taskstats version 1 worker2 # [ 0.902808] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.926950] Loading compiled-in X.509 certificates lb # [ 0.927611] registered taskstats version 1 worker2 # [ 0.906212] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.927758] Loading compiled-in X.509 certificates worker2 # [ 0.907288] hub 1-0:1.0: USB hub found worker2 # [ 0.907762] hub 1-0:1.0: 6 ports detected worker2 # [ 0.908854] usbcore: registered new interface driver usbhid worker2 # [ 0.908864] usbhid: USB HID core driver worker1 # [ 0.935736] Demotion targets for Node 0: null worker1 # [ 0.935847] Key type .fscrypt registered worker1 # [ 0.935854] Key type fscrypt-provisioning registered lb # [ 0.936605] Demotion targets for Node 0: null worker1 # [ 0.935947] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.936715] Key type .fscrypt registered worker1 # [ 0.935967] ima: Allocated hash algorithm: sha1 lb # [ 0.936722] Key type fscrypt-provisioning registered worker2 # [ 0.911254] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.935989] ima: No architecture policies found lb # [ 0.936813] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.912831] drop_monitor: Initializing network drop monitor service lb # [ 0.936832] ima: Allocated hash algorithm: sha1 worker2 # [ 0.913026] NET: Registered PF_INET6 protocol family worker1 # [ 0.939929] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.936854] ima: No architecture policies found worker2 # [ 0.915983] Segment Routing with IPv6 worker2 # [ 0.916003] In-situ OAM (IOAM) with IPv6 lb # [ 0.940868] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.916032] NET: Registered PF_PACKET protocol family worker2 # [ 0.917670] 9pnet: Installing 9P2000 support worker2 # [ 0.917724] Key type dns_resolver registered worker2 # [ 0.924407] registered taskstats version 1 worker2 # [ 0.924548] Loading compiled-in X.509 certificates worker1 # [ 0.958406] clk: Disabling unused clocks worker1 # [ 0.958429] PM: genpd: Disabling unused power domains worker2 # [ 0.933455] Demotion targets for Node 0: null lb # [ 0.958794] clk: Disabling unused clocks worker2 # [ 0.933567] Key type .fscrypt registered lb # [ 0.958822] PM: genpd: Disabling unused power domains worker2 # [ 0.933574] Key type fscrypt-provisioning registered worker2 # [ 0.933670] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.933688] ima: Allocated hash algorithm: sha1 worker2 # [ 0.933709] ima: No architecture policies found worker1 # [ 0.962642] Freeing unused kernel memory: 4736K worker2 # [ 0.937767] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.962846] Run /init as init process lb # [ 0.963013] Freeing unused kernel memory: 4736K lb # [ 0.963205] Run /init as init process lb # [ 0.977171] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.980677] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.955574] clk: Disabling unused clocks worker2 # [ 0.955606] PM: genpd: Disabling unused power domains worker2 # [ 0.959717] Freeing unused kernel memory: 4736K worker2 # [ 0.959921] Run /init as init process worker2 # [ 0.977034] systemd[1]: Successfully made /usr/ read-only. client # [ 1.150565] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.158528] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.159029] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.154951] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.303031] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.309053] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.309115] systemd[1]: Detected virtualization qemu. client # [ 1.309192] systemd[1]: Detected architecture arm64. client # [ 1.309215] systemd[1]: Running in initrd. client # [ 1.310183] systemd[1]: Initializing machine ID from random generator. client # [ 1.310456] systemd[1]: Hostname set to . lb # [ 1.311470] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.311060] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.316531] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.316584] systemd[1]: Detected virtualization qemu. worker1 # [ 1.317119] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.316653] systemd[1]: Detected architecture arm64. lb # [ 1.316674] systemd[1]: Running in initrd. worker1 # [ 1.317176] systemd[1]: Detected virtualization qemu. lb # [ 1.317530] systemd[1]: Initializing machine ID from random generator. lb # [ 1.317780] systemd[1]: Hostname set to . worker1 # [ 1.317250] systemd[1]: Detected architecture arm64. worker1 # [ 1.317274] systemd[1]: Running in initrd. worker1 # [ 1.318204] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.307449] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.336658] systemd[1]: Hostname set to . client # [ 1.390746] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.313481] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.313538] systemd[1]: Detected virtualization qemu. worker2 # [ 1.313617] systemd[1]: Detected architecture arm64. worker2 # [ 1.313640] systemd[1]: Running in initrd. worker2 # [ 1.314547] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.314854] systemd[1]: Hostname set to . lb # [ 1.391128] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.394616] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.391138] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.510574] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.510980] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.514526] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.510923] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.567066] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.592099] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.645949] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.622211] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.668972] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.632912] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.633868] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.674706] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.634924] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.636233] systemd[1]: Reached target Path Units. lb # [ 1.636263] systemd[1]: Reached target Slice Units. lb # [ 1.636280] systemd[1]: Reached target Swaps. lb # [ 1.636296] systemd[1]: Reached target Timer Units. lb # [ 1.636443] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.636594] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.636716] systemd[1]: Listening on Journal Sockets. lb # [ 1.636829] systemd[1]: Listening on udev Control Socket. lb # [ 1.636929] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.636948] systemd[1]: Reached target Socket Units. lb # [ 1.645026] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.646004] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.655113] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.662138] systemd[1]: Starting Journal Service... worker2 # [ 1.637353] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.673444] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.676105] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.676203] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.677868] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.683239] systemd[1]: Starting Load Kernel Modules... lb # [ 1.683347] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.687551] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.663414] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.697119] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.696112] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.671122] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.698574] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.698633] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.698671] systemd[1]: Reached target Path Units. worker1 # [ 1.698693] systemd[1]: Reached target Slice Units. worker1 # [ 1.698715] systemd[1]: Reached target Swaps. worker1 # [ 1.698738] systemd[1]: Reached target Timer Units. worker1 # [ 1.698948] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.699120] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.699286] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.699438] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.699560] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.699585] systemd[1]: Reached target Socket Units. worker1 # [ 1.713587] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.714918] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.759305] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.715062] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.715909] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.726330] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.770245] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.771541] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.772843] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.773914] systemd[1]: Reached target Path Units. lb # [ 1.727338] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.774783] systemd[1]: Reached target Slice Units. client # [ 1.775611] systemd[1]: Reached target Swaps. client # [ 1.776398] systemd[1]: Reached target Timer Units. client # [ 1.777407] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.778665] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.779784] systemd[1]: Listening on Journal Sockets. client # [ 1.780775] systemd[1]: Listening on udev Control Socket. client # [ 1.781807] systemd[1]: Listening on udev Kernel Socket. client # [ 1.782748] systemd[1]: Reached target Socket Units. client # [ 1.785158] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.786319] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.748745] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.794664] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.750642] systemd[1]: Starting Journal Service... client # [ 1.802821] systemd[1]: Starting Journal Service... lb # [ 1.758844] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.761128] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.762073] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.767583] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.744582] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.770089] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.774881] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.753170] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.754320] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.779312] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.755687] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.756702] systemd[1]: Reached target Path Units. worker2 # [ 1.757489] systemd[1]: Reached target Slice Units. worker2 # [ 1.758274] systemd[1]: Reached target Swaps. client # [ 1.826762] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.759022] systemd[1]: Reached target Timer Units. client # [ 1.827730] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.759993] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.761204] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.762301] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.763325] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.786619] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.787405] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.764425] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.765295] systemd[1]: Reached target Socket Units. lb # [ 1.788167] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.788407] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.788416] [drm] features: -context_init worker2 # [ 1.767741] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.789119] [drm] number of scanouts: 1 lb # [ 1.789137] [drm] number of cap sets: 0 worker2 # [ 1.768866] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.776189] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.849112] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.810722] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.787169] systemd[1]: Starting Journal Service... worker1 # [ 1.812713] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.859159] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.815533] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.815896] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.815924] systemd[1]: Reached target Local File Systems. client # [ 1.861373] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.862043] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.819808] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.819822] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.827949] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.871611] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.834835] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.811301] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.811417] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.846522] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.849291] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.843318] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.830629] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.856539] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.856773] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.856783] [drm] features: -context_init worker1 # [ 1.857475] [drm] number of scanouts: 1 worker1 # [ 1.857493] [drm] number of cap sets: 0 worker1 # [ 1.861160] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.840309] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.915185] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.918821] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.875541] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.884355] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.859001] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.859736] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.878836] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.878853] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.862614] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.936995] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.899030] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.899356] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.899384] systemd[1]: Reached target Local File Systems. lb # [ 1.899304] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.903294] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.913022] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.892759] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.958907] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.965165] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.965423] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.965432] [drm] features: -context_init client # [ 1.966172] [drm] number of scanouts: 1 client # [ 1.966190] [drm] number of cap sets: 0 lb # [ 1.931677] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.907531] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.909725] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.936769] systemd[1]: Started Journal Service. worker1 # [ 1.939026] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 1.912202] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.983202] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.983553] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.983583] systemd[1]: Reached target Local File Systems. worker2 # [ 1.920994] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.932292] systemd-modules-load[73]: Inserted module 'dm_mod' worker2 # [ 1.921260] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.921270] [drm] features: -context_init worker2 # [ 1.921994] [drm] number of scanouts: 1 lb # [ 1.933987] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker2 # [ 1.922012] [drm] number of cap sets: 0 lb # [ 1.935081] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 1.990852] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.954462] systemd[1]: Finished Load Kernel Modules. lb # [ 1.941262] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 1.942371] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 1.957603] systemd[1]: Starting Apply Kernel Variables... lb # [ 1.943647] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 1.998819] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.998836] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.956889] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.943205] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.943223] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.018735] Console: switching to colour frame buffer device 160x50 lb # [ 1.976172] systemd[1]: Finished Create System Files and Directories. client # [ 2.020569] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.021826] systemd-modules-load[74]: Module 'virtio_balloon' is built in client # [ 2.022914] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 1.969391] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.969761] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.039491] systemd[1]: Started Journal Service. worker2 # [ 1.969788] systemd[1]: Reached target Local File Systems. worker2 # [ 1.973692] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.984498] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.999408] systemd[1]: Started Journal Service. worker1 # [ 1.988174] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 1.989302] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 1.990373] systemd-modules-load[73]: Module 'virtio_console' is built in worker1 # [ 1.991411] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 1.975103] Console: switching to colour frame buffer device 160x50 client # [ 2.052373] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.040769] systemd[1]: Starting Create System Files and Directories... client # [ 2.048628] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 2.052163] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.010010] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.011207] systemd[1]: Starting Create System Files and Directories... client # [ 2.057599] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.003700] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.016732] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker1 # [ 2.027198] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.019572] systemd[1]: Finished Load Kernel Modules. client # [ 2.080400] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.084193] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.027251] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.045072] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.015374] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.016650] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.047417] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.024463] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.044572] systemd[1]: Started Journal Service. worker2 # [ 2.032314] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.062702] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.035419] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.116294] systemd[1]: Finished Create System Files and Directories. client # [ 2.120570] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.053198] systemd[1]: Starting Create System Files and Directories... client # [ 2.130392] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.062227] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker2 # [ 2.073069] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.120257] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.122721] systemd[1]: Starting Virtual Console Setup... lb # [ 2.123850] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.095872] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.109273] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.183925] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.172483] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.175875] systemd[1]: Finished Virtual Console Setup. client # [ 2.232439] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.235650] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.172103] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.216474] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.219879] systemd[1]: Finished Virtual Console Setup. lb # [ 2.746713] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.747720] systemd[1]: Reached target System Initialization. lb # [ 2.752152] systemd[1]: Reached target Basic System. worker1 # [ 2.780118] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.781073] systemd[1]: Reached target System Initialization. worker1 # [ 2.782090] systemd[1]: Reached target Basic System. client # [ 2.834160] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.835121] systemd[1]: Reached target System Initialization. client # [ 2.835941] systemd[1]: Reached target Basic System. worker2 # [ 2.865449] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.866367] systemd[1]: Reached target System Initialization. worker2 # [ 2.867184] systemd[1]: Reached target Basic System. lb # [ 2.908431] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.922062] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.967036] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.932222] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.946471] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.961840] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.008618] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.972151] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.015892] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.000071] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.005419] systemd[1]: Reached target Initrd Root Device. lb # [ 3.009796] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.065467] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.068153] systemd[1]: Reached target Initrd Root Device. client # [ 3.071010] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.032886] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.042527] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.046918] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.029903] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.064290] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.039584] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.069802] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.044741] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.074186] systemd[1]: Mounting /sysroot... client # [ 3.125627] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.131031] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.092931] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.099552] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.146250] systemd[1]: Mounting /sysroot... worker1 # [ 3.104791] systemd[1]: Mounting /sysroot... lb # [ 3.123279] EXT4-fs (vda): mounted filesystem ecc6f207-dc8e-431d-b120-fb330412fd42 r/w with ordered data mode. Quota mode: none. lb # [ 3.117556] systemd[1]: Mounted /sysroot. lb # [ 3.118619] systemd[1]: Reached target Initrd Root File System. lb # [ 3.122499] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.185292] EXT4-fs (vda): mounted filesystem 46f38149-c1f1-4a74-96ca-737ecde35846 r/w with ordered data mode. Quota mode: none. client # [ 3.177103] systemd[1]: Mounted /sysroot. client # [ 3.179209] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.108524] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.151807] EXT4-fs (vda): mounted filesystem 60eb5a0c-8c20-428a-b205-a289251928cb r/w with ordered data mode. Quota mode: none. worker1 # [ 3.142367] systemd[1]: Mounted /sysroot. worker2 # [ 3.112999] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.144823] systemd[1]: Reached target Initrd Root File System. client # [ 3.188136] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.118885] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.147857] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.151913] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.155711] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.156766] systemd[1]: Reloading... client # [ 3.213188] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.175598] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.220112] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.222555] systemd[1]: Reloading... worker1 # [ 3.180119] systemd[1]: Reload requested from client PID 117 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.185385] systemd[1]: Reloading... worker2 # [ 3.165328] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.172182] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.175418] systemd[1]: Mounting /sysroot... worker2 # [ 3.228574] EXT4-fs (vda): mounted filesystem 7967d1d0-0434-48a8-b218-64dc68f647f4 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.216978] systemd[1]: Mounted /sysroot. worker2 # [ 3.218035] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.224704] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.247839] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.254186] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.258993] systemd[1]: Reloading... lb # [ 3.381141] systemd[1]: Reloading finished in 226 ms. client # [ 3.430846] systemd[1]: Reloading finished in 210 ms. worker1 # [ 3.388606] systemd[1]: Reloading finished in 206 ms. lb # [ 3.410190] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... lb # [ 3.413079] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... lb # [ 3.419188] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.421671] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.423446] systemd-sysroot-fstab-check[117]: Requesting initrd-fs.target/start/replace... client # [ 3.467031] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.423817] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.426161] systemd-sysroot-fstab-check[117]: Requesting swap.target/start/replace... client # [ 3.469824] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.472864] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.432475] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.476859] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.434338] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.478481] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.435520] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.463228] systemd[1]: Reloading finished in 205 ms. worker2 # [ 3.497470] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.501557] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.509479] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.512253] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.514245] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.687331] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.698642] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.713552] systemd[1]: Mounting /sysroot/run... lb # [ 3.726783] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.757201] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.764272] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.776796] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.791050] systemd[1]: Mounting /sysroot/run... lb # [ 3.803240] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.804250] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.805970] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.850698] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.835549] fuse: init (API version 7.45) worker1 # [ 3.824153] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.867819] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.872233] systemd[1]: Mounting /sysroot/run... lb # [ 3.828184] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.849937] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.850787] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.890440] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.864640] virtiofs virtio7: discovered new tag: shared lb # [ 3.865391] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.828189] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.874280] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.877579] fuse: init (API version 7.45) lb # [ 3.877408] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 3.838048] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.872401] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.873544] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.894985] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.895798] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.853783] systemd[1]: Mounting /sysroot/run... client # [ 3.926705] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.882931] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.884872] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 3.888459] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.948935] fuse: init (API version 7.45) lb # [ 3.894393] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 3.867157] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.912255] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.913047] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 3.920247] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.920994] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.968449] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.883684] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.969256] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.967293] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.927182] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.984169] virtiofs virtio7: discovered new tag: shared client # [ 3.984922] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 3.931676] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.934877] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.935597] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 3.993116] virtiofs virtio8: discovered new tag: xchg client # [ 3.993863] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 3.939192] systemd[1]: Mounted /sysroot/nix/store. lb # [ 3.941211] systemd[1]: Reached target Initrd File Systems. lb # [ 3.943334] systemd[1]: Starting Find NixOS closure... worker1 # [ 3.945598] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.951434] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.955087] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.013444] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.959602] fuse: init (API version 7.45) client # [ 4.017072] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.945051] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.968273] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.981143] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.969096] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.982268] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.026979] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.984626] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.032880] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.996303] systemd[1]: Finished Find NixOS closure. worker2 # [ 3.985707] virtiofs virtio7: discovered new tag: shared client # [ 4.042099] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 3.986514] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.999606] systemd[1]: Reached target Initrd Default Target. lb # [ 4.006137] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.052685] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.997725] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.998518] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.055373] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.993035] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.996290] systemd[1]: Mounted /sysroot/run. lb # [ 4.037521] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.040874] systemd[1]: Stopped target Basic System. worker2 # [ 4.013074] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.041933] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.015965] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.045163] systemd[1]: Stopped target Path Units. worker2 # [ 4.018282] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.046753] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.050823] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.056672] systemd[1]: Stopped target Slice Units. lb # [ 4.058475] systemd[1]: Stopped target Socket Units. lb # [ 4.060126] systemd[1]: Stopped target System Initialization. worker2 # [ 4.034266] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.035356] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.064184] systemd[1]: Stopped target Swaps. lb # [ 4.065906] systemd[1]: Stopped target Timer Units. lb # [ 4.067621] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 4.040088] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.070890] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.074136] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.077098] systemd[1]: Stopped Find NixOS closure. lb # [ 4.079740] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.085945] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.090227] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.094748] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.099420] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.100489] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.105230] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.106359] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.109154] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.110086] systemd[1]: Stopped target Local File Systems. lb # [ 4.110862] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.111810] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.112935] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.113717] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.114731] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.087040] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.115747] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.087886] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.090020] systemd[1]: Starting Find NixOS closure... lb # [ 4.121170] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.124232] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 4.098327] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.127968] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.129445] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.130349] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.131291] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.132401] systemd[1]: systemd-udevd.service: Consumed 1.091s CPU time over 2.306s wall clock time, 17.1M memory peak. lb # [ 4.133794] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.134790] systemd[1]: Closed udev Control Socket. lb # [ 4.135524] systemd[1]: Starting Cleanup udev Database... lb # [ 4.140222] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.144160] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.145339] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.146583] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.123142] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.152123] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.153147] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.139214] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.142354] systemd[1]: Reached target Initrd Default Target. lb # [ 4.173016] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 4.175456] systemd[1]: Finished Cleanup udev Database. lb # [ 4.177002] systemd[1]: Reached target Switch Root. worker2 # [ 4.151130] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.180280] systemd[1]: Starting NixOS Activation... worker2 # [ 4.176159] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 4.180240] systemd[1]: Stopped target Basic System. worker2 # [ 4.185284] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.186119] systemd[1]: Stopped target Path Units. worker2 # [ 4.187913] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 4.190311] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 4.196407] systemd[1]: Stopped target Slice Units. worker2 # [ 4.198123] systemd[1]: Stopped target Socket Units. worker2 # [ 4.199882] systemd[1]: Stopped target System Initialization. worker2 # [ 4.203196] systemd[1]: Stopped target Swaps. worker2 # [ 4.204149] systemd[1]: Stopped target Timer Units. worker2 # [ 4.209299] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 4.211214] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.212902] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 4.215589] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.218247] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.221082] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 4.230239] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.258619] initrd-nixos-activation-start[171]: booting system configuration /nix/store/qvl0xxkr124pdkzdvanphvcpy7197xra-nixos-system-lb-test worker2 # [ 4.231146] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 4.234438] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.238593] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.243563] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 4.245334] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 4.246359] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 4.247250] systemd[1]: Stopped target Local File Systems. worker2 # [ 4.252477] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 4.256284] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 4.257329] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 4.259822] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 4.261436] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.262453] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.263197] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.291917] initrd-nixos-activation-start[171]: running activation script... worker2 # [ 4.264303] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 4.265196] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.266173] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.266992] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 4.267898] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 4.272245] systemd[1]: systemd-udevd.service: Consumed 1.087s CPU time over 2.295s wall clock time, 17.9M memory peak. worker2 # [ 4.276285] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 4.347478] (udev-worker)[90]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.280306] systemd[1]: Closed udev Control Socket. worker2 # [ 4.281064] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.281966] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.284178] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 4.354988] (udev-worker)[90]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.285108] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.288168] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.292193] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.293183] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.310638] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 4.312576] systemd[1]: Finished Cleanup udev Database. worker2 # [ 4.314717] systemd[1]: Reached target Switch Root. client # [ 4.387191] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.315829] systemd[1]: Starting NixOS Activation... client # [ 4.389098] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.345360] (udev-worker)[90]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.392210] systemd[1]: Stopping Virtual Console Setup... client # [ 4.393013] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.350141] (udev-worker)[90]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.409042] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.411326] systemd[1]: Stopped Virtual Console Setup. client # [ 4.412590] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.387817] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.390365] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.391442] systemd[1]: Stopping Virtual Console Setup... client # [ 4.434063] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.437532] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.396383] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.417623] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.420306] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.398614] initrd-nixos-activation-start[168]: booting system configuration /nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test worker2 # [ 4.430540] initrd-nixos-activation-start[168]: running activation script... lb # [ 4.519999] initrd-nixos-activation-start[194]: setting up /etc... lb # [ 4.637949] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 4.640866] systemd[1]: Finished NixOS Activation. lb # [ 4.642078] systemd[1]: Starting Switch Root... lb # [ 4.663237] systemd[1]: Switching root. worker2 # [ 4.651012] initrd-nixos-activation-start[191]: setting up /etc... worker1 # [ 4.768411] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.765351] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 4.768125] systemd[1]: Finished NixOS Activation. worker2 # [ 4.769597] systemd[1]: Starting Switch Root... client # [ 4.849475] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.792155] systemd[1]: Switching root. worker1 # [ 4.834321] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.837366] systemd[1]: Reached target Initrd File Systems. lb # [ 4.849855] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 4.843691] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.852763] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.905363] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.908203] systemd[1]: Reached target Initrd File Systems. client # [ 4.910396] systemd[1]: Starting Find NixOS closure... client # [ 4.917698] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.889844] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.893635] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.907612] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.910644] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.916373] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.964689] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.970630] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.979039] systemd[1]: Finished Find NixOS closure. client # [ 4.981884] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.940491] systemd[1]: Stopped target Initrd Default Target. client # [ 4.983484] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.942225] systemd[1]: Stopped target Basic System. worker1 # [ 4.944851] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.948512] systemd[1]: Stopped target Path Units. worker1 # [ 4.949569] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.956223] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.957813] systemd[1]: Stopped target Slice Units. worker1 # [ 4.959864] systemd[1]: Stopped target Socket Units. worker1 # [ 4.962080] systemd[1]: Stopped target System Initialization. worker1 # [ 4.964530] systemd[1]: Stopped target Swaps. worker1 # [ 4.967361] systemd[1]: Stopped target Timer Units. client # [ 5.013831] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.972603] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.015687] systemd[1]: Stopped target Basic System. worker1 # [ 4.973651] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.017166] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.974690] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.975952] systemd[1]: Stopped Find NixOS closure. client # [ 5.020373] systemd[1]: Stopped target Path Units. client # [ 5.021696] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.025218] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.984916] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.028194] systemd[1]: Stopped target Slice Units. client # [ 5.032115] systemd[1]: Stopped target Socket Units. worker1 # [ 4.987989] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.036706] systemd[1]: Stopped target System Initialization. client # [ 5.038521] systemd[1]: Stopped target Swaps. worker1 # [ 4.996814] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.040209] systemd[1]: Stopped target Timer Units. worker1 # [ 4.997703] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.042259] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.000383] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.988901] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.004299] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.048842] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.007900] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.052355] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.009732] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.053523] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.012271] systemd[1]: Stopped Create System Files and Directories. client # [ 5.055763] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.014401] systemd[1]: Stopped target Local File Systems. client # [ 5.057580] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.058556] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.016395] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.059335] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.061878] systemd[1]: Stopped Load Kernel Modules. client # [ 5.062647] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.019581] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 5.020958] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.063770] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.021754] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.022773] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.023774] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.024794] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.025729] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.026633] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.027547] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.072490] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.032292] systemd[1]: systemd-udevd.service: Consumed 1.423s CPU time over 3.115s wall clock time, 22M memory peak. client # [ 5.076211] systemd[1]: Stopped Create System Files and Directories. client # [ 5.078164] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.036350] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.080514] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.040390] systemd[1]: Closed udev Control Socket. worker1 # [ 5.041307] systemd[1]: Starting Cleanup udev Database... client # [ 5.084170] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.085249] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.042138] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.086030] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.044158] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.088290] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.090954] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.048274] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.092101] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.049434] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.093063] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.050415] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.093968] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.052187] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.094872] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.056386] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.057436] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.100480] systemd[1]: systemd-udevd.service: Consumed 1.378s CPU time over 3.092s wall clock time, 22.2M memory peak. client # [ 5.102058] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.108163] systemd[1]: Closed udev Control Socket. client # [ 5.109005] systemd[1]: Starting Cleanup udev Database... client # [ 5.109833] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.110934] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.116208] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.117426] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.118427] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.119411] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.079199] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.124451] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.125488] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.082550] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.083348] systemd[1]: Reached target Switch Root. worker1 # [ 5.084899] systemd[1]: Starting NixOS Activation... client # [ 5.152456] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.154303] systemd[1]: Finished Cleanup udev Database. client # [ 5.156586] systemd[1]: Reached target Switch Root. client # [ 5.160548] systemd[1]: Starting NixOS Activation... worker1 # [ 5.168461] initrd-nixos-activation-start[177]: booting system configuration /nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test client # [ 5.242061] initrd-nixos-activation-start[176]: booting system configuration /nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test worker1 # [ 5.200670] initrd-nixos-activation-start[177]: running activation script... client # [ 5.273971] initrd-nixos-activation-start[176]: running activation script... lb # [ 5.373028] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.381557] systemd[1]: Detected virtualization qemu. lb # [ 5.382849] systemd[1]: Detected architecture arm64. lb # [ 5.384260] systemd[1]: Detected first boot. lb # [ 5.387410] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.437332] initrd-nixos-activation-start[200]: setting up /etc... client # [ 5.507700] initrd-nixos-activation-start[199]: setting up /etc... worker2 # [ 5.522697] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 5.535529] systemd[1]: Detected virtualization qemu. worker2 # [ 5.538667] systemd[1]: Detected architecture arm64. worker2 # [ 5.542424] systemd[1]: Detected first boot. worker1 # [ 5.557060] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.559825] systemd[1]: Finished NixOS Activation. worker2 # [ 5.548273] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.564394] systemd[1]: Starting Switch Root... worker1 # [ 5.583362] systemd[1]: Switching root. client # [ 5.626555] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.629454] systemd[1]: Finished NixOS Activation. client # [ 5.630701] systemd[1]: Starting Switch Root... client # [ 5.651656] systemd[1]: Switching root. lb # [ 5.699218] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 5.764427] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.837033] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.864859] systemd[1]: Applying preset policy. worker2 # [ 5.875108] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.071425] systemd[1]: Applying preset policy. lb # [ 6.106684] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.259383] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.272200] systemd[1]: Detected virtualization qemu. worker1 # [ 6.275358] systemd[1]: Detected architecture arm64. worker1 # [ 6.279126] systemd[1]: Detected first boot. worker1 # [ 6.285005] systemd[1]: Initializing machine ID from random generator. client # [ 6.352563] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.359570] systemd[1]: Detected virtualization qemu. client # [ 6.361635] systemd[1]: Detected architecture arm64. client # [ 6.364555] systemd[1]: Detected first boot. client # [ 6.368288] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.302944] systemd[1]: Populated /etc with preset unit settings. lb # [ 6.341479] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.343274] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.346397] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.349724] systemd[1]: Created slice Slice /system/getty. lb # [ 6.351852] systemd[1]: Created slice User and Session Slice. lb # [ 6.353104] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.354927] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.356962] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.359275] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 6.360313] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.361867] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.363431] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.365706] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 6.366838] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.368483] systemd[1]: Reached target Path Units. lb # [ 6.369871] systemd[1]: Reached target Remote File Systems. lb # [ 6.371501] systemd[1]: Reached target Slice Units. lb # [ 6.373646] systemd[1]: Reached target Swaps. lb # [ 6.376094] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.379098] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.381360] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.383902] systemd[1]: Listening on Factory Reset Management. lb # [ 6.385134] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.389512] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.391877] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.396132] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.399134] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 6.401450] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.404032] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 6.409751] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.411163] systemd[1]: Listening on udev Control Socket. lb # [ 6.412816] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.416519] systemd[1]: Mounting Huge Pages File System... lb # [ 6.423843] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 6.427558] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.440537] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.454654] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.456953] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 6.469477] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.473029] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 6.483748] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.485703] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.499439] systemd[1]: Mounting FUSE Control File System... lb # [ 6.499866] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.531441] systemd[1]: Starting Journal Service... lb # [ 6.547327] systemd[1]: Starting Load Kernel Modules... worker2 # [ 6.529128] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 6.530448] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 6.533783] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 6.538114] systemd[1]: Created slice Slice /system/getty. worker2 # [ 6.540234] systemd[1]: Created slice User and Session Slice. worker2 # [ 6.541507] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 6.543464] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 6.545925] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 6.546995] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 6.548490] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 6.550071] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 6.551623] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 6.553928] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 6.577708] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.555138] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 6.557593] systemd[1]: Reached target Path Units. worker2 # [ 6.558536] systemd[1]: Reached target Remote File Systems. worker2 # [ 6.560112] systemd[1]: Reached target Slice Units. worker2 # [ 6.562306] systemd[1]: Reached target Swaps. worker2 # [ 6.564733] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.591814] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 6.567858] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 6.570095] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 6.572500] systemd[1]: Listening on Factory Reset Management. worker2 # [ 6.573729] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 6.577949] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 6.580253] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 6.583016] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.607255] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 6.584602] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 6.607678] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.586268] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.589468] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 6.595341] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 6.596749] systemd[1]: Listening on udev Control Socket. worker2 # [ 6.598366] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.625272] systemd[1]: Starting Coldplug All udev Devices... client # [ 6.668401] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.602064] systemd[1]: Mounting Huge Pages File System... lb # [ 6.629311] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 6.607195] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 6.639637] systemd-journald[265]: Collecting audit messages is enabled. lb # [ 6.641060] systemd[1]: Mounted Huge Pages File System. lb # [ 6.643739] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 6.618377] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.647302] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.651748] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 6.629465] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.657112] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.662692] systemd[1]: Mounted Kernel Configuration File System. lb # [ 6.650773] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.668479] systemd[1]: Started Journal Service. worker2 # [ 6.647498] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 6.647893] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 6.661299] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 6.661713] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 6.661983] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 6.662258] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 6.686589] systemd[1]: Mounting FUSE Control File System... worker2 # [ 6.687088] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 6.712086] systemd[1]: Starting Journal Service... worker2 # [ 6.739055] systemd[1]: Starting Load Kernel Modules... lb # [ 6.763171] EXT4-fs (vda): re-mounted ecc6f207-dc8e-431d-b120-fb330412fd42. worker2 # [ 6.753006] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 6.778784] systemd[1]: Applying preset policy. worker2 # [ 6.771355] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 6.773144] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.784100] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 6.789742] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 6.795492] systemd[1]: Mounted Huge Pages File System. worker2 # [ 6.798630] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 6.801677] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 6.804781] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 6.807941] systemd[1]: Mounted Kernel Configuration File System. client # [ 6.879100] systemd[1]: Applying preset policy. worker2 # [ 6.808415] systemd[1]: Mounted FUSE Control File System. worker2 # [ 6.827631] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 6.834358] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 6.914619] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.918601] systemd-journald[262]: Collecting audit messages is enabled. worker2 # [ 6.931724] systemd[1]: Starting Firewall... worker2 # [ 6.938992] EXT4-fs (vda): re-mounted 7967d1d0-0434-48a8-b218-64dc68f647f4. worker2 # [ 6.948185] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 6.955860] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.956491] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 6.956814] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.969077] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.975021] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.989209] systemd[1]: Started Journal Service. worker2 # [ 6.982231] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.983474] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 6.994564] systemd-modules-load[263]: Module 'atkbd' is built in worker2 # [ 7.008123] systemd-modules-load[263]: Module 'loop' is built in worker1 # [ 7.053251] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.025015] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.139360] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.104847] systemd-oomd[265]: No swap; memory pressure usage will be degraded worker2 # [ 7.139439] systemd-journald[262]: Received client request to flush runtime journal. worker2 # [ 7.193579] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 7.194890] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.204669] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 7.205610] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.206416] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.207357] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.212483] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 6.656438] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 7.296777] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.296446] systemd-journald[265]: Received client request to flush runtime journal. worker1 # [ 7.298128] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.303175] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.307475] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.309618] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.312066] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.314531] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.316745] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.318738] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.320672] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.322738] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.323784] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.325366] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.327091] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.329616] systemd[1]: Reached target Path Units. worker1 # [ 7.330744] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.333141] systemd[1]: Reached target Slice Units. worker1 # [ 7.334176] systemd[1]: Reached target Swaps. worker1 # [ 7.337904] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.341140] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.385728] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.343645] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.387174] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.346190] systemd[1]: Listening on Factory Reset Management. client # [ 7.390452] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.347547] systemd[1]: Listening on Hostname Service Socket. client # [ 7.393414] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.352131] systemd[1]: Starting Journal Log Access Socket... client # [ 7.395762] systemd[1]: Created slice User and Session Slice. client # [ 7.397047] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.354312] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.358360] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.399180] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.399573] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.360047] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.399950] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.400285] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.400548] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.361793] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.400790] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.401028] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.401337] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.363915] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.401695] systemd[1]: Reached target Path Units. client # [ 7.401958] systemd[1]: Reached target Remote File Systems. client # [ 7.402186] systemd[1]: Reached target Slice Units. client # [ 7.402549] systemd[1]: Reached target Swaps. client # [ 7.412793] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.370223] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.371813] systemd[1]: Listening on udev Control Socket. client # [ 7.416146] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.373494] systemd[1]: Listening on udev Varlink Socket. client # [ 7.418562] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.421099] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.377542] systemd[1]: Mounting Huge Pages File System... client # [ 7.422374] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.335769] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 7.337530] systemd[1]: Reached target Preparation for Local File Systems. client # [ 7.427566] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.386787] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.430269] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.344447] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.372991] systemd-modules-load[266]: Module 'atkbd' is built in client # [ 7.434608] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.437277] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.394671] systemd[1]: Mounting Kernel Debug File System... client # [ 7.439698] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.442298] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.386757] systemd-modules-load[266]: Module 'loop' is built in client # [ 7.448355] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 7.449786] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.406566] systemd[1]: Mounting Kernel Trace File System... client # [ 7.451557] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.396333] systemd[1]: Finished Load Kernel Modules. client # [ 7.456083] systemd[1]: Mounting Huge Pages File System... lb # [ 7.399104] systemd[1]: Starting Firewall... lb # [ 7.401967] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.403101] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.420151] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.463624] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.421489] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.414086] systemd[1]: Mounted FUSE Control File System. lb # [ 7.415288] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.474786] systemd[1]: Mounting Kernel Debug File System... client # [ 7.483838] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.426901] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.439639] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.440097] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.440458] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.440897] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.433090] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.498055] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 7.440847] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.500319] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.448577] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.456128] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.457962] systemd-oomd[267]: No swap; memory pressure usage will be degraded lb # [ 7.459481] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 7.519864] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.475862] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.476428] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.522253] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.472558] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.473566] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.530973] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.474476] systemd[1]: Reached target First Boot Complete. lb # [ 7.475307] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 7.533761] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.498660] systemd[1]: Starting Journal Service... lb # [ 7.488227] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 7.489217] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.490186] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.491181] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 7.512293] systemd[1]: Starting Load Kernel Modules... client # [ 7.557390] systemd[1]: Mounting FUSE Control File System... lb # [ 7.500272] systemd[1]: Mounting /run/wrappers... lb # [ 7.501008] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.557858] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.526955] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.545086] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.546606] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.594916] systemd[1]: Starting Journal Service... worker2 # [ 7.515135] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.560659] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.610752] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.567159] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.570798] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.573875] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.577973] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.582842] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.585242] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.585718] systemd[1]: Mounted FUSE Control File System. lb # [ 7.576788] systemd[1]: Mounted /run/wrappers. lb # [ 7.577659] systemd[1]: Reached target Local File Systems. client # [ 7.639869] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.607865] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.595728] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 7.658305] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.615263] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.660841] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.603169] systemd[1]: Starting register-nix-paths.service... lb # [ 7.610643] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.611608] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 7.671442] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.620902] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 7.596831] systemd[1]: Mounted /run/wrappers. worker2 # [ 7.597662] systemd[1]: Reached target Local File Systems. worker2 # [ 7.598480] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 7.685945] systemd-journald[269]: Collecting audit messages is enabled. client # [ 7.692784] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.696565] systemd[1]: Mounted Huge Pages File System. client # [ 7.698215] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.642389] systemd[1]: Starting Create System Files and Directories... client # [ 7.690365] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.623996] systemd[1]: Starting register-nix-paths.service... client # [ 7.697162] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.656755] systemd-udevd[291]: Using default interface naming scheme 'v261'. client # [ 7.702855] systemd-modules-load[270]: Module 'atkbd' is built in client # [ 7.720231] systemd[1]: Started Journal Service. worker2 # [ 7.635428] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 7.709087] systemd-modules-load[270]: Module 'loop' is built in worker2 # [ 7.636608] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 7.712271] systemd[1]: Mounted Kernel Debug File System. client # [ 7.715688] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.644898] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 7.718631] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.719584] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.691421] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.653222] systemd[1]: Starting Create System Files and Directories... worker1 # [ 7.699511] systemd[1]: Starting Firewall... client # [ 7.730764] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.711035] systemd[1]: Starting Apply Kernel Variables... client # [ 7.748278] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.723642] systemd-journald[270]: Collecting audit messages is enabled. worker1 # [ 7.726332] EXT4-fs (vda): re-mounted 60eb5a0c-8c20-428a-b205-a289251928cb. client # [ 7.769664] EXT4-fs (vda): re-mounted 46f38149-c1f1-4a74-96ca-737ecde35846. client # [ 7.769948] systemd[1]: Starting Apply Kernel Variables... client # [ 7.772619] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.783251] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.751422] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.784781] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.752109] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.752473] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.769164] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.798412] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.799458] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.771376] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.810975] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.746350] systemd-udevd[301]: Using default interface naming scheme 'v261'. client # [ 7.820868] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.798694] systemd[1]: Started Journal Service. client # [ 7.828494] systemd-oomd[272]: No swap; memory pressure usage will be degraded lb # [ 7.785915] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 7.757971] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 7.772441] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 7.805079] systemd[1]: Queued start job for default target Multi-User System. lb # [ 7.805194] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 7.806460] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 7.818052] systemd-modules-load[271]: Module 'atkbd' is built in client # [ 7.862852] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.832770] systemd-modules-load[271]: Module 'loop' is built in worker1 # [ 7.841324] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.934339] systemd-journald[269]: Received client request to flush runtime journal. lb # [ 7.889439] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 7.907010] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.916742] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.918173] systemd-oomd[272]: No swap; memory pressure usage will be degraded client # [ 7.976909] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.977964] systemd[1]: Finished Apply Kernel Variables. client # [ 7.978756] systemd[1]: Reached target First Boot Complete. lb # [ 7.935495] systemd[1]: Finished Create System Files and Directories. worker2 # [ 7.914054] systemd[1]: Finished Create System Files and Directories. worker1 # [ 7.944354] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 7.988939] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 7.948727] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 7.952800] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.955678] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.923954] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 7.964401] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 7.966212] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 7.940481] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 7.992569] systemd-journald[270]: Received client request to flush runtime journal. worker2 # [ 8.028848] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.062752] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.082982] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.085032] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.089069] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.093753] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.147424] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.194736] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.201053] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.173708] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 8.196971] systemd[1]: Starting Update is Completed... worker2 # [ 8.189618] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.194291] systemd[1]: Starting Update is Completed... worker1 # [ 8.225682] systemd-udevd[313]: Using default interface naming scheme 'v261'. worker1 # [ 8.285612] systemd[1]: Mounting /run/wrappers... lb # [ 8.298521] systemd[1]: Finished Update is Completed. worker2 # [ 8.325778] systemd[1]: Finished Update is Completed. worker1 # [ 8.370625] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.371441] systemd[1]: Reached target Local File Systems. worker1 # [ 8.382692] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.383701] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.391416] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.401168] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.402454] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.421654] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.428237] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.540886] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.548376] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.622848] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.628878] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.631114] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.635005] systemd[1]: Mounting /run/wrappers... client # [ 8.644108] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.680436] systemd[1]: Mounted /run/wrappers. client # [ 8.681650] systemd[1]: Reached target Local File Systems. client # [ 8.685477] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.690627] systemd[1]: Starting register-nix-paths.service... client # [ 8.694299] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.696522] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.702285] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.666326] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.680361] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.727585] systemd-udevd[300]: Using default interface naming scheme 'v261'. worker1 # [ 8.688551] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.739768] systemd[1]: Starting Create System Files and Directories... lb # [ 8.701876] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.832275] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.836393] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.806878] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.890675] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.906106] systemd[1]: Finished Create System Files and Directories. client # [ 8.915916] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.919454] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.851101] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 8.855811] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 8.889682] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.894334] systemd[1]: Starting Update is Completed... client # [ 9.028402] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.995659] systemd[1]: Finished Update is Completed. client # [ 9.066716] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.072850] systemd[1]: Starting Update is Completed... lb # [ 9.078035] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.082796] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.144612] systemd[1]: Finished Update is Completed. lb # [ 9.125483] systemd[1]: Finished register-nix-paths.service. lb # [ 9.129116] systemd[1]: Reached target System Initialization. lb # [ 9.130041] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.131045] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.131930] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 9.139452] systemd[1]: Reached target Timer Units. lb # [ 9.141539] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 9.146734] systemd[1]: Listening on niks3 server socket. lb # [ 9.147659] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.152269] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.153493] systemd[1]: Reached target Socket Units. lb # [ 9.154250] systemd[1]: Reached target Basic System. lb # [ 9.154980] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 9.167531] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.179398] systemd[1]: Starting Post-Boot Actions... lb # [ 9.207200] systemd[1]: Started Reset console on configuration changes. worker2 # [ 9.205510] systemd[1]: Finished Firewall. lb # [ 9.236258] systemd[1]: Starting resolvconf update... lb # [ 9.259635] systemd[1]: Started rustfs.service. worker2 # [ 9.265402] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 9.311864] systemd[1]: Starting rustfs-bucket.service... client # [ 9.358959] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.369714] nsncd[446]: Sep 22 16:58:33.984 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.387554] systemd[1]: Finished Post-Boot Actions. lb # [ 9.390078] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 9.439694] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.427858] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 9.432119] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.436697] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.436355] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.468491] systemd[1]: Starting D-Bus System Message Bus... lb # [ 9.502816] systemd[1]: Starting User Login Management... lb # [ 9.573434] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 9.650034] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.658040] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.661601] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.718524] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 9.688696] systemd[1]: Finished Firewall. worker2 # [ 9.715908] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 9.737616] systemd[1]: Finished register-nix-paths.service. worker2 # [ 9.740230] systemd[1]: Reached target System Initialization. worker2 # [ 9.741131] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 9.744536] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 9.745742] systemd[1]: Reached target Timer Units. worker2 # [ 9.746484] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 9.747415] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 9.757693] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 9.763616] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 9.783590] systemd[1]: Reached target Socket Units. worker2 # [ 9.789489] systemd[1]: Reached target Basic System. worker2 # [ 9.797466] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 9.801394] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.835830] dbus-broker-launch[479]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 9.809999] systemd[1]: Starting Post-Boot Actions... worker2 # [ 9.815604] systemd[1]: Started Reset console on configuration changes. worker2 # [ 9.824976] systemd[1]: Starting resolvconf update... lb # [ 9.857893] dbus-broker-launch[479]: NSS returned no entry for 'systemd-timesync' lb # [ 9.859094] dbus-broker-launch[479]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 9.835116] systemd[1]: Starting D-Bus System Message Bus... client # [ 9.937875] (udev-worker)[353]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 9.890203] systemd-logind[481]: New seat seat0. lb # [ 9.901665] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.902676] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.903611] systemd[1]: Stopped target User and Group Name Lookups. client # [ 9.948527] (udev-worker)[349]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.950507] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.914033] systemd[1]: Stopping User and Group Name Lookups... lb # [ 9.914901] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 9.915835] systemd[1]: Started User Login Management. lb # [ 9.925754] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.926655] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 9.916537] systemd[1]: Finished Post-Boot Actions. lb # [ 9.956310] systemd[1]: Starting linger-users.service... worker2 # [ 9.935297] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 9.945110] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.974385] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.986241] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 9.957793] nsncd[483]: Sep 22 16:58:34.561 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 9.969053] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 9.977434] systemd[1]: Starting User Login Management... lb # [ 10.027077] dbus-broker-launch[479]: Ready worker2 # [ 9.998999] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.011193] dbus-broker-launch[487]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.022460] dbus-broker-launch[487]: NSS returned no entry for 'systemd-timesync' lb # [ 10.065221] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 10.027253] dbus-broker-launch[487]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.043190] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.089533] dbus-broker-launch[487]: Ready lb # [ 10.122246] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.127358] systemd[1]: Finished linger-users.service. lb # [ 10.148918] nsncd[553]: Sep 22 16:58:34.762 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.150997] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.151954] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.163177] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.211757] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 10.217986] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 10.222405] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 10.223392] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 10.186819] systemd[1]: Finished resolvconf update. worker2 # [ 10.158562] systemd-logind[496]: New seat seat0. worker1 # [ 10.189731] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.232418] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 10.234659] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 10.163488] systemd[1]: Started User Login Management. worker2 # [ 10.171161] systemd[1]: Starting linger-users.service... client # [ 10.243334] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 10.200753] systemd[1]: Starting DHCP Client... client # [ 10.248475] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.251819] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.193321] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.196489] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 10.211376] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.308430] mousedev: PS/2 mouse device common for all mice worker2 # [ 10.221420] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.230722] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.234164] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.239281] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.243133] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.273500] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.275192] systemd[1]: Reached target System Initialization. worker1 # [ 10.276945] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.279302] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.282691] systemd[1]: Reached target Timer Units. worker1 # [ 10.288789] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.296400] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.272761] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.277214] systemd[1]: Finished linger-users.service. worker1 # [ 10.307153] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.320783] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.328370] systemd[1]: Reached target Socket Units. worker1 # [ 10.332384] systemd[1]: Reached target Basic System. worker1 # [ 10.342299] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.317021] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.349179] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.320159] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.325586] nsncd[552]: Sep 22 16:58:34.943 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.359098] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.333732] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.372575] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.382359] systemd[1]: Starting resolvconf update... worker2 # [ 10.357393] systemd[1]: Finished resolvconf update. worker2 # [ 10.359682] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.389787] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.368466] systemd[1]: Starting DHCP Client... worker1 # [ 10.399884] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 10.371644] systemd[1]: Starting Extra networking commands.... worker1 # [ 10.415515] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 10.446520] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.481179] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.489450] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.497981] nsncd[488]: Sep 22 16:58:35.092 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.513735] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.518274] systemd[1]: Starting User Login Management... worker1 # [ 10.544851] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.555567] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 10.559590] dhcpcd[604]: dhcpcd-10.3.2 starting worker1 # [ 10.566325] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.571818] dbus-broker-launch[492]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 10.575375] dhcpcd[617]: dev: loaded udev lb # [ 10.580366] dhcpcd[617]: no valid interfaces found lb # [ 10.581292] dhcpcd[617]: no valid interfaces found worker1 # [ 10.591282] systemd[1]: Started D-Bus System Message Bus. lb # [ 10.635279] systemd[1]: Finished Firewall. worker1 # [ 10.638946] dbus-broker-launch[492]: Ready lb # [ 10.641671] systemd[1]: Reached target Preparation for Network. worker2 # [ 10.623129] dhcpcd[585]: dhcpcd-10.3.2 starting lb # [ 10.652277] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.636558] dhcpcd[628]: dev: loaded udev worker2 # [ 10.642663] dhcpcd[628]: no valid interfaces found worker2 # [ 10.645909] dhcpcd[628]: no valid interfaces found worker2 # [ 10.667591] systemd[1]: Finished Extra networking commands.. worker1 # [ 10.702390] systemd-logind[500]: New seat seat0. worker2 # [ 10.672941] systemd[1]: Reached target Network. worker2 # [ 10.676456] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.708403] systemd[1]: Started User Login Management. worker1 # [ 10.714379] systemd[1]: Starting linger-users.service... worker1 # [ 10.733056] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.734061] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.778688] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 10.739486] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.783793] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 10.744147] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.788342] systemd[1]: Reached target System Initialization. client # [ 10.789267] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.790249] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.791206] systemd[1]: Reached target Timer Units. client # [ 10.791944] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.723397] systemd[1]: Finished Permit User Sessions. worker1 # [ 10.753017] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.726927] systemd[1]: Started Getty on tty1. client # [ 10.801858] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.762156] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.805000] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.808890] systemd[1]: Reached target Socket Units. worker2 # [ 10.736360] systemd[1]: Reached target Login Prompts. client # [ 10.812907] systemd[1]: Reached target Basic System. worker1 # [ 10.771010] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 10.814351] systemd[1]: Started backdoor.service. client # [ 10.816350] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.775359] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.823659] systemd[1]: Starting nftables firewall... client # [ 10.851059] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.813762] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.819910] systemd[1]: Finished linger-users.service. client # [ 10.878552] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.857964] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.903602] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.863169] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.867566] nsncd[557]: Sep 22 16:58:35.477 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.874232] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.886581] systemd[1]: Finished resolvconf update. client # [ 10.930275] systemd[1]: Starting resolvconf update... worker1 # [ 10.890651] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.896904] systemd[1]: Starting DHCP Client... worker1 # [ 10.901663] systemd[1]: Starting Extra networking commands.... client # connecting to host... worker2 # [ 10.926586] systemd[1]: Starting Virtual Console Setup... client # [ 11.020902] nsncd[452]: Sep 22 16:58:35.636 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.984410] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 11.036349] systemd[1]: Starting D-Bus System Message Bus... client # [ 11.037260] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.970252] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 10.976381] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 10.980768] systemd[1]: Starting Virtual Console Setup... client: Guest shell says: b'Spawning backdoor root shell...\n' lb # [ 11.022982] systemd[1]: Finished Extra networking commands.. lb # [ 11.029421] systemd[1]: Reached target Network. lb # [ 11.034306] systemd[1]: Starting PostgreSQL Server... worker2 # [ 11.010002] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 11.050083] systemd[1]: Starting Permit User Sessions... client # [ 11.098792] systemd[1]: Finished Post-Boot Actions. client # [ 11.106597] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.110787] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.076268] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 11.138006] systemd[1]: Starting User Login Management... client # [ 11.146153] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 11.092504] systemd[1]: Started backdoor.service. client: connected to guest root shell client: (connecting took 11.52 seconds) client: (finished: waiting for the VM to finish booting, in 11.52 seconds) lb # [ 11.152213] systemd[1]: Finished Permit User Sessions. lb # [ 11.167363] systemd[1]: Started Getty on tty1. lb # [ 11.170586] systemd[1]: Reached target Login Prompts. worker1 # [ 11.184947] dhcpcd[590]: dhcpcd-10.3.2 starting worker1 # connecting to host... worker1 # [ 11.201555] dhcpcd[631]: dev: loaded udev lb # [ 11.221619] systemd[1]: Starting Virtual Console Setup... worker1 # [ 11.243550] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.217135] systemd-vconsole-setup[654]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 11.225254] systemd[1]: Finished Virtual Console Setup. client # [ 11.305911] dbus-broker-launch[463]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 11.275832] (udev-worker)[347]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 11.323096] dbus-broker-launch[463]: NSS returned no entry for 'systemd-timesync' client # [ 11.328378] dbus-broker-launch[463]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 11.292807] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 11.294582] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.271125] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 11.300766] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 11.312630] (udev-worker)[348]: Network interface NamePolicy= disabled on kernel command line. lb # [ 11.312346] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 11.319136] systemd[1]: Starting Virtual Console Setup... worker2 # [ 11.308436] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 11.313184] systemd[1]: Started backdoor.service. worker1 # [ 11.360092] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.349205] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.350144] systemd[1]: Reached target Network. client # [ 11.397298] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.357706] systemd[1]: Starting Permit User Sessions... client # [ 11.406192] systemd-logind[482]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 11.417233] systemd-logind[482]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.418696] systemd-logind[482]: New seat seat0. worker1 # [ 11.399873] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.400389] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.432683] systemd[1]: Started User Login Management. worker1 # [ 11.403535] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.403876] cfg80211: failed to load regulatory.db client # [ 11.437049] systemd[1]: Starting linger-users.service... worker2 # [ 11.410174] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.424998] systemd[1]: Finished Permit User Sessions. client # [ 11.469251] dbus-broker-launch[463]: Ready worker1 # [ 11.430199] systemd[1]: Started Getty on tty1. worker1 # [ 11.430974] systemd[1]: Reached target Login Prompts. lb # [ 11.446196] postgresql-pre-start[695]: The files belonging to this database system will be owned by user "postgres". worker1 # [ 11.447424] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 11.447765] postgresql-pre-start[695]: This user must also own the server process. worker1 # [ 11.456296] systemd[1]: Starting Address configuration of eth1... lb # [ 11.465799] postgresql-pre-start[695]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 11.467219] postgresql-pre-start[695]: The default database encoding has accordingly been set to "UTF8". client # [ 11.512112] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # connecting to host... lb # [ 11.472628] postgresql-pre-start[695]: The default text search configuration will be set to "english". lb # [ 11.473879] postgresql-pre-start[695]: Data page checksums are enabled. lb # [ 11.474777] postgresql-pre-start[695]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok client # [ 11.521808] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.522800] systemd[1]: Stopped target User and Group Name Lookups. client # [ 11.523691] systemd[1]: Stopping User and Group Name Lookups... lb # [ 11.482736] postgresql-pre-start[695]: creating subdirectories ... ok worker2 # [ 11.453360] (udev-worker)[646]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 11.483740] postgresql-pre-start[695]: selecting dynamic shared memory implementation ... posix client # [ 11.540403] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.541411] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.542205] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 11.513817] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.504431] dhcpcd[631]: eth0: waiting for carrier worker2 # [ 11.477476] (udev-worker)[646]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.510779] dhcpcd[631]: eth0: waiting for carrier worker1 # [ 11.511561] dhcpcd[631]: eth0: carrier acquired worker2 # [ 11.496828] (udev-worker)[648]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.528233] dhcpcd[631]: DUID 00:01:00:01:32:45:70:3c:52:54:00:12:34:56 worker1 # [ 11.529239] dhcpcd[631]: eth0: IAID 00:12:34:56 worker1 # [ 11.529877] dhcpcd[631]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.576607] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.522634] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.581700] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.586797] systemd[1]: Finished linger-users.service. worker1 # [ 11.568178] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.570543] network-addresses-eth1-start[661]: adding address 192.168.1.3/24... done worker2 # [ 11.567840] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.568414] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.583373] network-addresses-eth1-start[661]: adding address 2001:db8:1::3/64... done worker2 # [ 11.572046] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.572424] cfg80211: failed to load regulatory.db lb # [ 11.597904] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 11.602324] systemd[1]: Finished Address configuration of eth1. client # [ 11.666579] systemd[1]: Finished resolvconf update. client # [ 11.676334] nsncd[549]: Sep 22 16:58:36.294 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.640661] postgresql-pre-start[695]: selecting default "max_connections" ... 100 client # [ 11.685044] systemd[1]: Starting DHCP Client... client # [ 11.685816] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.689892] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.690845] systemd[1]: Reached target User and Group Name Lookups. lb # [ 11.662407] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 11.673744] systemd[1]: Started backdoor.service. worker2 # [ 11.675943] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.681649] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.786597] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.772832] dhcpcd[628]: eth0: waiting for carrier worker2 # [ 11.778461] dhcpcd[628]: eth0: carrier acquired worker2 # [ 11.800504] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.814948] systemd-logind[500]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 11.796878] dhcpcd[628]: DUID 00:01:00:01:32:45:70:3c:52:54:00:12:34:56 lb # [ 11.827480] postgresql-pre-start[695]: selecting default "shared_buffers" ... 128MB worker2 # [ 11.800824] dhcpcd[628]: eth0: IAID 00:12:34:56 worker2 # [ 11.801561] dhcpcd[628]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.805945] network-addresses-eth1-start[673]: adding address 192.168.1.4/24... done lb # connecting to host... lb # [ 11.856411] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.818171] network-addresses-eth1-start[673]: adding address 2001:db8:1::4/64... done lb # [ 11.859920] (udev-worker)[626]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.836574] systemd[1]: Finished Address configuration of eth1. client # [ 11.910247] dhcpcd[577]: dhcpcd-10.3.2 starting lb # [ 11.866745] (udev-worker)[633]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 11.884850] mousedev: PS/2 mouse device common for all mice worker2 # [ 11.846448] systemd-logind[496]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.920488] dhcpcd[596]: dev: loaded udev lb # [ 11.886804] (udev-worker)[633]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.980621] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.975045] mousedev: PS/2 mouse device common for all mice lb # [ 12.002214] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 12.005316] systemd-logind[500]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.042308] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 12.042872] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 12.047551] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 12.047908] cfg80211: failed to load regulatory.db client # [ 12.104103] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 12.027226] systemd-logind[496]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 12.037873] systemd-logind[496]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 12.157875] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 12.158402] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 12.163099] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 12.163441] cfg80211: failed to load regulatory.db client # [ 12.232093] systemd[1]: Finished nftables firewall. client # [ 12.234710] systemd[1]: Reached target Preparation for Network. client # [ 12.244260] systemd[1]: Starting Address configuration of eth1... lb # [ 12.206262] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.252837] systemd[1]: Starting Extra networking commands.... lb # [ 12.216312] systemd[1]: Finished Virtual Console Setup. client # [ 12.277128] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.264214] dhcpcd[596]: eth0: waiting for carrier client # [ 12.266054] dhcpcd[596]: libudev: received NULL device client # [ 12.266989] dhcpcd[596]: libudev: received NULL device client # [ 12.267880] dhcpcd[596]: eth0: carrier acquired lb # [ 12.236860] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 12.243917] systemd[1]: Starting Address configuration of eth1... client # [ 12.291882] dhcpcd[596]: DUID 00:01:00:01:32:45:70:3c:52:54:00:12:34:56 client # [ 12.294733] dhcpcd[596]: eth0: IAID 00:12:34:56 client # [ 12.295460] dhcpcd[596]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.342889] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.386667] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 12.332357] dhcpcd[617]: eth0: waiting for carrier lb # [ 12.337225] dhcpcd[617]: eth0: carrier acquired lb # [ 12.354875] dhcpcd[617]: DUID 00:01:00:01:32:45:70:3c:52:54:00:12:34:56 client # [ 12.398749] network-addresses-eth1-start[620]: adding address 192.168.1.1/24... done lb # [ 12.358571] dhcpcd[617]: eth0: IAID 00:12:34:56 lb # [ 12.359706] systemd-logind[481]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 12.364649] dhcpcd[617]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.392998] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.424249] network-addresses-eth1-start[620]: adding address 2001:db8:1::1/64... done lb # [ 12.405674] network-addresses-eth1-start[722]: adding address 192.168.1.2/24... done client # [ 12.450516] systemd[1]: Finished Address configuration of eth1. lb # [ 12.423580] network-addresses-eth1-start[722]: adding address 2001:db8:1::2/64... done lb # [ 12.453810] systemd[1]: Finished Address configuration of eth1. lb # [ 12.507170] mousedev: PS/2 mouse device common for all mice client # [ 12.540300] systemd[1]: Finished Extra networking commands.. client # [ 12.545616] systemd[1]: Reached target Network. client # [ 12.549154] systemd[1]: Starting Permit User Sessions... client # [ 12.600132] systemd[1]: Finished Permit User Sessions. client # [ 12.612258] systemd[1]: Started Getty on tty1. client # [ 12.621722] systemd[1]: Reached target Login Prompts. client: (finished: waiting for unit network-addresses-eth1.service, in 13.02 seconds) lb # [ 12.638513] systemd-logind[481]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } worker1 # [ 12.680457] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.728583] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.07 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb: (finished: waiting for unit network-addresses-eth1.service, in 0.09 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 13.109272] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.068552] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.079908] systemd[1]: Starting Virtual Console Setup... worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } client # [ 13.139724] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.099653] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 13.143399] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.102622] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.103393] systemd[1]: Starting Virtual Console Setup... client # [ 13.168631] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 13.172441] systemd[1]: Starting Virtual Console Setup... worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit niks3.service client # [ 13.222564] systemd-logind[482]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.195017] systemd-logind[500]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.198985] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.232895] dhcpcd[631]: eth0: soliciting an IPv6 router worker1 # [ 13.234455] dhcpcd[631]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.204461] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.206546] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.235810] dhcpcd[631]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.207299] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 13.237389] dhcpcd[631]: eth0: adding route to fec0::/64 worker1 # [ 13.238440] dhcpcd[631]: eth0: adding default route via fe80::2 worker2 # [ 13.217046] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.232643] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.238876] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.241112] systemd[1]: Starting Virtual Console Setup... client # [ 13.318917] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.322517] systemd[1]: Finished Virtual Console Setup. worker1 # [ 13.281224] systemd-vconsole-setup[702]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.284504] systemd[1]: Finished Virtual Console Setup. worker1 # [ 13.289724] dhcpcd[631]: eth0: soliciting a DHCP lease worker1 # [ 13.296541] dhcpcd[631]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.304259] dhcpcd[631]: eth0: probing address 10.0.2.15/24 worker2 # [ 13.359951] systemd-vconsole-setup[706]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.364254] systemd[1]: Finished Virtual Console Setup. client # [ 13.733750] dhcpcd[596]: eth0: soliciting a DHCP lease lb # [ 13.691630] dhcpcd[617]: eth0: soliciting a DHCP lease lb # [ 13.696581] dhcpcd[617]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 13.741243] dhcpcd[596]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 13.744466] dhcpcd[596]: eth0: probing address 10.0.2.15/24 lb # [ 13.704240] dhcpcd[617]: eth0: probing address 10.0.2.15/24 worker2 # [ 13.726599] dhcpcd[628]: eth0: soliciting a DHCP lease worker2 # [ 13.730117] dhcpcd[628]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 13.736520] dhcpcd[628]: eth0: probing address 10.0.2.15/24 worker2 # [ 13.862563] dhcpcd[628]: eth0: soliciting an IPv6 router worker2 # [ 13.866391] dhcpcd[628]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.869056] dhcpcd[628]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.871867] dhcpcd[628]: eth0: adding route to fec0::/64 worker2 # [ 13.874209] dhcpcd[628]: eth0: adding default route via fe80::2 lb # [ 14.128534] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 14.133690] systemd[1]: Stopped Virtual Console Setup. lb # [ 14.134536] systemd[1]: Stopping Virtual Console Setup... lb # [ 14.138451] systemd[1]: Starting Virtual Console Setup... lb # [ 14.187388] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 14.234642] systemd-logind[481]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 14.412665] dhcpcd[596]: eth0: soliciting an IPv6 router client # [ 14.416182] dhcpcd[596]: eth0: Router Advertisement from fe80::2 client # [ 14.418751] dhcpcd[596]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.421759] dhcpcd[596]: eth0: adding route to fec0::/64 client # [ 14.423946] dhcpcd[596]: eth0: adding default route via fe80::2 lb # [ 14.562967] postgresql-pre-start[695]: selecting default time zone ... UTC lb # [ 14.566482] postgresql-pre-start[695]: creating configuration files ... ok lb # [ 14.593750] systemd-vconsole-setup[773]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.596814] systemd[1]: Finished Virtual Console Setup. lb # [ 14.807250] postgresql-pre-start[695]: running bootstrap script ... ok lb # [ 15.252789] dhcpcd[617]: eth0: soliciting an IPv6 router lb # [ 15.254222] dhcpcd[617]: eth0: Router Advertisement from fe80::2 lb # [ 15.255083] dhcpcd[617]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 15.255986] dhcpcd[617]: eth0: adding route to fec0::/64 lb # [ 15.259541] dhcpcd[617]: eth0: adding default route via fe80::2 lb # [ 15.307174] postgresql-pre-start[695]: performing post-bootstrap initialization ... ok lb # [ 15.508232] postgresql-pre-start[695]: syncing data to disk ... ok lb # [ 15.510294] postgresql-pre-start[695]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.511550] postgresql-pre-start[695]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.513746] postgresql-pre-start[695]: Success. You can now start the database server using: lb # [ 15.514898] postgresql-pre-start[695]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.603921] postgres[801]: [801] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.607091] postgres[801]: [801] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.608390] postgres[801]: [801] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.610067] postgres[801]: [801] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.620915] postgres[810]: [810] LOG: database system was shut down at 2026-09-22 16:58:39 GMT lb # [ 15.627039] postgres[801]: [801] LOG: database system is ready to accept connections lb # [ 15.631791] systemd[1]: Started PostgreSQL Server. lb # [ 15.639330] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.807968] postgresql-setup-start[821]: CREATE DATABASE lb # [ 15.846809] postgresql-setup-start[826]: CREATE ROLE lb # [ 15.863197] postgresql-setup-start[828]: ALTER DATABASE lb # [ 15.868703] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.870714] systemd[1]: Reached target PostgreSQL. worker1 # [ 17.837977] dhcpcd[631]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 17.842098] dhcpcd[631]: eth0: adding route to 10.0.2.0/24 worker1 # [ 17.846599] dhcpcd[631]: eth0: adding default route via 10.0.2.2 worker1 # [ 18.015650] systemd[1]: Started DHCP Client. worker1 # [ 18.017442] systemd[1]: Reached target Multi-User System. worker1 # [ 18.019446] systemd[1]: Startup finished in 953ms (kernel) + 4.873s (initrd) + 12.192s (userspace) = 18.018s. client # [ 18.542764] dhcpcd[596]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.542992] dhcpcd[596]: eth0: adding route to 10.0.2.0/24 client # [ 18.543070] dhcpcd[596]: eth0: adding default route via 10.0.2.2 client # [ 18.653213] systemd[1]: Started DHCP Client. client # [ 18.655492] systemd[1]: Reached target Multi-User System. client # [ 18.657903] systemd[1]: Startup finished in 948ms (kernel) + 4.972s (initrd) + 12.733s (userspace) = 18.655s. worker2 # [ 18.678720] dhcpcd[628]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 18.681734] dhcpcd[628]: eth0: adding route to 10.0.2.0/24 worker2 # [ 18.684005] dhcpcd[628]: eth0: adding default route via 10.0.2.2 worker2 # [ 18.828410] systemd[1]: Started DHCP Client. worker2 # [ 18.830345] systemd[1]: Reached target Multi-User System. worker2 # [ 18.831630] systemd[1]: Startup finished in 945ms (kernel) + 4.131s (initrd) + 13.754s (userspace) = 18.831s. lb # [ 19.241256] dhcpcd[617]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 19.245665] dhcpcd[617]: eth0: adding route to 10.0.2.0/24 lb # [ 19.247977] dhcpcd[617]: eth0: adding default route via 10.0.2.2 lb # [ 19.389760] systemd[1]: Started DHCP Client. lb # [ 19.392448] systemd[1]: Reached target Network is Online. lb # [ 19.397644] systemd[1]: Started Envoy reverse proxy. lb # [ 19.402271] systemd[1]: Started mock-oidc.service. lb # [ 19.577234] mock-oidc-server[920]: Mock OIDC Server running lb # [ 19.579685] mock-oidc-server[920]: OIDC Address: 192.168.1.2:8080 lb # [ 19.581483] mock-oidc-server[920]: Issue Address: 0.0.0.0:8081 lb # [ 19.582504] mock-oidc-server[920]: Issuer: http://192.168.1.2:8080/oidc lb # [ 19.583909] mock-oidc-server[920]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 19.586002] mock-oidc-server[920]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 19.587562] mock-oidc-server[920]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 19.730844] envoy[919]: [2026-09-22 16:58:44.347][919][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 19.733814] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 19.735268] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 19.737332] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 19.742799] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 19.746225] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 19.750133] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 19.755922] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 19.760961] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 19.764662] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 19.814875] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 19.823309] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 19.829833] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 19.831800] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 19.834547] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 19.843993] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 19.848343] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 19.850279] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 19.851917] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 19.855172] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 19.859840] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 19.864372] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 19.866397] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 19.875522] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 19.877858] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 19.882243] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 19.885152] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 19.888955] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 19.890741] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 19.893676] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.895976] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 19.898110] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 19.902997] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 19.906205] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 19.912412] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 19.914936] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 19.918553] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 19.920439] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 19.922526] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 19.925656] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 19.929264] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 19.938189] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default worker2 # [ 19.909028] systemd[1]: Reached target Network is Online. lb # [ 19.941332] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 19.943155] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata worker2 # [ 19.919210] systemd[1]: Starting scheduler-token.service... lb # [ 19.949333] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 19.954442] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory worker1 # [ 19.956478] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 19.957523] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 19.960436] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 19.964905] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 19.967992] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 19.970537] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 19.972222] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 19.974234] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 19.989243] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 19.996954] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 20.000614] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 20.002305] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 20.004569] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 20.006280] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 20.008420] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 20.014814] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 20.016463] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 20.023828] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 20.029665] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 20.035612] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 20.039050] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header worker2 # [ 20.014548] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 20.046005] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm worker2 # [ 20.023968] systemd[1]: Finished scheduler-token.service. lb # [ 20.053464] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder worker2 # [ 20.025760] systemd[1]: scheduler-token.service: Consumed 64ms CPU time over 98ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 20.055712] envoy[919]: [2026-09-22 16:58:44.349][919][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest worker2 # [ 20.031890] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 20.061570] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 20.063770] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 20.068266] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 20.070825] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 20.072558] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 20.084146] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 20.086003] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 20.087851] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 20.089836] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 20.092858] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 20.094545] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 20.096639] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 20.099513] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 20.102017] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 20.103871] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 20.105919] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 20.109083] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 20.110807] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 20.113258] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 20.115266] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 20.119080] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 20.120949] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default worker1 # [ 20.124379] nix-grpc-daemon[767]: ts=2026-09-22T16:58:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 20.122657] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 20.125257] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 20.127753] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 20.130206] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection worker1 # [ 20.132447] nix-grpc-daemon[767]: ts=2026-09-22T16:58:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 20.132057] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats worker1 # [ 20.134480] nix-grpc-daemon[767]: ts=2026-09-22T16:58:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 20.134497] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 20.137000] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 20.138800] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 20.141234] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 20.146416] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe worker1 # [ 20.147562] nix-grpc-daemon[767]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 20.149221] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default worker1 # [ 20.150619] nix-grpc-daemon[767]: E0000 00:00:1790096324.763877 767 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 20.151109] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 20.152773] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher worker1 # [ 20.155535] nix-grpc-daemon[767]: E0000 00:00:1790096324.772335 767 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 20.155583] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 20.164325] envoy[919]: [2026-09-22 16:58:44.350][919][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 20.168027] envoy[919]: [2026-09-22 16:58:44.372][919][info][main] [source/server/server.cc:531] HTTP header map info: worker1 # [ 20.174561] nix-grpc-daemon[767]: ts=2026-09-22T16:58:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 20.177160] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 20.180553] nix-grpc-daemon[767]: E0000 00:00:1790096324.797065 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 20.169489] envoy[919]: [2026-09-22 16:58:44.372][919][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 20.189723] envoy[919]: [2026-09-22 16:58:44.372][919][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 20.191211] envoy[919]: [2026-09-22 16:58:44.372][919][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id worker1 # [ 20.202469] nix-grpc-daemon[767]: E0000 00:00:1790096324.819269 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 20.204621] envoy[919]: [2026-09-22 16:58:44.372][919][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status worker2 # [ 20.177224] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 20.206357] envoy[919]: [2026-09-22 16:58:44.443][919][info][main] [source/server/server.cc:968] runtime: {} lb # [ 20.207593] envoy[919]: [2026-09-22 16:58:44.445][919][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. worker1 # [ 20.208045] nix-grpc-daemon[767]: E0000 00:00:1790096324.822126 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 20.209511] envoy[919]: [2026-09-22 16:58:44.445][919][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 20.211084] envoy[919]: [2026-09-22 16:58:44.445][919][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 worker2 # [ 20.185298] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 20.215818] envoy[919]: [2026-09-22 16:58:44.446][919][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration worker2 # [ 20.187324] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 20.217649] envoy[919]: [2026-09-22 16:58:44.446][919][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 20.219202] envoy[919]: [2026-09-22 16:58:44.446][919][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 20.221423] envoy[919]: [2026-09-22 16:58:44.549][919][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 20.222904] envoy[919]: [2026-09-22 16:58:44.557][919][info][config] [source/server/configuration_impl.cc:164] loading stats configuration worker1 # [ 20.224058] nix-grpc-daemon[767]: E0000 00:00:1790096324.840054 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 20.224898] envoy[919]: [2026-09-22 16:58:44.562][919][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. worker2 # [ 20.198048] nix-grpc-daemon[774]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 20.229641] envoy[919]: [2026-09-22 16:58:44.562][919][info][main] [source/server/server.cc:1077] starting main dispatch loop worker2 # [ 20.200139] nix-grpc-daemon[774]: E0000 00:00:1790096324.821659 774 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 20.229673] nix-grpc-daemon[767]: ts=2026-09-22T16:58:44Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 20.231104] envoy[919]: [2026-09-22 16:58:44.817][919][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 20.232884] envoy[919]: [2026-09-22 16:58:44.817][919][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 20.234596] envoy[919]: [2026-09-22 16:58:44.817][919][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager worker2 # [ 20.206189] nix-grpc-daemon[774]: E0000 00:00:1790096324.829965 774 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 20.236333] envoy[919]: [2026-09-22 16:58:44.845][919][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 20.223868] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 20.228878] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 20.234142] nix-grpc-daemon[774]: E0000 00:00:1790096324.856939 815 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.251888] nix-grpc-daemon[774]: E0000 00:00:1790096324.875489 815 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.264202] nix-grpc-daemon[774]: E0000 00:00:1790096324.878350 815 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.273606] nix-grpc-daemon[774]: E0000 00:00:1790096324.896867 815 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.283991] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.285769] nix-grpc-daemon[774]: ts=2026-09-22T16:58:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.430337] nix-grpc-daemon[767]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.487405] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.492923] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.694019] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.699269] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.831006] nix-grpc-daemon[767]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.900406] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.905461] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.106678] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.112834] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.316121] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.320008] nix-grpc-daemon[774]: ts=2026-09-22T16:58:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.516999] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.528524] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.632636] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.635242] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.726928] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.732254] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.836277] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.841343] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.933370] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.938560] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.042166] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.047309] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 22.097467] rustfs-bucket-start[954]: mb s3://farm lb # [ 22.110171] systemd[1]: Finished rustfs-bucket.service. lb # [ 22.122468] systemd[1]: Starting niks3 server... worker2 # [ 22.139920] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.145027] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.247990] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.253118] nix-grpc-daemon[767]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 22.270700] postgres[970]: [970] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 22.271971] postgres[970]: [970] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 22.296004] niks3-server[964]: 2026/09/22 16:58:46 OK 20241026095416_initial_model.sql (14.31ms) lb # [ 22.299206] niks3-server[964]: 2026/09/22 16:58:46 OK 20251210153512_drop_unused_gin_index.sql (3.47ms) lb # [ 22.304377] niks3-server[964]: 2026/09/22 16:58:46 OK 20251218171726_add_pins.sql (5.12ms) lb # [ 22.308745] niks3-server[964]: 2026/09/22 16:58:46 OK 20260628120000_add_object_size_and_stats.sql (4.33ms) lb # [ 22.313933] niks3-server[964]: 2026/09/22 16:58:46 OK 20260905000000_add_claims.sql (5.15ms) lb # [ 22.317424] niks3-server[964]: 2026/09/22 16:58:46 OK 20260920000000_drop_claims.sql (3.39ms) lb # [ 22.319335] niks3-server[964]: 2026/09/22 16:58:46 goose: successfully migrated database to version: 20260920000000 lb # [ 22.322899] niks3-server[964]: 2026/09/22 16:58:46 OK 1_commit_pending_closure.sql (5.43ms) lb # [ 22.325507] niks3-server[964]: 2026/09/22 16:58:46 OK 2_object_stats_trigger.sql (2.47ms) lb # [ 22.327405] niks3-server[964]: 2026/09/22 16:58:46 goose: up to current file version: 2 lb # [ 22.329308] niks3-server[964]: 2026/09/22 16:58:46 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 22.349307] niks3-server[964]: 2026/09/22 16:58:46 INFO Read proxy enabled — serving cache objects from S3 lb # [ 22.357524] niks3-server[964]: 2026/09/22 16:58:46 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 22.362042] niks3-server[964]: 2026/09/22 16:58:46 INFO systemd watchdog enabled interval=15s lb # [ 22.363308] systemd[1]: Started niks3 server. lb # [ 22.363962] niks3-server[964]: 2026/09/22 16:58:46 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 22.367413] systemd[1]: Reached target Multi-User System. lb # [ 22.374958] systemd[1]: Startup finished in 952ms (kernel) + 3.987s (initrd) + 17.418s (userspace) = 22.359s. worker2 # [ 22.347750] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.351994] nix-grpc-daemon[774]: ts=2026-09-22T16:58:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 9.30 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 22.453754] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.459039] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.03 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.03 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 22.520817] envoy[919]: conn peer=[::1]:47308 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=2 lb: (finished: waiting for TCP port 50051 on localhost, in 0.02 seconds) worker2 # [ 22.548565] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.553667] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.661319] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.665174] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.651566] systemd[1]: Starting scheduler-token.service... worker2 # [ 22.724279] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 22.730934] systemd[1]: Finished scheduler-token.service. worker2 # [ 22.732858] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 22.754523] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.758132] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 22.862069] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.867236] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 22.959363] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.964380] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.068142] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.073247] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.165604] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.170917] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.273887] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.279110] nix-grpc-daemon[767]: ts=2026-09-22T16:58:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.373103] nix-grpc-daemon[774]: ts=2026-09-22T16:58:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.377715] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.479870] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.484999] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.578818] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.584144] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.687432] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.691552] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.785246] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.790496] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.887796] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 23.892971] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 23.991438] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.996938] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.093656] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.098973] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.198039] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.203360] nix-grpc-daemon[774]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.299665] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.304811] nix-grpc-daemon[767]: ts=2026-09-22T16:58:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.406511] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.410626] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.505576] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.510463] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.607250] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.613105] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.713060] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.717436] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.814710] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.824266] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.918349] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.923417] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 25.023989] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.029140] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.124303] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.129291] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.229947] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.235071] nix-grpc-daemon[774]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.329986] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.335146] nix-grpc-daemon[767]: ts=2026-09-22T16:58:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.437981] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.442204] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.536057] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.541509] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.638677] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.641406] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.744221] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.748227] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.842678] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.847866] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.944856] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.949933] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 26.048976] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.054011] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 26.150854] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.156157] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.254946] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.260184] nix-grpc-daemon[774]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.356850] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.361564] nix-grpc-daemon[767]: ts=2026-09-22T16:58:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.463070] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.467123] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.562301] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.567404] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.663954] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.669329] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.770134] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.774263] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.870428] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.875681] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.970733] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.976001] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 27.076829] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.081877] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 27.176858] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.181980] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.282914] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.287989] nix-grpc-daemon[774]: ts=2026-09-22T16:58:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.382808] nix-grpc-daemon[767]: ts=2026-09-22T16:58:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.388001] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.491260] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.495511] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.588844] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.593986] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.691885] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.697089] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.796540] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.800601] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.897956] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.902904] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.996940] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.001833] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.103860] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.108852] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 28.202508] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.207700] nix-grpc-daemon[767]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 28.309971] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.315074] nix-grpc-daemon[774]: ts=2026-09-22T16:58:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.408401] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.413276] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.517954] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.522007] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.614254] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.619500] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.718747] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.723573] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.822195] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.826298] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.924655] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.929790] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.022651] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.027673] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.130833] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.135797] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.228322] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.233162] nix-grpc-daemon[767]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 29.336768] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.341856] nix-grpc-daemon[774]: ts=2026-09-22T16:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.434105] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.439146] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.544608] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.548636] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.640145] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.645105] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.745284] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.750449] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.847875] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.852505] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.951387] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.956570] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.048452] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.053674] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.157618] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.162811] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.254292] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.259278] nix-grpc-daemon[767]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 30.363878] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.369107] nix-grpc-daemon[774]: ts=2026-09-22T16:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.460079] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.464921] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.571801] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.575989] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.665823] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.670877] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.772720] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.777819] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.873463] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.877264] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.978813] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.983902] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.074032] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.079217] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.184982] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.190167] nix-grpc-daemon[774]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.279502] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.285850] nix-grpc-daemon[767]: ts=2026-09-22T16:58:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 31.391057] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.396527] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 31.484747] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.490034] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.599728] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.603953] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.690832] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.696268] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.800414] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.805354] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.899026] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.903294] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.006531] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.011702] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.099410] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.104823] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.212729] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.218356] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.305420] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.310447] nix-grpc-daemon[767]: ts=2026-09-22T16:58:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.336895] nix-grpc-daemon[774]: ts=2026-09-22T16:58:56Z level=info event=scheduler_take_over lb # [ 32.367601] niks3-server[964]: 2026/09/22 16:58:56 INFO lead: acquired remote=192.168.1.4:37628 lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 32.419249] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.424133] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 32.511272] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.516519] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.627470] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.631708] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.717345] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.722535] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.828182] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.833266] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.925257] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.929553] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.034676] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.039950] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.125807] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.130817] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.241081] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.246519] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.331489] nix-grpc-daemon[767]: ts=2026-09-22T16:58:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.308837] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:44234 worker2 # [ 33.313960] nix-grpc-daemon[774]: ts=2026-09-22T16:58:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 33.447382] nix-grpc-daemon[774]: ts=2026-09-22T16:58:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.460159] nix-grpc-daemon[774]: ts=2026-09-22T16:58:58Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 33.465959] nix-grpc-daemon[774]: ts=2026-09-22T16:58:58Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:44234 worker2 # [ 33.471369] nix-grpc-daemon[774]: ts=2026-09-22T16:58:58Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 11.39 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 34.503123] nix-grpc-daemon[767]: ts=2026-09-22T16:58:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 34.518648] systemd[1]: Started Nix Daemon. worker1 # [ 34.718616] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 34.739703] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5) worker1 # [ 34.788480] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5) worker1 # [ 34.901236] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5) worker1 # [ 35.298297] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 248 ms (attempt 4/5) worker1 # [ 35.550063] nix-grpc-daemon[767]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 35.555567] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 35.571357] nix-grpc-daemon[767]: ts=2026-09-22T16:59:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 35.585145] nix-grpc-daemon[767]: ts=2026-09-22T16:59:00Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 35.579509] nix-grpc-daemon[774]: ts=2026-09-22T16:59:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 35.598060] systemd[1]: Started Nix Daemon. worker2 # [ 35.780790] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 35.798003] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5) worker2 # [ 35.900940] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5) worker2 # [ 35.999729] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5) worker2 # [ 36.250778] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 531 ms (attempt 4/5) worker2 # [ 36.785348] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 36.791251] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 36.802489] nix-grpc-daemon[774]: ts=2026-09-22T16:59:01Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker2 # [ 37.130736] nix-grpc-daemon[774]: ts=2026-09-22T16:59:01Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:44240 worker2 # [ 37.142691] nix-grpc-daemon[774]: ts=2026-09-22T16:59:01Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:44240 worker2 # [ 37.154083] nix-grpc-daemon[774]: ts=2026-09-22T16:59:01Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 37.201536] nix-grpc-daemon[767]: ts=2026-09-22T16:59:01Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 37.201830] nix-grpc-daemon[767]: ts=2026-09-22T16:59:01Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:53218 worker1 # [ 37.217137] nix-grpc-daemon[767]: ts=2026-09-22T16:59:01Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 37.421676] nix-grpc-daemon[774]: ts=2026-09-22T16:59:02Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 37.589981] nix-grpc-daemon[774]: ts=2026-09-22T16:59:02Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.39 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 37.783116] nix-grpc-daemon[767]: ts=2026-09-22T16:59:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 37.757253] nix-grpc-daemon[774]: ts=2026-09-22T16:59:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 37.795163] nix-grpc-daemon[774]: ts=2026-09-22T16:59:02Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker2 # [ 37.801686] nix-grpc-daemon[774]: ts=2026-09-22T16:59:02Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 37.875794] nix-grpc-daemon[767]: ts=2026-09-22T16:59:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 37.892348] nix-grpc-daemon[767]: ts=2026-09-22T16:59:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying 1 paths to 'lb:50051'... worker1 # [ 37.906388] nix-grpc-daemon[767]: ts=2026-09-22T16:59:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 37.932183] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 37.971093] nix-grpc-daemon[767]: ts=2026-09-22T16:59:02Z level=info event=niks3_push_started pid=844 worker1 # [ 38.141046] nix-grpc-daemon[844]: time=2026-09-22T16:59:02.755Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 38.253737] nix-daemon[831]: accepted connection from pid 857, user nix-grpc-daemon (trusted) worker1 # [ 38.274579] nix-daemon[831]: reaped child process 860, status = succeeded lb # [ 38.275676] niks3-server[964]: 2026/09/22 16:59:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 38.296874] nix-grpc-daemon[844]: time=2026-09-22T16:59:02.912Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 38.298308] nix-grpc-daemon[844]: time=2026-09-22T16:59:02.912Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 38.325173] niks3-server[964]: 2026/09/22 16:59:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 38.355872] niks3-server[964]: 2026/09/22 16:59:02 INFO Signed narinfos id=1 count=1 worker1 # [ 38.360355] nix-grpc-daemon[844]: time=2026-09-22T16:59:02.976Z level=INFO msg="Uploading 1 narinfos" lb # [ 38.367575] niks3-server[964]: 2026/09/22 16:59:02 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 38.385196] niks3-server[964]: 2026/09/22 16:59:02 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 38.397786] niks3-server[964]: 2026/09/22 16:59:03 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls worker1 # [ 38.406015] nix-grpc-daemon[844]: time=2026-09-22T16:59:03.022Z level=INFO msg="Upload complete. (265ms)" lb # [ 38.408078] niks3-server[964]: 2026/09/22 16:59:03 INFO Completed upload id=1 worker1 # [ 38.417150] nix-grpc-daemon[767]: ts=2026-09-22T16:59:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 38.436586] nix-grpc-daemon[767]: ts=2026-09-22T16:59:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 38.449457] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 38.453553] nix-daemon[831]: reaped child process 842, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 38.469846] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 39.328524] nix-daemon[831]: reaped child process 864, status = succeeded worker1 # [ 39.431079] nix-daemon[831]: accepted connection from pid 881, user nix-grpc-daemon (trusted) worker1 # [ 39.449795] nix-daemon[831]: reaped child process 884, status = succeeded lb # [ 39.454627] niks3-server[964]: 2026/09/22 16:59:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 39.476178] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.090Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 39.478134] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.090Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 39.521195] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 39.531735] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 39.539468] niks3-server[964]: 2026/09/22 16:59:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign worker1 # [ 39.545731] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.161Z level=INFO msg="Uploading 1 narinfos" lb # [ 39.547113] niks3-server[964]: 2026/09/22 16:59:04 INFO Signed narinfos id=2 count=1 lb # [ 39.551100] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 39.560243] niks3-server[964]: 2026/09/22 16:59:04 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 39.569604] niks3-server[964]: 2026/09/22 16:59:04 INFO Completed upload id=2 worker1 # [ 39.571024] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.187Z level=INFO msg="Upload complete. (289ms)" lb # [ 39.572145] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 39.585245] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 39.561270] nix-grpc-daemon[774]: ts=2026-09-22T16:59:04Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 39.567359] nix-grpc-daemon[774]: ts=2026-09-22T16:59:04Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 39.604674] nix-daemon[831]: reaped child process 862, status = succeeded worker1 # [ 39.646586] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 39.663368] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 39.678109] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.294Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 39.689007] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 39.710762] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 39.880166] nix-daemon[831]: accepted connection from pid 888, user nix-grpc-daemon (trusted) worker1 # [ 39.897930] nix-daemon[831]: reaped child process 891, status = succeeded lb # [ 39.903665] niks3-server[964]: 2026/09/22 16:59:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 39.915739] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.532Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 39.917612] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.532Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 39.934324] envoy[919]: conn peer=[2001:db8:1::1]:52328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=2368 worker1 # [ 39.950786] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.567Z level=INFO msg="Uploading 1 narinfos" lb # [ 39.951877] envoy[919]: conn peer=[2001:db8:1::1]:52332 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=73 lb # [ 39.963248] envoy[919]: conn peer=[2001:db8:1::1]:52336 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=10 lb # [ 39.970936] envoy[919]: conn peer=[2001:db8:1::1]:52338 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=12 lb # [ 39.982409] niks3-server[964]: 2026/09/22 16:59:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 39.986632] niks3-server[964]: 2026/09/22 16:59:04 INFO Signed narinfos id=3 count=1 worker1 # [ 39.988581] nix-grpc-daemon[844]: time=2026-09-22T16:59:04.605Z level=INFO msg="Upload complete. (260ms)" lb # [ 39.991348] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 39.999581] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 40.003667] niks3-server[964]: 2026/09/22 16:59:04 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 40.008476] niks3-server[964]: 2026/09/22 16:59:04 INFO Completed upload id=3 worker1 # [ 40.000360] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 40.010687] niks3-server[964]: 2026/09/22 16:59:04 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 40.012703] nix-grpc-daemon[767]: ts=2026-09-22T16:59:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 40.027303] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 40.029865] nix-daemon[831]: reaped child process 886, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 40.043847] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 41.511366] nix-daemon[831]: reaped child process 895, status = succeeded worker1 # [ 41.626768] nix-daemon[831]: accepted connection from pid 912, user nix-grpc-daemon (trusted) worker1 # [ 41.647546] nix-daemon[831]: reaped child process 915, status = succeeded lb # [ 41.645709] niks3-server[964]: 2026/09/22 16:59:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 41.656830] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.273Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 41.659900] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.273Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 41.689594] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls worker1 # [ 41.704104] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.320Z level=INFO msg="Uploading 1 narinfos" lb # [ 41.705773] niks3-server[964]: 2026/09/22 16:59:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 41.715302] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 41.723758] niks3-server[964]: 2026/09/22 16:59:06 INFO Signed narinfos id=4 count=1 lb # [ 41.731485] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker1 # [ 41.732668] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.349Z level=INFO msg="Upload complete. (277ms)" lb # [ 41.735873] niks3-server[964]: 2026/09/22 16:59:06 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 41.741076] niks3-server[964]: 2026/09/22 16:59:06 INFO Completed upload id=4 lb # [ 41.743709] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 41.745198] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 41.723763] nix-grpc-daemon[774]: ts=2026-09-22T16:59:06Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker2 # [ 41.729807] nix-grpc-daemon[774]: ts=2026-09-22T16:59:06Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 41.763728] nix-daemon[831]: reaped child process 893, status = succeeded worker1 # [ 41.797113] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 41.814088] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 41.830022] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.446Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 41.841408] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 41.859169] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 42.022624] nix-daemon[831]: accepted connection from pid 919, user nix-grpc-daemon (trusted) worker1 # [ 42.040907] nix-daemon[831]: reaped child process 922, status = succeeded lb # [ 42.046421] niks3-server[964]: 2026/09/22 16:59:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.055412] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.671Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 42.057222] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.671Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 42.066148] niks3-server[964]: 2026/09/22 16:59:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker1 # [ 42.072001] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.688Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.073699] niks3-server[964]: 2026/09/22 16:59:06 INFO Signed narinfos id=5 count=1 lb # [ 42.077834] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 42.085498] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls worker1 # [ 42.090811] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.707Z level=INFO msg="Upload complete. (214ms)" lb # [ 42.089910] niks3-server[964]: 2026/09/22 16:59:06 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 42.093129] niks3-server[964]: 2026/09/22 16:59:06 INFO Completed upload id=5 worker1 # [ 42.096549] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 42.096588] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 42.104807] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 42.114724] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 42.118825] nix-daemon[831]: reaped child process 917, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker1 # [ 42.130451] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 43.220386] nix-daemon[831]: reaped child process 926, status = succeeded worker1 # [ 43.292946] nix-daemon[831]: accepted connection from pid 942, user nix-grpc-daemon (trusted) worker1 # [ 43.312761] nix-daemon[831]: reaped child process 945, status = succeeded lb # [ 43.314163] niks3-server[964]: 2026/09/22 16:59:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.328417] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 43.330062] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.943Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 43.352743] niks3-server[964]: 2026/09/22 16:59:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 43.362880] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.979Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.368342] niks3-server[964]: 2026/09/22 16:59:07 INFO Signed narinfos id=6 count=1 lb # [ 43.376712] niks3-server[964]: 2026/09/22 16:59:07 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 43.384930] niks3-server[964]: 2026/09/22 16:59:07 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 43.393051] niks3-server[964]: 2026/09/22 16:59:08 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 43.403751] niks3-server[964]: 2026/09/22 16:59:08 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst worker1 # [ 43.411964] nix-grpc-daemon[844]: time=2026-09-22T16:59:08.028Z level=INFO msg="Upload complete. (272ms)" lb # [ 43.410015] niks3-server[964]: 2026/09/22 16:59:08 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo lb # [ 43.419021] niks3-server[964]: 2026/09/22 16:59:08 INFO Completed upload id=6 worker1 # [ 43.425000] nix-grpc-daemon[767]: ts=2026-09-22T16:59:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 43.443272] nix-daemon[831]: reaped child process 924, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.84 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.36 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 44.687921] nix-grpc-daemon[767]: ts=2026-09-22T16:59:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # [ 44.704684] nix-grpc-daemon[767]: ts=2026-09-22T16:59:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 44.715714] nix-grpc-daemon[844]: time=2026-09-22T16:59:09.331Z level=INFO msg="All 2 paths already cached" worker2 # [ 44.705100] nix-grpc-daemon[774]: ts=2026-09-22T16:59:09Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.31 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.20 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 worker2 # [ 45.586650] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44240 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 45.596458] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 45.622825] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker2 # [ 45.628358] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 45.698577] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 45.713921] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 45.729538] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.345Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 45.740548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 45.754843] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 45.915880] nix-daemon[831]: accepted connection from pid 963, user nix-grpc-daemon (trusted) worker1 # [ 45.936542] nix-daemon[831]: reaped child process 966, status = succeeded lb # [ 45.942188] niks3-server[964]: 2026/09/22 16:59:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.953353] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.569Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.954854] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.569Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 45.970088] niks3-server[964]: 2026/09/22 16:59:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker1 # [ 45.981078] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.597Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.982475] niks3-server[964]: 2026/09/22 16:59:10 INFO Signed narinfos id=7 count=1 lb # [ 45.992074] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 46.007278] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 46.020300] niks3-server[964]: 2026/09/22 16:59:10 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker1 # [ 46.028786] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.645Z level=INFO msg="Upload complete. (256ms)" lb # [ 46.030045] niks3-server[964]: 2026/09/22 16:59:10 INFO Completed upload id=7 lb # [ 46.033929] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 46.037332] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 46.054952] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 46.062068] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 46.067300] nix-daemon[831]: reaped child process 961, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 46.080234] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 47.518366] nix-daemon[831]: reaped child process 970, status = succeeded worker1 # [ 47.628095] nix-daemon[831]: accepted connection from pid 986, user nix-grpc-daemon (trusted) worker1 # [ 47.645093] nix-daemon[831]: reaped child process 989, status = succeeded lb # [ 47.650575] niks3-server[964]: 2026/09/22 16:59:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 47.664815] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.281Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 47.666336] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.281Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 47.705980] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 47.719261] niks3-server[964]: 2026/09/22 16:59:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker1 # [ 47.726749] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.343Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.728567] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 47.736794] niks3-server[964]: 2026/09/22 16:59:12 INFO Signed narinfos id=8 count=1 lb # [ 47.738711] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 47.750093] niks3-server[964]: 2026/09/22 16:59:12 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 47.755690] niks3-server[964]: 2026/09/22 16:59:12 INFO Completed upload id=8 worker1 # [ 47.757090] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.373Z level=INFO msg="Upload complete. (266ms)" lb # [ 47.759242] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 47.771466] nix-grpc-daemon[767]: ts=2026-09-22T16:59:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 47.793473] nix-daemon[831]: reaped child process 968, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.42 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.05 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 worker1 # [ 48.511541] nix-grpc-daemon[767]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 48.497497] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 48.543906] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker2 # [ 48.554841] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 48.596366] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 48.619474] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 48.642035] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 48.661698] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 48.694408] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=info event=niks3_push_started pid=912 worker2 # [ 48.898890] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.522Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 49.024744] nix-daemon[859]: accepted connection from pid 924, user nix-grpc-daemon (trusted) worker2 # [ 49.049007] nix-daemon[859]: reaped child process 927, status = succeeded lb # [ 49.079361] niks3-server[964]: 2026/09/22 16:59:13 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 49.067026] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 49.071221] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.689Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 49.141444] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls worker2 # [ 49.126985] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.749Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.155221] niks3-server[964]: 2026/09/22 16:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 49.167452] niks3-server[964]: 2026/09/22 16:59:13 INFO Signed narinfos id=9 count=1 lb # [ 49.183848] niks3-server[964]: 2026/09/22 16:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 49.191751] niks3-server[964]: 2026/09/22 16:59:13 INFO Completed upload id=9 worker2 # [ 49.163241] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.786Z level=INFO msg="Upload complete. (260ms)" lb # [ 49.196775] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst worker2 # [ 49.176409] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 49.207406] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 49.187100] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 49.203661] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 49.209977] nix-daemon[859]: reaped child process 910, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker2 # [ 49.227103] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) lb # [ 49.938400] envoy[919]: conn peer=[2001:db8:1::1]:52346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9493 tx=4368 ms=5661 lb # [ 49.943997] envoy[919]: conn peer=[2001:db8:1::1]:48152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=68 lb # [ 49.951029] envoy[919]: conn peer=[2001:db8:1::1]:48154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1895 ms=2169 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 50.189463] nix-grpc-daemon[767]: E0000 00:00:1790096354.802841 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client # farm-a> LOG-farm-a-sig worker1 # [ 50.246286] nix-grpc-daemon[767]: E0000 00:00:1790096354.862633 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 50.257441] nix-daemon[859]: reaped child process 931, status = succeeded worker1 # [ 50.295931] nix-grpc-daemon[767]: E0000 00:00:1790096354.868476 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 50.300144] nix-grpc-daemon[767]: E0000 00:00:1790096354.908422 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 50.363081] nix-daemon[859]: accepted connection from pid 947, user nix-grpc-daemon (trusted) worker2 # [ 50.382442] nix-daemon[859]: reaped child process 950, status = succeeded lb # [ 50.422215] niks3-server[964]: 2026/09/22 16:59:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 50.406498] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 50.407973] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.028Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 50.491360] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 50.499971] niks3-server[964]: 2026/09/22 16:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 50.506323] niks3-server[964]: 2026/09/22 16:59:15 INFO Signed narinfos id=10 count=1 worker2 # [ 50.476783] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.100Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.513631] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 50.519234] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 50.528923] niks3-server[964]: 2026/09/22 16:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 50.536518] niks3-server[964]: 2026/09/22 16:59:15 INFO Completed upload id=10 worker2 # [ 50.508444] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.132Z level=INFO msg="Upload complete. (269ms)" lb # [ 50.540273] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker2 # [ 50.533620] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker2 # [ 50.551227] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker2 # [ 50.555918] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 50.561316] nix-daemon[859]: reaped child process 929, status = succeeded worker2 # [ 50.574844] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 50.595768] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 50.615981] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.239Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 50.630289] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 50.646998] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 50.806847] nix-daemon[859]: accepted connection from pid 956, user nix-grpc-daemon (trusted) worker2 # [ 50.828869] nix-daemon[859]: reaped child process 959, status = succeeded lb # [ 50.861425] niks3-server[964]: 2026/09/22 16:59:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 50.843546] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.467Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 50.845880] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.467Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 50.895154] niks3-server[964]: 2026/09/22 16:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker2 # [ 50.872495] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.496Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.907042] niks3-server[964]: 2026/09/22 16:59:15 INFO Signed narinfos id=11 count=1 lb # [ 50.911211] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 50.930146] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls worker2 # [ 50.907596] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.531Z level=INFO msg="Upload complete. (242ms)" lb # [ 50.937202] niks3-server[964]: 2026/09/22 16:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 50.940792] niks3-server[964]: 2026/09/22 16:59:15 INFO Completed upload id=11 lb # [ 50.943836] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 50.916319] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 50.928412] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 50.941285] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 50.946489] nix-daemon[859]: reaped child process 954, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker2 # [ 50.957459] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 51.848832] nix-daemon[859]: reaped child process 963, status = succeeded worker2 # [ 51.941163] nix-daemon[859]: accepted connection from pid 979, user nix-grpc-daemon (trusted) worker2 # [ 51.960498] nix-daemon[859]: reaped child process 982, status = succeeded lb # [ 51.995925] niks3-server[964]: 2026/09/22 16:59:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.977481] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 51.979143] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.600Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 52.040860] niks3-server[964]: 2026/09/22 16:59:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker2 # [ 52.022677] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.646Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.057259] niks3-server[964]: 2026/09/22 16:59:16 INFO Signed narinfos id=12 count=1 lb # [ 52.065201] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 52.078308] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls worker2 # [ 52.053880] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.677Z level=INFO msg="Upload complete. (284ms)" lb # [ 52.086869] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst lb # [ 52.092377] niks3-server[964]: 2026/09/22 16:59:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 52.096680] niks3-server[964]: 2026/09/22 16:59:16 INFO Completed upload id=12 lb # [ 52.099595] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker2 # [ 52.071563] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker2 # [ 52.089948] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 52.094089] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 52.098531] nix-daemon[859]: reaped child process 961, status = succeeded worker2 # [ 52.128124] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 52.143090] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 52.157623] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.781Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 52.168711] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 52.182576] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 52.338496] nix-daemon[859]: accepted connection from pid 988, user nix-grpc-daemon (trusted) worker2 # [ 52.358062] nix-daemon[859]: reaped child process 991, status = succeeded lb # [ 52.395027] niks3-server[964]: 2026/09/22 16:59:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.373073] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.996Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.374590] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.996Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" worker2 # [ 52.398688] nix-grpc-daemon[912]: time=2026-09-22T16:59:17.022Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.427484] niks3-server[964]: 2026/09/22 16:59:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 52.435596] niks3-server[964]: 2026/09/22 16:59:17 INFO Signed narinfos id=13 count=1 lb # [ 52.440890] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 52.449455] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker2 # [ 52.427281] nix-grpc-daemon[912]: time=2026-09-22T16:59:17.051Z level=INFO msg="Upload complete. (227ms)" lb # [ 52.458161] niks3-server[964]: 2026/09/22 16:59:17 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 52.464353] niks3-server[964]: 2026/09/22 16:59:17 INFO Completed upload id=13 worker2 # [ 52.435703] nix-grpc-daemon[774]: ts=2026-09-22T16:59:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 52.466356] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 52.447466] nix-grpc-daemon[774]: ts=2026-09-22T16:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 52.456227] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 52.458963] nix-daemon[859]: reaped child process 986, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker2 # [ 52.472375] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 53.364833] nix-daemon[859]: reaped child process 995, status = succeeded worker2 # [ 53.445061] nix-daemon[859]: accepted connection from pid 1011, user nix-grpc-daemon (trusted) worker2 # [ 53.466296] nix-daemon[859]: reaped child process 1014, status = succeeded lb # [ 53.494705] niks3-server[964]: 2026/09/22 16:59:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 53.476606] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.099Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 53.480393] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.099Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 53.536636] niks3-server[964]: 2026/09/22 16:59:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 53.515890] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.139Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.552164] niks3-server[964]: 2026/09/22 16:59:18 INFO Signed narinfos id=14 count=1 lb # [ 53.554894] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst lb # [ 53.571874] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls worker2 # [ 53.548742] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.172Z level=INFO msg="Upload complete. (266ms)" lb # [ 53.578585] niks3-server[964]: 2026/09/22 16:59:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 53.581851] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 53.586047] niks3-server[964]: 2026/09/22 16:59:18 INFO Completed upload id=14 lb # [ 53.588456] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 53.561110] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 53.579145] nix-daemon[859]: reaped child process 993, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.45 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.20 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 54.018995] nix-grpc-daemon[767]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:39222 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 54.059169] nix-grpc-daemon[767]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:39222 worker1 # [ 54.068666] nix-grpc-daemon[767]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 54.086012] nix-grpc-daemon[767]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 54.100538] nix-grpc-daemon[767]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 54.297036] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:41062 worker2 # [ 54.338827] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:41062 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/896" client # removing stale temporary roots file "/nix/var/nix/temproots/941" client # removing stale temporary roots file "/nix/var/nix/temproots/905" client # removing stale temporary roots file "/nix/var/nix/temproots/884" client # removing stale temporary roots file "/nix/var/nix/temproots/918" client # removing stale temporary roots file "/nix/var/nix/temproots/926" client # removing stale temporary roots file "/nix/var/nix/temproots/863" client # removing stale temporary roots file "/nix/var/nix/temproots/846" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 54.737595] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 54.746164] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 54.757520] nix-grpc-daemon[912]: time=2026-09-22T16:59:19.380Z level=INFO msg="All 1 paths already cached" worker1 # [ 54.796101] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 54.776277] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 54.785041] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:44240 client # copying 3 paths... worker1 # [ 54.830779] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker1 # [ 54.861083] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 54.868686] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 54.877261] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 54.882597] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 54.861251] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 54.868515] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.53 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 6.72 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 55.052811] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 0edca4d2b8df40c29eef9d643341df50 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker1 # [ 55.193860] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 55.244063] nix[994]: don't know how to build these paths: client # [ 55.245977] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 55.182439] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 55.188589] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 55.257278] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 55.272989] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 55.329196] nix[994]: copying 1 paths to 'lb:50051'... worker1 # [ 55.287472] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 55.333554] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 55.305197] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 55.483569] nix-daemon[831]: accepted connection from pid 996, user nix-grpc-daemon (trusted) worker1 # [ 55.507870] nix-daemon[831]: reaped child process 999, status = succeeded lb # [ 55.506817] niks3-server[964]: 2026/09/22 16:59:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 55.518021] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 55.519491] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.134Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" worker1 # [ 55.549885] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.166Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.540882] niks3-server[964]: 2026/09/22 16:59:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 55.552349] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 55.562641] niks3-server[964]: 2026/09/22 16:59:20 INFO Signed narinfos id=15 count=1 lb # [ 55.572867] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker1 # [ 55.580771] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.197Z level=INFO msg="Upload complete. (256ms)" lb # [ 55.581483] niks3-server[964]: 2026/09/22 16:59:20 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete worker1 # [ 55.588548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 55.591484] niks3-server[964]: 2026/09/22 16:59:20 INFO Completed upload id=15 lb # [ 55.595596] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 55.601176] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 55.613922] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 55.619878] nix-daemon[831]: reaped child process 994, status = succeeded worker1 # [ 55.630092] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # [ 55.672472] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client # [ 56.142733] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: d21e033d5bdb4b6c94f137f3f2473276 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) worker1 # [ 56.337841] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 56.392668] nix[1010]: this derivation will be built: worker1 # [ 56.350905] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 56.400078] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 56.336599] nix-grpc-daemon[774]: ts=2026-09-22T16:59:20Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 56.369960] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 56.417273] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 59.719740] nix-daemon[831]: reaped child process 1003, status = succeeded worker1 # [ 59.785306] nix-daemon[831]: accepted connection from pid 1033, user nix-grpc-daemon (trusted) worker1 # [ 59.801713] nix-daemon[831]: reaped child process 1036, status = succeeded lb # [ 59.809067] niks3-server[964]: 2026/09/22 16:59:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 59.820389] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.435Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 59.821965] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.435Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 59.858191] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 59.869132] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 59.879538] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.495Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.881744] niks3-server[964]: 2026/09/22 16:59:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 59.890715] niks3-server[964]: 2026/09/22 16:59:24 INFO Signed narinfos id=16 count=1 lb # [ 59.896412] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 59.907008] niks3-server[964]: 2026/09/22 16:59:24 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 59.913382] niks3-server[964]: 2026/09/22 16:59:24 INFO Completed upload id=16 worker1 # [ 59.913058] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.529Z level=INFO msg="Upload complete. (269ms)" lb # [ 59.917583] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 59.925121] nix-grpc-daemon[767]: ts=2026-09-22T16:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 lb # [ 59.940756] envoy[919]: conn peer=[2001:db8:1::1]:48160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9472 tx=4355 ms=5143 lb # [ 59.944640] envoy[919]: conn peer=[2001:db8:1::1]:56766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=179 lb # [ 59.948446] envoy[919]: conn peer=[2001:db8:1::1]:56770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=86 worker1 # [ 59.944422] nix-grpc-daemon[767]: ts=2026-09-22T16:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 59.953469] nix-daemon[831]: reaped child process 1001, status = succeeded lb # [ 59.952589] envoy[919]: conn peer=[2001:db8:1::1]:56792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=54 lb # [ 59.955787] envoy[919]: conn peer=[2001:db8:1::1]:56784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=70 lb # [ 59.958792] envoy[919]: conn peer=[2001:db8:1::1]:56796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=4750 client # [ 60.005836] systemd[1]: dedup1.service: Deactivated successfully. client # [ 60.011226] systemd[1]: dedup1.service: Consumed 171ms CPU time over 4.957s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 60.046379] systemd[1]: dedup2.service: Deactivated successfully. client # [ 60.054907] systemd[1]: dedup2.service: Consumed 213ms CPU time over 3.910s wall clock time, 35.3M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 61.273633] nix-grpc-daemon[767]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 61.260974] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 61.261383] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 61.303971] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 61.321076] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 61.336746] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 61.353599] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 61.509854] nix-daemon[859]: accepted connection from pid 1045, user nix-grpc-daemon (trusted) worker2 # [ 61.527678] nix-daemon[859]: reaped child process 1048, status = succeeded lb # [ 61.566159] niks3-server[964]: 2026/09/22 16:59:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 61.545286] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.168Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 61.546833] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.168Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 61.595420] niks3-server[964]: 2026/09/22 16:59:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 61.575812] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.199Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.610499] niks3-server[964]: 2026/09/22 16:59:26 INFO Signed narinfos id=17 count=1 lb # [ 61.618894] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 61.604742] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.228Z level=INFO msg="Upload complete. (234ms)" lb # [ 61.633368] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 61.643973] niks3-server[964]: 2026/09/22 16:59:26 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker2 # [ 61.614095] nix-grpc-daemon[774]: ts=2026-09-22T16:59:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 61.647550] niks3-server[964]: 2026/09/22 16:59:26 INFO Completed upload id=17 lb # [ 61.650595] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 61.624796] nix-grpc-daemon[774]: ts=2026-09-22T16:59:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 61.635471] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 61.641283] nix-daemon[859]: reaped child process 1043, status = succeeded worker2 # [ 61.652148] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 67.468943] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 67.495170] nix-daemon[859]: reaped child process 1050, status = succeeded worker2 # [ 67.497061] nix-daemon[859]: reaped child process 1052, status = succeeded worker2 # [ 67.779377] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 67.794680] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker2 # [ 67.801109] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 67.835664] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 67.851548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 67.865938] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 67.881113] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 68.045522] nix-daemon[831]: accepted connection from pid 1049, user nix-grpc-daemon (trusted) worker1 # [ 68.066360] nix-daemon[831]: reaped child process 1052, status = succeeded lb # [ 68.067430] niks3-server[964]: 2026/09/22 16:59:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 68.078437] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.694Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 68.080098] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.694Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 68.104523] niks3-server[964]: 2026/09/22 16:59:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 68.114287] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.730Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.117397] niks3-server[964]: 2026/09/22 16:59:32 INFO Signed narinfos id=18 count=1 lb # [ 68.124452] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 68.134542] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls worker1 # [ 68.144118] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.760Z level=INFO msg="Upload complete. (246ms)" lb # [ 68.143374] niks3-server[964]: 2026/09/22 16:59:32 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 68.146995] niks3-server[964]: 2026/09/22 16:59:32 INFO Completed upload id=18 lb # [ 68.151616] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 68.156138] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 68.172677] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 68.180397] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 68.185922] nix-daemon[831]: reaped child process 1047, status = succeeded worker1 # [ 68.197274] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) lb # [ 69.939747] envoy[919]: conn peer=[2001:db8:1::1]:56802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1598 ms=3652 lb # [ 69.945181] envoy[919]: conn peer=[2001:db8:1::1]:42492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3027 tx=978 ms=6323 worker1 # [ 74.413866] nix-grpc-daemon[767]: ts=2026-09-22T16:59:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 74.431256] nix-daemon[831]: reaped child process 1054, status = succeeded worker1 # [ 74.432563] nix-daemon[831]: reaped child process 1056, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.57 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 74.733483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: d954108ada2d4cbf9a1690ddcdb4ee12 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 74.877073] nix-grpc-daemon[767]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 client # [ 74.927411] nix[1083]: don't know how to build these paths: client # [ 74.928501] nix[1083]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 74.870178] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 74.879140] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 74.887651] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 74.895379] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 client # [ 74.976251] nix[1083]: copying 1 paths to 'lb:50051'... client # [ 74.979185] nix[1083]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 74.907682] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 74.922959] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 75.074929] nix-daemon[859]: accepted connection from pid 1079, user nix-grpc-daemon (trusted) worker2 # [ 75.096344] nix-daemon[859]: reaped child process 1082, status = succeeded lb # [ 75.128692] niks3-server[964]: 2026/09/22 16:59:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 75.110350] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 75.111879] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.733Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 75.162326] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst worker2 # [ 75.146203] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.769Z level=INFO msg="Uploading 1 narinfos" lb # [ 75.176398] niks3-server[964]: 2026/09/22 16:59:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 75.186731] niks3-server[964]: 2026/09/22 16:59:39 INFO Signed narinfos id=19 count=1 lb # [ 75.192891] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 75.203827] niks3-server[964]: 2026/09/22 16:59:39 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker2 # [ 75.181804] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.805Z level=INFO msg="Upload complete. (242ms)" lb # [ 75.210976] niks3-server[964]: 2026/09/22 16:59:39 INFO Completed upload id=19 lb # [ 75.215803] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 75.192378] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 75.211942] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 75.219983] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 75.226742] nix-daemon[859]: reaped child process 1077, status = succeeded client # [ 75.312555] nix[1083]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 75.241365] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # [ 75.829260] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 9c8bd7543c074f289c01a62c4f9cfe05 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) client: must succeed: sleep 2; systemctl kill -s INT share1 worker2 # [ 75.968284] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44240 client # [ 76.050615] nix[1099]: this derivation will be built: worker1 # [ 76.013836] nix-grpc-daemon[767]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 76.025769] nix-grpc-daemon[767]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'... client # [ 76.086933] nix[1099]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 76.022444] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 76.025344] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 client # [ 76.102101] nix[1099]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 78.052674] systemd[1]: share1.service: Sent signal SIGINT to main process 1083 (nix) on client request. client # [ 78.062793] nix[1083]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 78.076428] nix[1083]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 78.132827] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 78.136591] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 78.139171] systemd[1]: share1.service: Consumed 162ms CPU time over 3.404s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 79.492577] nix-daemon[859]: reaped child process 1086, status = succeeded worker2 # [ 79.608814] nix-daemon[859]: accepted connection from pid 1118, user nix-grpc-daemon (trusted) worker2 # [ 79.626459] nix-daemon[859]: reaped child process 1121, status = succeeded lb # [ 79.661622] niks3-server[964]: 2026/09/22 16:59:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 79.644179] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.265Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 79.645640] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.265Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 79.696515] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 79.712438] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 79.721453] niks3-server[964]: 2026/09/22 16:59:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker2 # [ 79.699245] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.322Z level=INFO msg="Uploading 1 narinfos" lb # [ 79.732337] niks3-server[964]: 2026/09/22 16:59:44 INFO Signed narinfos id=20 count=1 lb # [ 79.740121] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls worker2 # [ 79.724225] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.348Z level=INFO msg="Upload complete. (261ms)" lb # [ 79.752548] niks3-server[964]: 2026/09/22 16:59:44 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 79.758837] niks3-server[964]: 2026/09/22 16:59:44 INFO Completed upload id=20 lb # [ 79.764378] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 79.736957] nix-grpc-daemon[774]: ts=2026-09-22T16:59:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 79.754951] nix-grpc-daemon[774]: ts=2026-09-22T16:59:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 79.761245] nix-daemon[859]: reaped child process 1084, status = succeeded client # [ 79.883807] systemd[1]: share2.service: Deactivated successfully. client # [ 79.888553] systemd[1]: share2.service: Consumed 184ms CPU time over 4.060s wall clock time, 33.3M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic. lb # [ 79.941182] envoy[919]: conn peer=[2001:db8:1::1]:42496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=952 ms=6666 lb # [ 79.946119] envoy[919]: conn peer=[2001:db8:1::1]:57688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=772 ms=3164 lb # [ 79.950927] envoy[919]: conn peer=[2001:db8:1::1]:57694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1605 tx=1598 ms=3820 client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 5.96 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 80.764092] systemd[1]: Started Nix Daemon. client # [ 80.877715] nix-daemon[1151]: accepted connection from pid 1148, user root (trusted) worker2 # [ 80.953284] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 80.959611] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 80.978718] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 80.984829] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 81.014895] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 81.030799] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 81.045112] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 81.060160] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 81.224347] nix-daemon[831]: accepted connection from pid 1092, user nix-grpc-daemon (trusted) worker1 # [ 81.246252] nix-daemon[831]: reaped child process 1095, status = succeeded lb # [ 81.244739] niks3-server[964]: 2026/09/22 16:59:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 81.258896] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 81.260572] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.875Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" worker1 # [ 81.285987] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.902Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.279435] niks3-server[964]: 2026/09/22 16:59:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 81.288989] niks3-server[964]: 2026/09/22 16:59:45 INFO Signed narinfos id=21 count=1 lb # [ 81.290969] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 81.299792] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 81.309929] niks3-server[964]: 2026/09/22 16:59:45 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 81.316205] niks3-server[964]: 2026/09/22 16:59:45 INFO Completed upload id=21 worker1 # [ 81.315727] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.932Z level=INFO msg="Upload complete. (238ms)" lb # [ 81.319477] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 81.324317] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 81.342142] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 81.349088] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 81.354355] nix-daemon[831]: reaped child process 1090, status = succeeded worker1 # [ 81.365299] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 82.225937] nix-daemon[831]: reaped child process 1099, status = succeeded worker1 # [ 82.356491] nix-daemon[831]: accepted connection from pid 1115, user nix-grpc-daemon (trusted) worker1 # [ 82.375173] nix-daemon[831]: reaped child process 1118, status = succeeded lb # [ 82.381879] niks3-server[964]: 2026/09/22 16:59:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.392448] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 82.395439] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.007Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 82.425879] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 82.438086] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.054Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.441561] niks3-server[964]: 2026/09/22 16:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 82.450126] niks3-server[964]: 2026/09/22 16:59:47 INFO Signed narinfos id=22 count=1 lb # [ 82.458108] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls worker1 # [ 82.465006] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.081Z level=INFO msg="Upload complete. (286ms)" lb # [ 82.464376] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 82.468567] niks3-server[964]: 2026/09/22 16:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 82.474907] niks3-server[964]: 2026/09/22 16:59:47 INFO Completed upload id=22 worker1 # [ 82.475715] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 lb # [ 82.481322] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker2 # [ 82.453150] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:44240 worker1 # [ 82.494787] nix-daemon[831]: reaped child process 1097, status = succeeded worker2 # [ 82.466075] nix-grpc-daemon[774]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 82.502836] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 82.509443] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 82.787323] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:44240 worker1 # [ 82.821161] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 82.831461] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.448Z level=INFO msg="All 1 paths already cached" worker2 # [ 82.820131] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker2 # [ 82.826439] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 82.859507] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 82.874932] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 82.889235] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.505Z level=INFO msg="All 1 paths already cached" worker1 # [ 82.898958] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 82.913036] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 83.071098] nix-daemon[831]: accepted connection from pid 1122, user nix-grpc-daemon (trusted) worker1 # [ 83.092648] nix-daemon[831]: reaped child process 1125, status = succeeded lb # [ 83.089348] niks3-server[964]: 2026/09/22 16:59:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 83.099435] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 83.101117] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.715Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 83.119845] niks3-server[964]: 2026/09/22 16:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker1 # [ 83.127361] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.743Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.134108] niks3-server[964]: 2026/09/22 16:59:47 INFO Signed narinfos id=23 count=1 lb # [ 83.143113] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 83.153381] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst lb # [ 83.165775] niks3-server[964]: 2026/09/22 16:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete worker1 # [ 83.172405] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.789Z level=INFO msg="Upload complete. (242ms)" lb # [ 83.171035] niks3-server[964]: 2026/09/22 16:59:47 INFO Completed upload id=23 lb # [ 83.175622] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 83.184349] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 83.200377] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 83.210756] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 83.218164] nix-daemon[831]: reaped child process 1120, status = succeeded worker1 # [ 83.231319] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 84.274329] nix-daemon[831]: reaped child process 1129, status = succeeded worker1 # [ 84.395863] nix-daemon[831]: accepted connection from pid 1145, user nix-grpc-daemon (trusted) worker1 # [ 84.411236] nix-daemon[831]: reaped child process 1148, status = succeeded lb # [ 84.418795] niks3-server[964]: 2026/09/22 16:59:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 84.432456] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.045Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 84.434087] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.045Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 84.467134] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker1 # [ 84.479266] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.095Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.480932] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 84.490800] niks3-server[964]: 2026/09/22 16:59:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 84.500529] niks3-server[964]: 2026/09/22 16:59:49 INFO Signed narinfos id=24 count=1 lb # [ 84.503121] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst worker1 # [ 84.512845] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.129Z level=INFO msg="Upload complete. (267ms)" lb # [ 84.514112] niks3-server[964]: 2026/09/22 16:59:49 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 84.517869] niks3-server[964]: 2026/09/22 16:59:49 INFO Completed upload id=24 lb # [ 84.521424] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 84.525278] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 84.544380] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 84.551909] nix-daemon[831]: reaped child process 1127, status = succeeded worker1 # [ 84.573321] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 84.580137] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 84.839830] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 84.850384] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 84.871032] nix-grpc-daemon[912]: time=2026-09-22T16:59:49.493Z level=INFO msg="All 1 paths already cached" worker2 # [ 84.891417] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 84.924738] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 84.898108] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 84.940834] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 84.956166] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.572Z level=INFO msg="All 2 paths already cached" worker1 # [ 84.966940] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 84.982562] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 85.149333] nix-daemon[831]: accepted connection from pid 1152, user nix-grpc-daemon (trusted) worker1 # [ 85.173891] nix-daemon[831]: reaped child process 1155, status = succeeded lb # [ 85.171012] niks3-server[964]: 2026/09/22 16:59:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 85.181912] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.798Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 85.183795] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.798Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 85.206103] niks3-server[964]: 2026/09/22 16:59:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 85.214334] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.830Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.221477] niks3-server[964]: 2026/09/22 16:59:49 INFO Signed narinfos id=25 count=1 lb # [ 85.228896] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 85.239786] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls worker1 # [ 85.246374] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.863Z level=INFO msg="Upload complete. (247ms)" lb # [ 85.246908] niks3-server[964]: 2026/09/22 16:59:49 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 85.252047] niks3-server[964]: 2026/09/22 16:59:49 INFO Completed upload id=25 lb # [ 85.255526] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 85.257310] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 85.272229] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 85.280720] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 85.283166] nix-daemon[831]: reaped child process 1150, status = succeeded worker1 # [ 85.296462] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 86.568991] nix-daemon[831]: reaped child process 1159, status = succeeded worker1 # [ 86.695770] nix-daemon[831]: accepted connection from pid 1175, user nix-grpc-daemon (trusted) worker1 # [ 86.720266] nix-daemon[831]: reaped child process 1178, status = succeeded lb # [ 86.740525] niks3-server[964]: 2026/09/22 16:59:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 86.753015] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.369Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 86.760992] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.372Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 86.810628] niks3-server[964]: 2026/09/22 16:59:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 86.818042] niks3-server[964]: 2026/09/22 16:59:51 INFO Signed narinfos id=26 count=1 lb # [ 86.821144] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker1 # [ 86.820697] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.436Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.828851] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 86.836797] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 86.851416] niks3-server[964]: 2026/09/22 16:59:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 86.857722] niks3-server[964]: 2026/09/22 16:59:51 INFO Completed upload id=26 worker1 # [ 86.857756] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.474Z level=INFO msg="Upload complete. (350ms)" lb # [ 86.860158] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 86.877990] nix-grpc-daemon[767]: ts=2026-09-22T16:59:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 86.910748] nix-daemon[831]: reaped child process 1157, status = succeeded worker1 # [ 86.929767] nix-grpc-daemon[767]: ts=2026-09-22T16:59:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 86.954619] nix-grpc-daemon[774]: ts=2026-09-22T16:59:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 86.980994] nix-grpc-daemon[774]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 87.008865] nix-grpc-daemon[774]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 87.033806] nix-grpc-daemon[774]: ts=2026-09-22T16:59:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=232 client # [ 87.282617] nix-daemon[1151]: reaped child process 1153, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.68 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.81 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.18 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 88.264280] nix-daemon[1151]: accepted connection from pid 1234, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 88.339746] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:44240 client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 88.348574] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker2 # [ 88.359477] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 88.376933] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 88.552167] nix-daemon[859]: accepted connection from pid 1137, user nix-grpc-daemon (trusted) worker2 # [ 88.574985] nix-daemon[859]: reaped child process 1140, status = succeeded lb # [ 88.603715] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 88.585803] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.208Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 88.587399] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.208Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 88.634829] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker2 # [ 88.618036] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.241Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.651624] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 88.660783] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=27 count=1 lb # [ 88.669080] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls worker2 # [ 88.645679] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.269Z level=INFO msg="Upload complete. (248ms)" lb # [ 88.678573] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 88.684265] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=27 worker2 # [ 88.656313] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=120 lb # [ 88.688067] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker2 # [ 88.671932] nix-daemon[859]: reaped child process 1135, status = succeeded worker2 # [ 88.676457] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker2 # [ 88.682115] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 88.723711] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 88.739034] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 client # copying 2 paths to 'lb:50051'... worker2 # [ 88.756738] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 88.774025] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 88.932114] nix-daemon[859]: accepted connection from pid 1146, user nix-grpc-daemon (trusted) worker2 # [ 88.953654] nix-daemon[859]: reaped child process 1149, status = succeeded lb # [ 88.984871] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 88.994359] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 88.978381] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 88.981477] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker2 # [ 88.984162] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 89.033669] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 89.051516] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 89.065726] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 89.075598] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign worker2 # [ 89.048115] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.671Z level=INFO msg="Uploading 3 narinfos" lb # [ 89.085727] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=28 count=1 lb # [ 89.091120] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 89.098873] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=29 count=2 lb # [ 89.103655] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 89.115442] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 89.124617] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 89.133411] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=28 lb # [ 89.136671] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete worker2 # [ 89.107317] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.731Z level=INFO msg="Upload complete. (314ms)" lb # [ 89.139193] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=29 lb # [ 89.141719] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 89.146013] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker2 # [ 89.120335] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 89.132379] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 89.147751] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 89.155168] nix-daemon[859]: reaped child process 1144, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker2 # [ 89.170665] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 89.807304] nix-daemon[859]: reaped child process 1153, status = succeeded worker2 # [ 89.905714] nix-daemon[859]: accepted connection from pid 1169, user nix-grpc-daemon (trusted) lb # [ 89.942957] envoy[919]: conn peer=[2001:db8:1::1]:57806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1926 ms=1617 lb # [ 89.948091] envoy[919]: conn peer=[2001:db8:1::1]:57822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=68 worker2 # [ 89.923655] nix-daemon[859]: reaped child process 1172, status = succeeded lb # [ 89.952716] envoy[919]: conn peer=[2001:db8:1::1]:57832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=2155 ms=1821 lb # [ 89.961624] envoy[919]: conn peer=[2001:db8:1::1]:57838 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=69 lb # [ 89.966968] envoy[919]: conn peer=[2001:db8:1::1]:57840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4369 tx=2407 ms=2262 worker2 # [ 89.946447] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.568Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 89.973301] envoy[919]: conn peer=[2001:db8:1::1]:57844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=168 lb # [ 89.984987] niks3-server[964]: 2026/09/22 16:59:54 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 90.006650] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 90.014982] niks3-server[964]: 2026/09/22 16:59:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 89.990904] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.614Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.022121] niks3-server[964]: 2026/09/22 16:59:54 INFO Signed narinfos id=30 count=1 lb # [ 90.027631] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 90.040048] niks3-server[964]: 2026/09/22 16:59:54 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 90.045793] niks3-server[964]: 2026/09/22 16:59:54 INFO Completed upload id=30 worker2 # [ 90.015809] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.639Z level=INFO msg="Upload complete. (230ms)" lb # [ 90.048890] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 90.029143] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 90.043694] nix-daemon[859]: reaped child process 1151, status = succeeded client # copying 1 paths... worker1 # [ 90.110051] nix-grpc-daemon[767]: ts=2026-09-22T16:59:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 90.118230] nix-grpc-daemon[767]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 90.133323] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 90.140572] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=120 client # [ 90.300915] nix-daemon[1151]: reaped child process 1237, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.23 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 90.472930] nix-daemon[1151]: accepted connection from pid 1256, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 90.577174] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:53218 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 90.585446] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 90.593021] nix-grpc-daemon[844]: time=2026-09-22T16:59:55.207Z level=INFO msg="All 1 paths already cached" worker2 # [ 90.574916] nix-grpc-daemon[774]: ts=2026-09-22T16:59:55Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 90.608155] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker2 # [ 90.581819] nix-grpc-daemon[774]: ts=2026-09-22T16:59:55Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 90.623069] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 90.641867] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 90.657381] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 90.679080] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5) worker1 # [ 90.720793] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 2/5) worker1 # [ 90.868891] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 292 ms (attempt 3/5) worker1 # [ 91.164632] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 523 ms (attempt 4/5) worker1 # [ 91.688856] nix-grpc-daemon[767]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 91.698518] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 91.705679] nix-grpc-daemon[767]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 91.878050] nix-daemon[831]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker1 # [ 91.899004] nix-daemon[831]: reaped child process 1196, status = succeeded lb # [ 91.903166] niks3-server[964]: 2026/09/22 16:59:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 91.914075] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.529Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 91.915836] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.530Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 91.931057] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 91.944081] niks3-server[964]: 2026/09/22 16:59:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker1 # [ 91.952647] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.569Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.957505] niks3-server[964]: 2026/09/22 16:59:56 INFO Signed narinfos id=31 count=1 lb # [ 91.965167] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 91.979801] niks3-server[964]: 2026/09/22 16:59:56 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete worker1 # [ 91.986591] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.603Z level=INFO msg="Upload complete. (267ms)" lb # [ 91.988241] niks3-server[964]: 2026/09/22 16:59:56 INFO Completed upload id=31 lb # [ 91.993041] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 91.996369] nix-grpc-daemon[767]: ts=2026-09-22T16:59:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 paths=1 nar_bytes_in=568 worker1 # [ 92.011163] nix-grpc-daemon[767]: ts=2026-09-22T16:59:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 92.025872] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) worker1 # [ 92.031519] nix-daemon[831]: reaped child process 1186, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 92.048192] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 92.914770] nix-daemon[831]: reaped child process 1200, status = succeeded worker1 # [ 93.051863] nix-daemon[831]: accepted connection from pid 1216, user nix-grpc-daemon (trusted) worker1 # [ 93.069845] nix-daemon[831]: reaped child process 1219, status = succeeded lb # [ 93.073768] niks3-server[964]: 2026/09/22 16:59:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.084443] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.700Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 93.099558] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 93.115529] niks3-server[964]: 2026/09/22 16:59:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker1 # [ 93.125172] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.741Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.127649] niks3-server[964]: 2026/09/22 16:59:57 INFO Signed narinfos id=32 count=1 lb # [ 93.133514] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 93.144911] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.761Z level=INFO msg="Upload complete. (284ms)" lb # [ 93.144644] niks3-server[964]: 2026/09/22 16:59:57 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 93.147800] niks3-server[964]: 2026/09/22 16:59:57 INFO Completed upload id=32 lb # [ 93.153702] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 93.157052] nix-grpc-daemon[767]: ts=2026-09-22T16:59:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker1 # [ 93.173871] nix-daemon[831]: reaped child process 1198, status = succeeded client # copying 1 paths... worker1 # [ 93.202419] nix-grpc-daemon[767]: ts=2026-09-22T16:59:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 93.206751] nix-grpc-daemon[774]: ts=2026-09-22T16:59:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 93.220085] nix-grpc-daemon[774]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... worker2 # [ 93.237821] nix-grpc-daemon[774]: ts=2026-09-22T16:59:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=120 client # [ 93.439455] nix-daemon[1151]: reaped child process 1259, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.14 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 93.609907] nix-daemon[1151]: accepted connection from pid 1277, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 93.715602] nix-grpc-daemon[767]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:53218 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 93.690169] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 93.701017] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.324Z level=INFO msg="All 1 paths already cached" worker2 # [ 93.718045] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 93.727138] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 93.735776] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 93.743283] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 93.753490] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.377Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 93.763750] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 93.779265] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 93.926459] nix-daemon[859]: accepted connection from pid 1181, user nix-grpc-daemon (trusted) worker2 # [ 93.946721] nix-daemon[859]: reaped child process 1184, status = succeeded lb # [ 93.978916] niks3-server[964]: 2026/09/22 16:59:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 93.959671] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.583Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 93.961846] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.583Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 94.011291] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls worker2 # [ 93.993977] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.617Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.024439] niks3-server[964]: 2026/09/22 16:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 94.035047] niks3-server[964]: 2026/09/22 16:59:58 INFO Signed narinfos id=33 count=1 lb # [ 94.042778] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 94.021239] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.645Z level=INFO msg="Upload complete. (226ms)" lb # [ 94.052957] niks3-server[964]: 2026/09/22 16:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 94.058029] niks3-server[964]: 2026/09/22 16:59:58 INFO Completed upload id=33 lb # [ 94.061760] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 94.032267] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 94.044543] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 94.052894] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker2 # [ 94.059145] nix-daemon[859]: reaped child process 1179, status = succeeded worker2 # [ 94.071921] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 95.183906] nix-daemon[859]: reaped child process 1188, status = succeeded worker2 # [ 95.256125] nix-daemon[859]: accepted connection from pid 1204, user nix-grpc-daemon (trusted) worker2 # [ 95.279327] nix-daemon[859]: reaped child process 1207, status = succeeded lb # [ 95.309993] niks3-server[964]: 2026/09/22 16:59:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.292510] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.914Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 95.342375] niks3-server[964]: 2026/09/22 16:59:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker2 # [ 95.322327] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.946Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.356659] niks3-server[964]: 2026/09/22 16:59:59 INFO Signed narinfos id=34 count=1 lb # [ 95.364729] niks3-server[964]: 2026/09/22 16:59:59 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 95.373312] niks3-server[964]: 2026/09/22 16:59:59 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 95.354682] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.978Z level=INFO msg="Upload complete. (269ms)" lb # [ 95.383751] niks3-server[964]: 2026/09/22 16:59:59 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 95.390682] niks3-server[964]: 2026/09/22 16:59:59 INFO Completed upload id=34 lb # [ 95.394432] niks3-server[964]: 2026/09/22 17:00:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 95.369054] nix-grpc-daemon[774]: ts=2026-09-22T16:59:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 client # copying 1 paths... worker2 # [ 95.386400] nix-daemon[859]: reaped child process 1186, status = succeeded worker1 # [ 95.415483] nix-grpc-daemon[767]: ts=2026-09-22T17:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 95.428756] nix-grpc-daemon[767]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 95.442492] nix-grpc-daemon[774]: ts=2026-09-22T17:00:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 worker2 # [ 95.449735] nix-grpc-daemon[774]: ts=2026-09-22T17:00:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=120 client # [ 95.605432] nix-daemon[1151]: reaped child process 1280, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.16 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 8.18 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds) worker2 # [ 96.550764] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 97.248651] nix-grpc-daemon[774]: ts=2026-09-22T17:00:01Z level=info event=signal_exit lb # [ 97.366518] niks3-server[964]: 2026/09/22 17:00:01 INFO lead: released remote=192.168.1.4:37628 worker1 # [ 98.370414] nix-grpc-daemon[767]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 98.347389] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting worker2 # [ 98.351638] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 98.355767] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44234 worker2 # [ 98.361093] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 98.365617] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44234 worker2 # [ 98.372491] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 98.376823] nix-grpc-daemon[774]: E0000 00:00:1790096402.995925 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker2 # [ 98.403030] nix-daemon[859]: reaped child process 861, status = succeeded worker2 # [ 98.406338] nix-grpc-daemon[774]: E0000 00:00:1790096403.030092 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 98.570601] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 98.576158] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 98.565887] nix-grpc-daemon[774]: ts=2026-09-22T17:00:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 98.776820] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 98.781943] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 98.982556] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 98.987634] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 99.090550] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 99.097103] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 99.099626] systemd[1]: nix-grpc-daemon.service: Consumed 2.943s CPU time over 1min 19.062s wall clock time, 65.7M memory peak, 93.2K incoming IP traffic, 224.6K outgoing IP traffic. worker2 # [ 99.118816] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 99.131024] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 99.133011] systemd[1]: nix-grpc-daemon.socket: Consumed 786us CPU time over 1min 29.380s wall clock time, 256K memory peak, 114.2K incoming IP traffic, 95.6K outgoing IP traffic. worker1 # [ 99.188525] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.193529] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 99.395963] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.399943] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 99.453757] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53218 worker1 # [ 99.463048] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53218 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 99.557995] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 99.596482] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.602835] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 99.668692] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 99.802509] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.807728] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 99.945492] envoy[919]: conn peer=[2001:db8:1::1]:57876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=32 lb # [ 99.950512] envoy[919]: conn peer=[2001:db8:1::1]:57860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4238 tx=2056 ms=1824 lb # [ 99.955431] envoy[919]: conn peer=[2001:db8:1::1]:57888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=2747 lb # [ 99.961090] envoy[919]: conn peer=[2001:db8:1::1]:42726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=94 lb # [ 99.965751] envoy[919]: conn peer=[2001:db8:1::1]:42740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2101 ms=1780 lb # [ 99.970516] envoy[919]: conn peer=[2001:db8:1::1]:42748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=28 lb # [ 99.976065] envoy[919]: conn peer=[2001:db8:1::4]:37094 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=51196 tx=2969 ms=78265 lb # [ 99.980978] envoy[919]: conn peer=[2001:db8:1::1]:42754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=28 worker1 # [ 100.008467] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 100.013687] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 100.087987] systemd[1]: Starting scheduler-token.service... worker1 # [ 100.203009] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=signal_exit worker2 # [ 100.175547] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 100.184800] systemd[1]: Finished scheduler-token.service. worker1 # [ 100.214074] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 100.187032] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 91ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker1 # [ 100.219170] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 100.211939] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 100.335191] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 100.337589] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 100.341514] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 100.347146] nix-grpc-daemon[1231]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 100.367445] nix-grpc-daemon[1231]: E0000 00:00:1790096404.970522 1231 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 100.396447] niks3-server[964]: 2026/09/22 17:00:05 INFO lead: acquired remote=192.168.1.4:54366 worker1 # [ 100.421074] nix-grpc-daemon[767]: ts=2026-09-22T17:00:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 100.394184] nix-grpc-daemon[1231]: E0000 00:00:1790096404.974351 1231 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 100.402430] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 100.404537] nix-grpc-daemon[1231]: E0000 00:00:1790096404.986307 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 100.407052] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=scheduler_take_over worker2 # [ 100.408205] nix-grpc-daemon[1231]: E0000 00:00:1790096405.006854 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 100.413193] nix-grpc-daemon[1231]: E0000 00:00:1790096405.007000 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 100.418002] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:60080 worker2 # [ 100.420351] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 100.422739] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 100.423919] nix-grpc-daemon[1231]: E0000 00:00:1790096405.025599 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 100.428270] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 100.429598] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 100.430839] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:60080 worker2 # [ 100.432578] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 101.379386] nix-grpc-daemon[767]: ts=2026-09-22T17:00:05Z level=info event=scheduler_restarting worker2 # [ 101.362658] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60080 worker2 # [ 101.362934] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 101.424666] nix-grpc-daemon[767]: ts=2026-09-22T17:00:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 101.442841] nix-daemon[831]: reaped child process 834, status = succeeded worker1 # [ 102.375779] nix-grpc-daemon[767]: E0000 00:00:1790096406.992214 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 102.382322] nix-grpc-daemon[767]: E0000 00:00:1790096406.992445 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 103.060981] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 103.068534] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 103.071117] systemd[1]: nix-grpc-daemon.service: Consumed 3.710s CPU time over 1min 23.117s wall clock time, 69M memory peak, 97.4K incoming IP traffic, 177.6K outgoing IP traffic. worker1 # [ 103.088513] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 103.090822] systemd[1]: Closed Nix worker-protocol over gRPC. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 103.102236] systemd[1]: nix-grpc-daemon.socket: Consumed 675us CPU time over 1min 32.814s wall clock time, 256K memory peak, 102.4K incoming IP traffic, 91.2K outgoing IP traffic. lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 103.366619] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 103.382069] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 103.411461] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5) worker2 # [ 103.444604] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5) worker2 # [ 103.467398] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 359 ms (attempt 3/5) worker2 # [ 103.835864] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 619 ms (attempt 4/5) worker2 # [ 104.458401] nix-grpc-daemon[1231]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 104.464098] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 104.488832] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 104.507610] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 104.538638] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=info event=niks3_push_started pid=1291 worker2 # [ 104.696510] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.319Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 104.804622] nix-daemon[859]: accepted connection from pid 1304, user nix-grpc-daemon (trusted) worker2 # [ 104.822954] nix-daemon[859]: reaped child process 1307, status = succeeded lb # [ 104.857213] niks3-server[964]: 2026/09/22 17:00:09 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 104.869170] niks3-server[964]: 2026/09/22 17:00:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 104.852094] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 104.853983] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 104.855857] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 104.944280] niks3-server[964]: 2026/09/22 17:00:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker2 # [ 104.931972] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.553Z level=INFO msg="Uploading 3 narinfos" lb # [ 104.958026] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 104.968129] niks3-server[964]: 2026/09/22 17:00:09 INFO Signed narinfos id=35 count=1 lb # [ 104.974540] niks3-server[964]: 2026/09/22 17:00:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 104.981110] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 104.983520] niks3-server[964]: 2026/09/22 17:00:09 INFO Signed narinfos id=36 count=2 lb # [ 104.989819] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 104.999151] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 105.006348] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 105.014697] niks3-server[964]: 2026/09/22 17:00:09 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 105.024073] niks3-server[964]: 2026/09/22 17:00:09 INFO Completed upload id=35 lb # [ 105.026223] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 105.000908] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.624Z level=INFO msg="Upload complete. (301ms)" lb # [ 105.032284] niks3-server[964]: 2026/09/22 17:00:09 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 105.037529] niks3-server[964]: 2026/09/22 17:00:09 INFO Completed upload id=36 lb # [ 105.040591] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 105.011930] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 105.029322] nix-daemon[859]: reaped child process 1289, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.90 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 105.181094] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 105.188602] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 105.341869] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 105.344772] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 105.348869] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 105.354506] nix-grpc-daemon[1243]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 105.355944] nix-grpc-daemon[1243]: E0000 00:00:1790096409.970441 1243 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 105.401589] nix-grpc-daemon[1243]: E0000 00:00:1790096409.976671 1243 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.404502] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 105.406276] nix-grpc-daemon[1243]: E0000 00:00:1790096409.983292 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.408686] nix-grpc-daemon[1243]: E0000 00:00:1790096410.000823 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.410885] nix-grpc-daemon[1243]: E0000 00:00:1790096410.001905 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.413317] nix-grpc-daemon[1243]: E0000 00:00:1790096410.017378 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.418627] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 105.421861] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.392949] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:60080 worker2 # [ 105.399333] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.97 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 106.664795] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=info event=scheduler_lock_lost status=200 lb # [ 106.663616] niks3-server[964]: 2026/09/22 17:00:11 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 106.671223] systemd[1]: Stopping niks3 server... lb # [ 106.672824] niks3-server[964]: 2026/09/22 17:00:11 INFO lead: released remote=192.168.1.4:54366 worker2 # [ 106.644569] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:11Z level=info event=scheduler_lock_lost status=200 lb # [ 106.694130] systemd[1]: niks3.service: Deactivated successfully. lb # [ 106.695894] systemd[1]: Stopped niks3 server. lb # [ 106.697619] systemd[1]: niks3.service: Consumed 670ms CPU time over 1min 24.573s wall clock time, 27.8M memory peak, 100.3K incoming IP traffic, 143.7K outgoing IP traffic. lb # [ 106.708427] systemd[1]: Starting niks3 server... lb # [ 106.767416] niks3-server[1105]: 2026/09/22 17:00:11 goose: no migrations to run. current version: 20260920000000 lb # [ 106.772538] niks3-server[1105]: 2026/09/22 17:00:11 OK 1_commit_pending_closure.sql (6.63ms) lb # [ 106.776260] niks3-server[1105]: 2026/09/22 17:00:11 OK 2_object_stats_trigger.sql (3.37ms) lb # [ 106.777438] niks3-server[1105]: 2026/09/22 17:00:11 goose: up to current file version: 2 lb # [ 106.779092] niks3-server[1105]: 2026/09/22 17:00:11 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 106.790449] niks3-server[1105]: 2026/09/22 17:00:11 INFO Read proxy enabled — serving cache objects from S3 lb # [ 106.794883] niks3-server[1105]: 2026/09/22 17:00:11 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 106.797410] niks3-server[1105]: 2026/09/22 17:00:11 INFO systemd watchdog enabled interval=15s lb # [ 106.799679] niks3-server[1105]: 2026/09/22 17:00:11 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 106.802264] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 107.294233] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 107.309320] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 107.332673] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 107.349009] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5) worker1 # [ 107.408854] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5) worker1 # [ 107.483685] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5) lb # [ 107.677123] niks3-server[1105]: 2026/09/22 17:00:12 INFO lead: acquired remote=192.168.1.4:48806 worker1 # [ 107.681551] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 206 ms (attempt 4/5) worker1 # [ 107.890963] nix-grpc-daemon[1243]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 107.896668] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 107.916265] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 107.916580] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 107.960474] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 107.978114] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34430 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 107.992675] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 108.008098] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 108.174276] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted) worker2 # [ 108.191519] nix-daemon[859]: reaped child process 1329, status = succeeded lb # [ 108.227362] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.213364] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 108.215095] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.836Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 108.266981] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 108.300759] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 108.313127] niks3-server[1105]: 2026/09/22 17:00:12 INFO Signed narinfos id=37 count=1 worker2 # [ 108.288111] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.911Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.325890] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 108.334911] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 108.340083] niks3-server[1105]: 2026/09/22 17:00:12 INFO Completed upload id=37 worker2 # [ 108.311781] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.935Z level=INFO msg="Upload complete. (285ms)" lb # [ 108.344534] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 108.322708] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 108.336341] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 108.350605] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 108.356700] nix-daemon[859]: reaped child process 1324, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker2 # [ 108.369172] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 109.879817] nix-daemon[859]: reaped child process 1333, status = succeeded lb # [ 109.945509] envoy[919]: conn peer=192.168.1.3:39116 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11455 tx=3263 ms=80253 lb # [ 109.950565] envoy[919]: conn peer=[2001:db8:1::1]:47384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=304 ms=1663 worker2 # [ 109.951369] nix-daemon[859]: accepted connection from pid 1349, user nix-grpc-daemon (trusted) worker2 # [ 109.968852] nix-daemon[859]: reaped child process 1352, status = succeeded lb # [ 110.003983] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 109.988482] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 109.990392] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.610Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 110.051885] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst worker2 # [ 110.039404] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.662Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.070076] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 110.081144] niks3-server[1105]: 2026/09/22 17:00:14 INFO Signed narinfos id=38 count=1 lb # [ 110.086667] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 110.100991] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 110.108206] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 110.112438] niks3-server[1105]: 2026/09/22 17:00:14 INFO Completed upload id=38 worker2 # [ 110.071045] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.694Z level=INFO msg="Upload complete. (288ms)" lb # [ 110.114860] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 110.089448] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 110.112419] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 110.118371] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 110.123161] nix-daemon[859]: reaped child process 1331, status = succeeded worker2 # [ 110.156219] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 110.172271] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 110.187265] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.811Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 110.198094] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 110.211910] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 110.371241] nix-daemon[859]: accepted connection from pid 1358, user nix-grpc-daemon (trusted) worker2 # [ 110.390200] nix-daemon[859]: reaped child process 1361, status = succeeded lb # [ 110.424178] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.406791] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.030Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 110.408554] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.030Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 110.459610] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls worker2 # [ 110.441698] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.065Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.473429] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign lb # [ 110.483344] niks3-server[1105]: 2026/09/22 17:00:15 INFO Signed narinfos id=39 count=1 lb # [ 110.489517] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker2 # [ 110.473659] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.097Z level=INFO msg="Upload complete. (245ms)" lb # [ 110.502591] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 110.509962] niks3-server[1105]: 2026/09/22 17:00:15 INFO Completed upload id=39 worker2 # [ 110.484327] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=656 lb # [ 110.515029] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 110.499635] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 110.507047] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 110.512646] nix-daemon[859]: reaped child process 1356, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 110.527253] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 111.390644] nix-daemon[859]: reaped child process 1365, status = succeeded worker2 # [ 111.531292] nix-daemon[859]: accepted connection from pid 1381, user nix-grpc-daemon (trusted) worker2 # [ 111.553244] nix-daemon[859]: reaped child process 1384, status = succeeded lb # [ 111.585643] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.567989] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 111.570629] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.190Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 111.620151] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker2 # [ 111.597255] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.220Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.628058] niks3-server[1105]: 2026/09/22 17:00:16 INFO Signed narinfos id=40 count=1 lb # [ 111.634811] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 111.642626] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 111.653609] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 111.660307] niks3-server[1105]: 2026/09/22 17:00:16 INFO Completed upload id=40 worker2 # [ 111.630218] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.254Z level=INFO msg="Upload complete. (291ms)" lb # [ 111.662847] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 111.669354] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 111.648623] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 111.664403] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 111.669121] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 111.673112] nix-daemon[859]: reaped child process 1363, status = succeeded worker2 # [ 111.691780] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 111.706966] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 111.723045] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.346Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 111.735323] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 111.749684] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 111.901808] nix-daemon[859]: accepted connection from pid 1390, user nix-grpc-daemon (trusted) worker2 # [ 111.919147] nix-daemon[859]: reaped child process 1393, status = succeeded lb # [ 111.953786] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.935866] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.559Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 111.937792] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.559Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" worker2 # [ 111.969654] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.593Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.988204] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 112.003076] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 112.015684] niks3-server[1105]: 2026/09/22 17:00:16 INFO Signed narinfos id=41 count=1 lb # [ 112.024926] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 112.031864] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 112.039211] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo lb # [ 112.043811] niks3-server[1105]: 2026/09/22 17:00:16 INFO Completed upload id=41 worker2 # [ 112.015609] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.639Z level=INFO msg="Upload complete. (250ms)" worker2 # [ 112.028378] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 112.042949] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 worker2 # [ 112.053850] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 112.062149] nix-daemon[859]: reaped child process 1388, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker2 # [ 112.071207] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 113.302291] nix-daemon[859]: reaped child process 1397, status = succeeded worker2 # [ 113.416585] nix-daemon[859]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) worker2 # [ 113.435180] nix-daemon[859]: reaped child process 1416, status = succeeded lb # [ 113.470708] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 113.455370] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 113.457110] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.078Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 113.509988] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign worker2 # [ 113.490816] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.114Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.526667] niks3-server[1105]: 2026/09/22 17:00:18 INFO Signed narinfos id=42 count=1 lb # [ 113.534625] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 113.543081] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 113.548234] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 113.551049] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 113.560113] niks3-server[1105]: 2026/09/22 17:00:18 INFO Completed upload id=42 worker2 # [ 113.531881] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.155Z level=INFO msg="Upload complete. (246ms)" lb # [ 113.563591] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 113.545109] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 113.564640] nix-daemon[859]: reaped child process 1395, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.54 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 7.26 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 114.019188] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 114.363317] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:18Z level=info event=signal_exit lb # [ 114.685337] niks3-server[1105]: 2026/09/22 17:00:19 INFO lead: released remote=192.168.1.4:48806 worker1 # [ 115.686823] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 115.662681] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting worker2 # [ 115.666805] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 115.671581] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60080 worker2 # [ 115.676712] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 115.681859] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60080 worker2 # [ 115.688342] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 115.713167] nix-daemon[859]: reaped child process 1283, status = succeeded worker1 # [ 115.886652] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.892325] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.877621] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 116.092829] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.099300] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.299180] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.304462] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 116.412434] nix-grpc-daemon[1231]: E0000 00:00:1790096421.036236 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker2 # [ 116.420121] nix-grpc-daemon[1231]: E0000 00:00:1790096421.043574 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 116.505272] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.510479] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.713068] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.717192] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.913614] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.918386] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 116.936009] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 116.942974] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 116.949712] systemd[1]: nix-grpc-daemon.service: Consumed 1.410s CPU time over 16.741s wall clock time, 39.1M memory peak, 46.1K incoming IP traffic, 78.2K outgoing IP traffic. worker2 # [ 116.967524] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 116.973187] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 116.974951] systemd[1]: nix-grpc-daemon.socket: Consumed 833us CPU time over 17.416s wall clock time, 256K memory peak, 38.2K incoming IP traffic, 28.4K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 117.119127] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.124535] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.325315] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.330488] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.531040] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.536402] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.671259] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_take_over lb # [ 117.670870] niks3-server[1105]: 2026/09/22 17:00:22 INFO lead: acquired remote=192.168.1.3:59016 worker1 # [ 117.738865] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.743028] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.939615] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.944789] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.147649] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 118.159212] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38876 worker1 # [ 118.168605] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.18 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 118.456384] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 118.466162] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 118.484999] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 118.491174] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 118.529521] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 118.544792] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 118.561105] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 118.577146] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 118.602941] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=info event=niks3_push_started pid=1322 worker1 # [ 118.751565] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.367Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 118.858817] nix-daemon[831]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker1 # [ 118.875215] nix-daemon[831]: reaped child process 1337, status = succeeded lb # [ 118.882240] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 118.893725] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 118.895198] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.509Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 118.920426] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 118.930504] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.546Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.932153] niks3-server[1105]: 2026/09/22 17:00:23 INFO Signed narinfos id=43 count=1 lb # [ 118.939780] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 118.954696] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 118.964401] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 118.971802] niks3-server[1105]: 2026/09/22 17:00:23 INFO Completed upload id=43 worker1 # [ 118.971736] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.588Z level=INFO msg="Upload complete. (216ms)" lb # [ 118.977264] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 118.985052] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 118.995413] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 119.004681] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 119.007470] nix-daemon[831]: reaped child process 1320, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker1 # [ 119.020886] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 119.654853] nix-daemon[831]: reaped child process 1341, status = succeeded worker1 # [ 119.747077] nix-daemon[831]: accepted connection from pid 1357, user nix-grpc-daemon (trusted) worker1 # [ 119.765094] nix-daemon[831]: reaped child process 1360, status = succeeded lb # [ 119.770029] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.784432] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 119.785884] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.399Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 119.805703] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 119.826779] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker1 # [ 119.835928] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.452Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.836616] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 119.839539] niks3-server[1105]: 2026/09/22 17:00:24 INFO Signed narinfos id=44 count=1 lb # [ 119.847654] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 119.855463] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete worker1 # [ 119.860200] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.476Z level=INFO msg="Upload complete. (229ms)" lb # [ 119.860800] niks3-server[1105]: 2026/09/22 17:00:24 INFO Completed upload id=44 lb # [ 119.863118] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 119.878082] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 119.894288] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 119.898445] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 119.902695] nix-daemon[831]: reaped child process 1339, status = succeeded worker1 # [ 119.925765] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 119.956965] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 lb # [ 119.951156] envoy[919]: conn peer=[2001:db8:1::1]:47392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9736 tx=4453 ms=6295 lb # [ 119.962974] envoy[919]: conn peer=[2001:db8:1::4]:35734 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1691 tx=425 ms=15453 worker1 # [ 119.978246] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.594Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 119.990523] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 120.005686] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 120.169419] nix-daemon[831]: accepted connection from pid 1366, user nix-grpc-daemon (trusted) worker1 # [ 120.190579] nix-daemon[831]: reaped child process 1369, status = succeeded lb # [ 120.187667] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.198486] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.814Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 120.199963] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.814Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 120.217140] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 120.226359] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.842Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.232522] niks3-server[1105]: 2026/09/22 17:00:24 INFO Signed narinfos id=45 count=1 lb # [ 120.235800] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 120.258281] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls worker1 # [ 120.263332] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.880Z level=INFO msg="Upload complete. (242ms)" lb # [ 120.265118] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 120.269378] niks3-server[1105]: 2026/09/22 17:00:24 INFO Completed upload id=45 lb # [ 120.272385] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 120.272336] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 120.292166] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 120.301969] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 120.305019] nix-daemon[831]: reaped child process 1364, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 120.318505] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 120.977841] nix-daemon[831]: reaped child process 1373, status = succeeded worker1 # [ 121.088335] nix-daemon[831]: accepted connection from pid 1389, user nix-grpc-daemon (trusted) worker1 # [ 121.109639] nix-daemon[831]: reaped child process 1392, status = succeeded lb # [ 121.112663] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.124543] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.739Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 121.126142] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.740Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 121.154505] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 121.168568] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.784Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.168979] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 121.178682] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 121.187883] niks3-server[1105]: 2026/09/22 17:00:25 INFO Signed narinfos id=46 count=1 lb # [ 121.196749] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls worker1 # [ 121.198080] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.814Z level=INFO msg="Upload complete. (269ms)" lb # [ 121.201218] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 121.208645] niks3-server[1105]: 2026/09/22 17:00:25 INFO Completed upload id=46 lb # [ 121.211823] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 121.216546] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 121.234773] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 121.241569] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 121.246690] nix-daemon[831]: reaped child process 1371, status = succeeded worker1 # [ 121.281730] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 121.296948] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 121.311004] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.927Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 121.322052] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 121.336123] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 121.491144] nix-daemon[831]: accepted connection from pid 1398, user nix-grpc-daemon (trusted) worker1 # [ 121.510163] nix-daemon[831]: reaped child process 1401, status = succeeded lb # [ 121.511825] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.522103] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.138Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 121.523635] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.138Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 121.542843] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls worker1 # [ 121.554217] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.170Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.553234] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 121.565212] niks3-server[1105]: 2026/09/22 17:00:26 INFO Signed narinfos id=47 count=1 lb # [ 121.571985] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 121.587764] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker1 # [ 121.594474] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.211Z level=INFO msg="Upload complete. (242ms)" lb # [ 121.594574] niks3-server[1105]: 2026/09/22 17:00:26 INFO Completed upload id=47 lb # [ 121.598607] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 121.604397] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 121.623185] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 121.630581] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 121.636903] nix-daemon[831]: reaped child process 1396, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 121.653659] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 122.922634] nix-daemon[831]: reaped child process 1405, status = succeeded worker1 # [ 123.052232] nix-daemon[831]: accepted connection from pid 1421, user nix-grpc-daemon (trusted) worker1 # [ 123.070449] nix-daemon[831]: reaped child process 1424, status = succeeded lb # [ 123.078961] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 123.091788] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.706Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 123.093805] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.706Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 123.118045] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign worker1 # [ 123.130886] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.747Z level=INFO msg="Uploading 1 narinfos" lb # [ 123.132813] niks3-server[1105]: 2026/09/22 17:00:27 INFO Signed narinfos id=48 count=1 lb # [ 123.143658] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst lb # [ 123.154492] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 123.163947] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 123.169763] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.786Z level=INFO msg="Upload complete. (302ms)" lb # [ 123.172115] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 123.178704] niks3-server[1105]: 2026/09/22 17:00:27 INFO Completed upload id=48 lb # [ 123.182424] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 123.181265] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 123.202751] nix-daemon[831]: reaped child process 1403, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.00 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds) worker2 # [ 123.479269] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 123.489485] systemd[1]: Starting scheduler-token.service... worker2 # [ 123.572383] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 123.576895] systemd[1]: Finished scheduler-token.service. worker2 # [ 123.578593] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 123.595862] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 123.715234] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 123.717568] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 123.721291] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 123.727813] nix-grpc-daemon[1437]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 123.737798] nix-grpc-daemon[1437]: E0000 00:00:1790096428.349587 1437 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 123.756121] nix-grpc-daemon[1437]: E0000 00:00:1790096428.351690 1437 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 123.758449] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 123.777231] nix-grpc-daemon[1437]: E0000 00:00:1790096428.366144 1476 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 123.780208] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 123.781192] nix-grpc-daemon[1437]: E0000 00:00:1790096428.385439 1476 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 123.783435] nix-grpc-daemon[1437]: E0000 00:00:1790096428.385648 1476 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 123.789082] nix-grpc-daemon[1437]: E0000 00:00:1790096428.400404 1476 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 123.793303] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 123.837881] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 123.838253] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38876 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 123.847610] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.28 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 124.194636] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 29dde1ba2f3f43909d28834975bbf36a client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker2 # [ 124.309334] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38088 worker2 # [ 124.322188] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted) client # [ 124.411474] nix[1404]: don't know how to build these paths: client # [ 124.413033] nix[1404]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 124.379727] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 124.385414] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 124.425356] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 124.441624] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40150 client # [ 124.499405] nix[1404]: copying 1 paths to 'lb:50051'... worker1 # [ 124.456113] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 client # [ 124.501922] nix[1404]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 124.470993] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 124.632267] nix-daemon[831]: accepted connection from pid 1437, user nix-grpc-daemon (trusted) worker1 # [ 124.652508] nix-daemon[831]: reaped child process 1440, status = succeeded lb # [ 124.655154] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 124.665172] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.281Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 124.666718] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.281Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 124.683967] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker1 # [ 124.692997] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.309Z level=INFO msg="Uploading 1 narinfos" lb # [ 124.696071] niks3-server[1105]: 2026/09/22 17:00:29 INFO Signed narinfos id=49 count=1 lb # [ 124.704912] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 124.714241] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst worker1 # [ 124.729016] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.345Z level=INFO msg="Upload complete. (240ms)" lb # [ 124.728488] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 124.736616] niks3-server[1105]: 2026/09/22 17:00:29 INFO Completed upload id=49 worker1 # [ 124.737216] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 124.738700] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 124.748328] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 worker1 # [ 124.760412] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) worker1 # [ 124.765840] nix-daemon[831]: reaped child process 1435, status = succeeded client # [ 124.818693] nix[1404]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 124.778922] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 126.942097] systemd-journald[262]: Received client request to rotate journal, rotating. worker2 # [ 126.961564] systemd-journald[262]: Vacuuming done, freed 0B of archived journals from /var/log/journal/695a74b3b1d743e7954e01a7d8cef4b5. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 127.039741] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 127.373320] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:31Z level=info event=draining lb # [ 129.946561] envoy[919]: conn peer=[2001:db8:1::1]:36190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9563 tx=4394 ms=4752 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 132.488095] nix-daemon[831]: reaped child process 1444, status = succeeded worker1 # [ 132.558619] nix-daemon[831]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) worker1 # [ 132.576461] nix-daemon[831]: reaped child process 1488, status = succeeded lb # [ 132.581646] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 132.595007] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 132.596732] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.209Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 132.639704] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 132.649965] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.266Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.654408] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 132.667779] niks3-server[1105]: 2026/09/22 17:00:37 INFO Signed narinfos id=50 count=1 lb # [ 132.671974] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 132.683895] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 132.693711] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo lb # [ 132.699484] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 132.704769] niks3-server[1105]: 2026/09/22 17:00:37 INFO Completed upload id=50 worker1 # [ 132.706128] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.322Z level=INFO msg="Upload complete. (309ms)" worker1 # [ 132.721187] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 132.739970] nix-daemon[831]: reaped child process 1442, status = succeeded client # [ 132.793927] systemd[1]: rs.service: Deactivated successfully. client # [ 132.799226] systemd[1]: rs.service: Consumed 178ms CPU time over 8.600s wall clock time, 39.3M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 133.375643] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:37Z level=info event=signal_exit lb # [ 133.678413] niks3-server[1105]: 2026/09/22 17:00:38 INFO lead: released remote=192.168.1.3:59016 lb # [ 133.769934] niks3-server[1105]: 2026/09/22 17:00:38 INFO lead: acquired remote=192.168.1.4:41600 worker2 # [ 133.746873] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:38Z level=info event=scheduler_take_over worker2 # [ 134.646819] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 134.681634] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting worker1 # [ 134.682070] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 134.682198] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:38876 worker1 # [ 134.682311] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 134.682423] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:38876 worker1 # [ 134.682591] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 134.730287] nix-daemon[831]: reaped child process 1295, status = succeeded worker2 # [ 134.849591] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 134.880601] nix-grpc-daemon[1243]: E0000 00:00:1790096439.497209 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 134.906693] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 135.115442] nix-grpc-daemon[1243]: E0000 00:00:1790096439.732008 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 135.132502] nix-grpc-daemon[1243]: E0000 00:00:1790096439.749189 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 135.951900] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 135.958390] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 135.933191] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 135.964360] systemd[1]: nix-grpc-daemon.service: Consumed 1.602s CPU time over 30.766s wall clock time, 38.9M memory peak, 47.6K incoming IP traffic, 91.2K outgoing IP traffic. worker1 # [ 135.983535] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 136.124424] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 136.126770] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 136.130551] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 136.135868] nix-grpc-daemon[1495]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 136.156132] nix-grpc-daemon[1495]: E0000 00:00:1790096440.751861 1495 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.158497] nix-grpc-daemon[1495]: E0000 00:00:1790096440.755466 1495 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.180665] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 136.182847] nix-grpc-daemon[1495]: E0000 00:00:1790096440.761463 1537 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.185536] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 136.186363] nix-grpc-daemon[1495]: E0000 00:00:1790096440.781701 1537 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.191543] nix-grpc-daemon[1495]: E0000 00:00:1790096440.781861 1537 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.195992] nix-grpc-daemon[1495]: E0000 00:00:1790096440.796485 1537 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.198430] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.938646] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48916 worker2 # [ 135.938808] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 136.167200] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48916 worker2 # [ 136.183303] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.91 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 137.099796] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: c5b69b5c34544c7c819645c08dcc7d3f client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds) worker2 # [ 137.234745] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38088 client # [ 137.315162] nix[1440]: don't know how to build these paths: client # [ 137.316447] nix[1440]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 137.254715] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 137.261910] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 137.299727] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38088 worker2 # [ 137.314717] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5) worker2 # [ 137.363982] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 2/5) worker2 # [ 137.561006] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 211 ms (attempt 3/5) worker2 # [ 137.773858] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 489 ms (attempt 4/5) worker2 # [ 138.268675] nix-grpc-daemon[1437]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 138.280720] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 138.300474] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38088 client # [ 138.385403] nix[1440]: copying 1 paths to 'lb:50051'... worker2 # [ 138.316091] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38088 client # [ 138.390417] nix[1440]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 138.331776] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted) worker2 # [ 138.366656] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=info event=niks3_push_started pid=1569 worker2 # [ 138.514844] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.138Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 138.622639] nix-daemon[859]: accepted connection from pid 1582, user nix-grpc-daemon (trusted) worker2 # [ 138.642608] nix-daemon[859]: reaped child process 1585, status = succeeded lb # [ 138.678073] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.659335] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 138.662711] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.282Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 138.716450] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls worker2 # [ 138.696477] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.320Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.725409] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 138.737733] niks3-server[1105]: 2026/09/22 17:00:43 INFO Signed narinfos id=51 count=1 lb # [ 138.747387] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 138.760000] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 138.767671] niks3-server[1105]: 2026/09/22 17:00:43 INFO Completed upload id=51 worker2 # [ 138.737452] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.361Z level=INFO msg="Upload complete. (218ms)" lb # [ 138.772584] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 138.748008] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38088 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 138.756302] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38088 worker2 # [ 138.767687] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted) worker2 # [ 138.772879] nix-daemon[859]: reaped child process 1566, status = succeeded worker2 # [ 138.783111] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted) client # [ 138.854802] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb # [ 139.947371] envoy[919]: conn peer=[2001:db8:1::1]:47346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1541 ms=8393 lb # [ 139.951947] envoy[919]: conn peer=[2001:db8:1::3]:33604 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2953 tx=948 ms=29474 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 141.014399] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1437 (nix-grpc-daemon) on client request. worker2 # [ 141.015962] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. worker2 # [ 141.038291] systemd[1]: Starting scheduler-token.service... lb # [ 141.065184] niks3-server[1105]: 2026/09/22 17:00:45 INFO lead: released remote=192.168.1.4:41600 client # [ 141.117191] nix[1440]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker1 # [ 141.073902] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" client # [ 141.123435] nix[1440]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 141.054633] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 141.061358] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 141.062643] systemd[1]: nix-grpc-daemon.service: Consumed 488ms CPU time over 17.463s wall clock time, 34.7M memory peak, 14.4K incoming IP traffic, 23.3K outgoing IP traffic. worker2 # [ 141.095435] nix-daemon[859]: reaped child process 1496, status = succeeded worker2 # [ 141.100418] nix-daemon[859]: reaped child process 1587, status = succeeded worker2 # [ 141.113601] nix-daemon[1589]: unexpected Nix daemon error: error: interrupted by the user lb # [ 141.144758] niks3-server[1105]: 2026/09/22 17:00:45 INFO lead: acquired remote=192.168.1.3:56450 worker2 # [ 141.119090] nix-daemon[859]: reaped child process 1589, status = failed with exit code 1 worker1 # [ 141.150279] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_take_over worker2 # [ 141.133063] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 141.135924] systemd[1]: Finished scheduler-token.service. worker2 # [ 141.138698] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 96ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 141.142527] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 141.147607] systemd[1]: Starting scheduler-token.service... worker2 # [ 141.188625] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 141.192158] systemd[1]: Finished scheduler-token.service. worker2 # [ 141.192941] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 141.196495] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 141.274058] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.293611] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 141.295794] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 141.299545] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 141.304069] nix-grpc-daemon[1632]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 141.305539] nix-grpc-daemon[1632]: E0000 00:00:1790096445.928043 1632 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 141.355513] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds) client: must succeed: sleep 5.0 worker2 # [ 141.352065] nix-grpc-daemon[1632]: E0000 00:00:1790096445.934271 1632 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 141.354386] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 141.358429] nix-grpc-daemon[1632]: E0000 00:00:1790096445.944954 1671 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 141.361401] nix-grpc-daemon[1632]: E0000 00:00:1790096445.962923 1671 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 141.363840] nix-grpc-daemon[1632]: E0000 00:00:1790096445.963068 1671 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 141.369235] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 141.372160] nix-grpc-daemon[1632]: E0000 00:00:1790096445.981713 1671 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 141.374986] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.376360] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 141.556422] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.561787] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" client # [ 141.621635] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting worker2 # [ 141.576232] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.582857] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 141.762405] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.767965] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 141.783979] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.790623] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 141.968752] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.975880] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 141.991821] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.044235] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 142.048703] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:42992 worker1 # [ 142.054042] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 142.177490] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.193645] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 142.394710] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.400927] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:42992 worker1 # [ 142.405712] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 client # [ 142.627985] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting worker1 # [ 144.596434] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:49Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 144.596820] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:49Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 144.617887] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 worker2 # [ 144.632959] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 144.651586] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 144.672201] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) client # [ 144.743845] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 146.692536] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:51Z level=info event=niks3_push_started pid=1713 worker2 # [ 146.729118] nix-daemon[859]: reaped child process 1688, status = succeeded worker2 # [ 146.824522] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.448Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 146.930465] nix-daemon[859]: accepted connection from pid 1725, user nix-grpc-daemon (trusted) worker2 # [ 146.949737] nix-daemon[859]: reaped child process 1728, status = succeeded lb # [ 146.979516] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 146.963402] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 146.967129] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.585Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 147.034328] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign worker2 # [ 147.015342] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.638Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.051986] niks3-server[1105]: 2026/09/22 17:00:51 INFO Signed narinfos id=52 count=1 lb # [ 147.063076] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 147.079991] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete worker2 # [ 147.060309] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.684Z level=INFO msg="Upload complete. (232ms)" lb # [ 147.094630] niks3-server[1105]: 2026/09/22 17:00:51 INFO Completed upload id=52 lb # [ 147.104621] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 147.111361] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 147.072283] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 lb # [ 147.116374] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo worker2 # [ 147.091256] nix-daemon[859]: reaped child process 1686, status = succeeded client # [ 147.180796] systemd[1]: mid.service: Deactivated successfully. client # [ 147.185870] systemd[1]: mid.service: Consumed 213ms CPU time over 10.082s wall clock time, 41.4M memory peak, 9.3K incoming IP traffic, 11.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.85 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 147.923106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 944ac6f3a5d64fc883c401a251bfcf6c client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker1 # [ 148.067326] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 148.081552] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) client # [ 148.142929] nix[1476]: don't know how to build these paths: client # [ 148.144463] nix[1476]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 148.108868] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 148.115727] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 148.161912] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 148.176857] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5) worker1 # [ 148.239673] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5) worker1 # [ 148.394652] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 3/5) worker1 # [ 148.435615] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 549 ms (attempt 4/5) worker1 # [ 148.986635] nix-grpc-daemon[1495]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 148.991702] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 149.008767] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 client # [ 149.067621] nix[1476]: copying 1 paths to 'lb:50051'... client # [ 149.068125] nix[1476]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 149.024346] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 149.044353] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 149.085426] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=info event=niks3_push_started pid=1594 worker1 # [ 149.247045] nix-grpc-daemon[1594]: time=2026-09-22T17:00:53.863Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 149.353852] nix-daemon[831]: accepted connection from pid 1607, user nix-grpc-daemon (trusted) worker1 # [ 149.369928] nix-daemon[831]: reaped child process 1610, status = succeeded lb # [ 149.377898] niks3-server[1105]: 2026/09/22 17:00:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.392068] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.008Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 149.393572] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.008Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 149.424532] niks3-server[1105]: 2026/09/22 17:00:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker1 # [ 149.434688] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.050Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.438548] niks3-server[1105]: 2026/09/22 17:00:54 INFO Signed narinfos id=53 count=1 lb # [ 149.447204] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 149.461151] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 149.470002] niks3-server[1105]: 2026/09/22 17:00:54 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 149.477747] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo lb # [ 149.482533] niks3-server[1105]: 2026/09/22 17:00:54 INFO Completed upload id=53 worker1 # [ 149.483155] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.099Z level=INFO msg="Upload complete. (232ms)" worker1 # [ 149.493357] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 149.508290] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 149.518725] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 149.525475] nix-daemon[831]: reaped child process 1591, status = succeeded worker1 # [ 149.536251] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) client # [ 149.579118] nix[1476]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... lb # [ 149.945496] envoy[919]: conn peer=[2001:db8:1::4]:37976 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1918 tx=311 ms=17261 lb # [ 149.950520] envoy[919]: conn peer=[2001:db8:1::1]:58048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2099 ms=9852 client: must succeed: systemctl kill -s INT intr client # [ 151.675126] systemd[1]: intr.service: Sent signal SIGINT to main process 1476 (nix) on client request. worker1 # [ 151.636405] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:56Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 151.684740] nix[1476]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 151.696962] nix[1476]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 151.721863] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 151.726550] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 151.728789] systemd[1]: intr.service: Consumed 167ms CPU time over 3.801s wall clock time, 43.5M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 151.764328] nix-daemon[831]: reaped child process 1612, status = succeeded worker1 # [ 151.767981] nix-daemon[1614]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 151.774411] nix-daemon[831]: reaped child process 1614, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 5.09 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 153.032790] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 07a7f0811b714c8bbe091c72bad7efe2 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds) client # [ 153.222720] nix[1497]: don't know how to build these paths: client # [ 153.224391] nix[1497]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 153.179076] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 153.194387] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 153.200312] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 153.213476] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 worker2 # [ 153.229246] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5) worker2 # [ 153.326635] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5) worker2 # [ 153.345549] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 3/5) worker2 # [ 153.496998] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 4/5) worker2 # [ 153.724270] nix-grpc-daemon[1632]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 153.730733] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 153.750010] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35378 client # [ 153.839944] nix[1497]: copying 1 paths to 'lb:50051'... worker2 # [ 153.766989] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35378 client # [ 153.840385] nix[1497]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 153.782099] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 153.951500] nix-daemon[859]: accepted connection from pid 1764, user nix-grpc-daemon (trusted) worker2 # [ 153.971747] nix-daemon[859]: reaped child process 1767, status = succeeded lb # [ 153.999070] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 153.986613] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.609Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 153.988221] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.609Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 154.039748] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker2 # [ 154.024034] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.646Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.055073] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 154.064902] niks3-server[1105]: 2026/09/22 17:00:58 INFO Signed narinfos id=54 count=1 lb # [ 154.067625] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 154.082088] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 154.090467] niks3-server[1105]: 2026/09/22 17:00:58 INFO Completed upload id=54 worker2 # [ 154.061048] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.684Z level=INFO msg="Upload complete. (261ms)" lb # [ 154.093045] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 154.076526] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 154.088551] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 worker2 # [ 154.098214] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 154.104498] nix-daemon[859]: reaped child process 1762, status = succeeded client # [ 154.186649] nix[1497]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2 # [ 154.117369] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # [ 155.971162] nixos[1803]: switching to system configuration /nix/store/x9vw27xc29pg9q8wzyqzk0qs9m96ksmm-nixos-system-worker2-test worker2 # activating the configuration... worker2 # [ 155.978890] systemd[1]: Stopped target Remote File Systems. worker2 # [ 155.979684] systemd[1]: Stopped target Local File Systems. worker2 # setting up /etc... worker2 # [ 156.537478] systemd[1]: Reload requested from client PID 1803 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 156.538991] systemd[1]: Reloading... lb # [ 159.948989] envoy[919]: conn peer=[2001:db8:1::1]:38880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=827 ms=3612 worker2 # [ 161.367385] systemd[1]: Reloading finished in 4817 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 161.493074] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # [ 161.619301] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service worker2 # [ 161.635040] systemd[1]: Reached target Reactivate sysinit units. worker2 # [ 161.635934] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 161.641466] dbus-broker-launch[487]: Caught SIGHUP, trigger reload. worker2 # [ 161.642457] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 161.649521] dbus-broker-launch[487]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 161.656593] dbus-broker-launch[487]: NSS returned no entry for 'systemd-timesync' worker2 # [ 161.666513] dbus-broker-launch[487]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 161.677598] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 161.694431] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 161.700838] systemd[1]: Reached target Remote File Systems. worker2 # [ 161.705354] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 161.706637] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 161.716240] systemd[1]: Reached target Local File Systems. worker2 # [ 161.721364] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 161.722790] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 161.723569] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 161.736122] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 161.738236] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 161.739734] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 161.744559] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 162.065970] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 162.067047] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 162.067789] systemd[1]: Starting linger-users.service... worker2 # [ 162.114407] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 162.115373] systemd[1]: Finished linger-users.service. worker2 # [ 162.319162] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:06Z level=info event=draining worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 162.368289] nixos[1803]: finished switching to system configuration /nix/store/x9vw27xc29pg9q8wzyqzk0qs9m96ksmm-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.84 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 162.655367] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35378 worker1 # [ 162.683107] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 162.708234] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 162.714976] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 162.767041] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 162.782060] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 client # copying 1 paths to 'lb:50051'... worker1 # [ 162.798384] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 162.822009] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 162.982789] nix-daemon[831]: accepted connection from pid 1669, user nix-grpc-daemon (trusted) worker1 # [ 163.003937] nix-daemon[831]: reaped child process 1672, status = succeeded lb # [ 163.007383] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 163.021189] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.636Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 163.022956] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.636Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 163.046339] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker1 # [ 163.055549] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.671Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.059126] niks3-server[1105]: 2026/09/22 17:01:07 INFO Signed narinfos id=55 count=1 lb # [ 163.067657] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 163.082001] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker1 # [ 163.087129] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.703Z level=INFO msg="Upload complete. (247ms)" lb # [ 163.089368] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 163.094630] niks3-server[1105]: 2026/09/22 17:01:07 INFO Completed upload id=55 lb # [ 163.098669] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 163.099717] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=528 worker1 # [ 163.116333] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 163.126527] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 163.130568] nix-daemon[831]: reaped child process 1667, status = succeeded worker1 # [ 163.144846] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 164.190229] nix-daemon[831]: reaped child process 1676, status = succeeded worker1 # [ 164.291135] nix-daemon[831]: accepted connection from pid 1692, user nix-grpc-daemon (trusted) worker1 # [ 164.307202] nix-daemon[831]: reaped child process 1695, status = succeeded lb # [ 164.313355] niks3-server[1105]: 2026/09/22 17:01:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.328507] nix-grpc-daemon[1594]: time=2026-09-22T17:01:08.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.333679] nix-grpc-daemon[1594]: time=2026-09-22T17:01:08.944Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 164.357581] niks3-server[1105]: 2026/09/22 17:01:08 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 164.383288] niks3-server[1105]: 2026/09/22 17:01:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker1 # [ 164.392538] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.008Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.393631] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 164.403000] niks3-server[1105]: 2026/09/22 17:01:09 INFO Signed narinfos id=56 count=1 lb # [ 164.411191] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker1 # [ 164.416697] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.033Z level=INFO msg="Upload complete. (256ms)" lb # [ 164.417237] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 164.420630] niks3-server[1105]: 2026/09/22 17:01:09 INFO Completed upload id=56 lb # [ 164.425626] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 164.429197] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 164.452851] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 164.458727] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 164.463276] nix-daemon[831]: reaped child process 1674, status = succeeded worker1 # [ 164.493395] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 164.505767] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 164.520220] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.136Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 164.536341] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 164.558816] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 164.724481] nix-daemon[831]: accepted connection from pid 1701, user nix-grpc-daemon (trusted) worker1 # [ 164.743398] nix-daemon[831]: reaped child process 1704, status = succeeded lb # [ 164.746260] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.757429] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.373Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 164.759001] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.373Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 164.778361] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker1 # [ 164.790388] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.406Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.792410] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 164.801383] niks3-server[1105]: 2026/09/22 17:01:09 INFO Signed narinfos id=57 count=1 lb # [ 164.803800] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker1 # [ 164.816169] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.432Z level=INFO msg="Upload complete. (240ms)" lb # [ 164.815935] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 164.823510] niks3-server[1105]: 2026/09/22 17:01:09 INFO Completed upload id=57 lb # [ 164.825329] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker1 # [ 164.828123] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 164.840198] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 164.848229] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 164.850694] nix-daemon[831]: reaped child process 1699, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker1 # [ 164.861125] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 165.752633] nix-daemon[831]: reaped child process 1708, status = succeeded worker1 # [ 165.839649] nix-daemon[831]: accepted connection from pid 1724, user nix-grpc-daemon (trusted) worker1 # [ 165.858477] nix-daemon[831]: reaped child process 1727, status = succeeded lb # [ 165.864978] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.876436] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.492Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 165.878477] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.492Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 165.900379] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 165.917450] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst worker1 # [ 165.931216] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.547Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.930171] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 165.938636] niks3-server[1105]: 2026/09/22 17:01:10 INFO Signed narinfos id=58 count=1 lb # [ 165.944979] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 165.954156] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete worker1 # [ 165.962698] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.579Z level=INFO msg="Upload complete. (289ms)" lb # [ 165.964461] niks3-server[1105]: 2026/09/22 17:01:10 INFO Completed upload id=58 lb # [ 165.966753] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 165.978831] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 166.000413] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 166.006168] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 166.011865] nix-daemon[831]: reaped child process 1706, status = succeeded worker1 # [ 166.041392] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 166.056264] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 166.069829] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.686Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 166.080981] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 166.103014] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 166.257232] nix-daemon[831]: accepted connection from pid 1733, user nix-grpc-daemon (trusted) worker1 # [ 166.276558] nix-daemon[831]: reaped child process 1736, status = succeeded lb # [ 166.280417] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.289816] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.906Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 166.291379] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.906Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" worker1 # [ 166.322076] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.938Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.312343] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 166.326456] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 166.338125] niks3-server[1105]: 2026/09/22 17:01:10 INFO Signed narinfos id=59 count=1 lb # [ 166.347871] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst worker1 # [ 166.357506] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.974Z level=INFO msg="Upload complete. (237ms)" lb # [ 166.358427] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 166.363604] niks3-server[1105]: 2026/09/22 17:01:10 INFO Completed upload id=59 lb # [ 166.367394] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker1 # [ 166.368908] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 166.383518] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 166.393137] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 166.396315] nix-daemon[831]: reaped child process 1731, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker1 # [ 166.408260] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 167.911803] nix-daemon[831]: reaped child process 1740, status = succeeded worker1 # [ 167.978153] nix-daemon[831]: accepted connection from pid 1756, user nix-grpc-daemon (trusted) worker1 # [ 167.998158] nix-daemon[831]: reaped child process 1759, status = succeeded lb # [ 168.003105] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.015669] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.630Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 168.017503] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.630Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 168.047690] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker1 # [ 168.061257] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.677Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.061644] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 168.070251] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 168.082868] niks3-server[1105]: 2026/09/22 17:01:12 INFO Signed narinfos id=60 count=1 worker1 # [ 168.090465] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.707Z level=INFO msg="Upload complete. (268ms)" lb # [ 168.090578] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 168.095129] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 168.101585] niks3-server[1105]: 2026/09/22 17:01:12 INFO Completed upload id=60 lb # [ 168.104742] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker1 # [ 168.104287] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker1 # [ 168.123851] nix-daemon[831]: reaped child process 1738, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.69 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 168.628449] nix-daemon[859]: reaped child process 1771, status = succeeded worker2 # [ 168.730388] nix-daemon[859]: accepted connection from pid 1944, user nix-grpc-daemon (trusted) worker2 # [ 168.748224] nix-daemon[859]: reaped child process 1947, status = succeeded lb # [ 168.781794] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.763807] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 168.766396] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.386Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 168.826810] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 168.838584] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker2 # [ 168.816406] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.439Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.848713] niks3-server[1105]: 2026/09/22 17:01:13 INFO Signed narinfos id=61 count=1 lb # [ 168.851608] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 168.865074] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls worker2 # [ 168.846635] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.470Z level=INFO msg="Upload complete. (297ms)" lb # [ 168.876505] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 168.880794] niks3-server[1105]: 2026/09/22 17:01:13 INFO Completed upload id=61 lb # [ 168.886857] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 168.861274] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker2 # [ 168.880462] nix-daemon[859]: reaped child process 1769, status = succeeded client # [ 168.971779] systemd[1]: sw.service: Deactivated successfully. client # [ 168.976281] systemd[1]: sw.service: Consumed 190ms CPU time over 15.943s wall clock time, 41.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic. worker2 # [ 169.324532] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:13Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 169.580702] systemd[1]: Starting scheduler-token.service... worker2 # [ 169.644356] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 169.649444] systemd[1]: Finished scheduler-token.service. worker2 # [ 169.650931] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 67ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 169.950907] envoy[919]: conn peer=[2001:db8:1::1]:44990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4412 ms=5443 lb # [ 169.955815] envoy[919]: conn peer=[2001:db8:1::1]:32774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3012 tx=1577 ms=15732 worker1 # [ 170.368324] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:42992 worker2 # [ 170.341045] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:14Z level=info event=scheduler_restarting worker1 # [ 170.368673] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:14Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 170.363595] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 170.391305] nix-daemon[859]: reaped child process 1684, status = succeeded worker2 # [ 171.067732] nix-grpc-daemon[1632]: E0000 00:00:1790096475.691518 1677 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker2 # [ 171.080723] nix-grpc-daemon[1632]: E0000 00:00:1790096475.704584 1677 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 171.360819] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 171.367003] systemd[1]: nix-grpc-daemon.service: Consumed 829ms CPU time over 30.313s wall clock time, 36.2M memory peak, 21.6K incoming IP traffic, 31.4K outgoing IP traffic. worker2 # [ 171.387750] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 171.538771] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 171.541205] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 171.545062] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 171.550299] nix-grpc-daemon[1964]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 171.551715] nix-grpc-daemon[1964]: E0000 00:00:1790096476.173640 1964 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.10 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 171.597402] nix-grpc-daemon[1964]: E0000 00:00:1790096476.179705 1964 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 171.599604] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 171.602164] nix-grpc-daemon[1964]: E0000 00:00:1790096476.187489 2006 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 171.605211] nix-grpc-daemon[1964]: E0000 00:00:1790096476.203621 2006 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 171.607623] nix-grpc-daemon[1964]: E0000 00:00:1790096476.204309 2006 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 171.610071] nix-grpc-daemon[1964]: E0000 00:00:1790096476.221252 2006 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 171.615332] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 171.618545] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.651503] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:42992 worker1 # [ 171.657460] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.96 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 173.085744] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 173.098963] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 173.119848] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 173.134560] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5) worker2 # [ 173.178243] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5) worker2 # [ 173.212921] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 3/5) worker2 # [ 173.314363] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 493 ms (attempt 4/5) worker2 # [ 173.809732] nix-grpc-daemon[1964]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 173.817132] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 173.893369] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 173.900636] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 173.906975] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 173.912317] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 173.946078] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 173.961505] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 173.976113] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 173.990541] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 174.147439] nix-daemon[831]: accepted connection from pid 1768, user nix-grpc-daemon (trusted) worker1 # [ 174.169859] nix-daemon[831]: reaped child process 1771, status = succeeded lb # [ 174.167882] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.180507] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 174.182811] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.796Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 174.196359] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 174.206961] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.823Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.210407] niks3-server[1105]: 2026/09/22 17:01:18 INFO Signed narinfos id=62 count=1 lb # [ 174.219303] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 174.229732] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 174.243961] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.860Z level=INFO msg="Upload complete. (237ms)" lb # [ 174.244362] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete worker1 # [ 174.252271] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 174.254881] niks3-server[1105]: 2026/09/22 17:01:18 INFO Completed upload id=62 lb # [ 174.259808] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 174.268412] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 174.276584] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 174.281879] nix-daemon[831]: reaped child process 1766, status = succeeded worker1 # [ 174.295397] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 175.727960] nix-daemon[831]: reaped child process 1775, status = succeeded worker1 # [ 175.823256] nix-daemon[831]: accepted connection from pid 1791, user nix-grpc-daemon (trusted) worker1 # [ 175.843610] nix-daemon[831]: reaped child process 1794, status = succeeded lb # [ 175.846784] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.859558] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 175.861194] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.473Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 175.881182] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 175.896722] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 175.908291] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker1 # [ 175.916493] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.533Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.918632] niks3-server[1105]: 2026/09/22 17:01:20 INFO Signed narinfos id=63 count=1 lb # [ 175.924310] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 175.937648] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.554Z level=INFO msg="Upload complete. (231ms)" lb # [ 175.936698] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 175.943296] niks3-server[1105]: 2026/09/22 17:01:20 INFO Completed upload id=63 lb # [ 175.947160] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 175.950920] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=1 worker1 # [ 175.973449] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 175.978714] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 175.984354] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 175.989554] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 175.994178] nix-daemon[831]: reaped child process 1773, status = succeeded worker1 # [ 175.997659] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 176.007471] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 176.136173] nix-daemon[831]: accepted connection from pid 1800, user nix-grpc-daemon (trusted) worker1 # [ 176.155973] nix-daemon[831]: reaped child process 1803, status = succeeded lb # [ 176.157293] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.167090] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.783Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 176.168675] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.783Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 176.186953] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 176.196850] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.813Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.201582] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 176.212004] niks3-server[1105]: 2026/09/22 17:01:20 INFO Signed narinfos id=64 count=1 worker1 # [ 176.220068] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.836Z level=INFO msg="Upload complete. (202ms)" lb # [ 176.221100] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 176.226871] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 176.227944] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 176.233463] niks3-server[1105]: 2026/09/22 17:01:20 INFO Completed upload id=64 lb # [ 176.235982] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 176.239959] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 176.243481] nix-daemon[831]: reaped child process 1798, status = succeeded worker1 # [ 176.250644] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 176.264138] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 176.933042] nix-daemon[831]: reaped child process 1807, status = succeeded worker1 # [ 177.051065] nix-daemon[831]: accepted connection from pid 1823, user nix-grpc-daemon (trusted) worker1 # [ 177.069941] nix-daemon[831]: reaped child process 1826, status = succeeded lb # [ 177.075028] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.088428] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.702Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 177.090024] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.702Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 177.117602] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 177.134511] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.750Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.132948] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 177.142520] niks3-server[1105]: 2026/09/22 17:01:21 INFO Signed narinfos id=65 count=1 lb # [ 177.151421] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 177.159861] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 177.169386] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 177.174314] niks3-server[1105]: 2026/09/22 17:01:21 INFO Completed upload id=65 worker1 # [ 177.175636] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.792Z level=INFO msg="Upload complete. (300ms)" lb # [ 177.177743] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 177.196153] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=1 worker1 # [ 177.216361] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 177.220754] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 177.224888] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 177.228592] nix-daemon[831]: reaped child process 1805, status = succeeded worker1 # [ 177.234780] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 177.248407] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 177.260441] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 177.407754] nix-daemon[831]: accepted connection from pid 1832, user nix-grpc-daemon (trusted) worker1 # [ 177.425116] nix-daemon[831]: reaped child process 1835, status = succeeded lb # [ 177.429165] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.438776] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.055Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 177.440436] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.055Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 177.457543] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 177.468140] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.084Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.470266] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 177.482048] niks3-server[1105]: 2026/09/22 17:01:22 INFO Signed narinfos id=66 count=1 lb # [ 177.490171] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst worker1 # [ 177.498628] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.115Z level=INFO msg="Upload complete. (223ms)" lb # [ 177.500966] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 177.506666] niks3-server[1105]: 2026/09/22 17:01:22 INFO Completed upload id=66 worker1 # [ 177.508238] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 177.510054] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 177.519202] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 177.526266] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 177.531232] nix-daemon[831]: reaped child process 1830, status = succeeded worker1 # [ 177.541844] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 179.210746] nix-daemon[831]: reaped child process 1839, status = succeeded worker1 # [ 179.325784] nix-daemon[831]: accepted connection from pid 1855, user nix-grpc-daemon (trusted) worker1 # [ 179.343649] nix-daemon[831]: reaped child process 1858, status = succeeded lb # [ 179.352451] niks3-server[1105]: 2026/09/22 17:01:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.363048] nix-grpc-daemon[1594]: time=2026-09-22T17:01:23.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 179.364807] nix-grpc-daemon[1594]: time=2026-09-22T17:01:23.978Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 179.392322] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker1 # [ 179.403403] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.019Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.407797] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 179.418045] niks3-server[1105]: 2026/09/22 17:01:24 INFO Signed narinfos id=67 count=1 lb # [ 179.423095] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls worker1 # [ 179.433243] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.049Z level=INFO msg="Upload complete. (279ms)" lb # [ 179.432423] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 179.444445] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 179.450788] niks3-server[1105]: 2026/09/22 17:01:24 INFO Completed upload id=67 worker1 # [ 179.450321] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=1 lb # [ 179.453311] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 179.469618] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 179.481370] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 179.494142] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 179.504212] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 179.507626] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 179.513881] nix-daemon[831]: reaped child process 1837, status = succeeded worker1 # [ 179.518303] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.097Z level=INFO msg="All 1 paths already cached" worker1 # [ 179.522209] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 179.643809] nix-daemon[831]: accepted connection from pid 1864, user nix-grpc-daemon (trusted) worker1 # [ 179.661594] nix-daemon[831]: reaped child process 1867, status = succeeded lb # [ 179.665448] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.675229] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 179.677161] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.291Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 179.694411] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 179.701969] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.318Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.707650] niks3-server[1105]: 2026/09/22 17:01:24 INFO Signed narinfos id=68 count=1 lb # [ 179.716170] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 179.730951] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 179.737431] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.354Z level=INFO msg="Upload complete. (224ms)" lb # [ 179.740378] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 179.745699] niks3-server[1105]: 2026/09/22 17:01:24 INFO Completed upload id=68 worker1 # [ 179.748287] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 179.750246] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 179.760336] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 179.770402] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 179.773645] nix-daemon[831]: reaped child process 1862, status = succeeded worker1 # [ 179.788426] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) lb # [ 179.951592] envoy[919]: conn peer=[2001:db8:1::4]:51512 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3760 tx=489 ms=28978 worker1 # [ 181.267994] nix-daemon[831]: reaped child process 1871, status = succeeded worker1 # [ 181.363679] nix-daemon[831]: accepted connection from pid 1887, user nix-grpc-daemon (trusted) worker1 # [ 181.382080] nix-daemon[831]: reaped child process 1890, status = succeeded lb # [ 181.388874] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.404086] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 181.406039] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.017Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" worker1 # [ 181.447611] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.063Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.436533] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 181.453919] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 181.464694] niks3-server[1105]: 2026/09/22 17:01:26 INFO Signed narinfos id=69 count=1 lb # [ 181.467249] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 181.479232] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 181.484370] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 181.487473] niks3-server[1105]: 2026/09/22 17:01:26 INFO Completed upload id=69 lb # [ 181.489689] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 181.485023] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.101Z level=INFO msg="Upload complete. (285ms)" worker1 # [ 181.504214] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=1 worker1 # [ 181.519880] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.528905] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 181.532856] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 181.537447] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 181.541743] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.157Z level=INFO msg="All 1 paths already cached" worker1 # [ 181.544292] nix-daemon[831]: reaped child process 1869, status = succeeded worker1 # [ 181.547718] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 181.556568] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 181.685893] nix-daemon[831]: accepted connection from pid 1896, user nix-grpc-daemon (trusted) worker1 # [ 181.702027] nix-daemon[831]: reaped child process 1899, status = succeeded lb # [ 181.707646] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.717145] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.333Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 181.718655] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.333Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 181.738051] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker1 # [ 181.744983] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.361Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.753559] niks3-server[1105]: 2026/09/22 17:01:26 INFO Signed narinfos id=70 count=1 lb # [ 181.759808] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 181.770671] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 181.779807] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.396Z level=INFO msg="Upload complete. (214ms)" lb # [ 181.779762] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 181.787282] niks3-server[1105]: 2026/09/22 17:01:26 INFO Completed upload id=70 worker1 # [ 181.787005] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 181.790344] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 181.802530] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 181.808664] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 181.813781] nix-daemon[831]: reaped child process 1894, status = succeeded worker1 # [ 181.826582] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 182.922085] nix-daemon[831]: reaped child process 1903, status = succeeded worker1 # [ 183.001850] nix-daemon[831]: accepted connection from pid 1919, user nix-grpc-daemon (trusted) worker1 # [ 183.021433] nix-daemon[831]: reaped child process 1922, status = succeeded lb # [ 183.026888] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.039523] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.654Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.041308] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.655Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 183.068146] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 183.078090] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.694Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.081230] niks3-server[1105]: 2026/09/22 17:01:27 INFO Signed narinfos id=71 count=1 lb # [ 183.091308] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 183.099427] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 183.110287] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker1 # [ 183.111209] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.727Z level=INFO msg="Upload complete. (274ms)" lb # [ 183.117977] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 183.122493] niks3-server[1105]: 2026/09/22 17:01:27 INFO Completed upload id=71 lb # [ 183.126369] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 183.128426] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=1 worker1 # [ 183.148367] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 183.156501] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 183.162235] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 183.168208] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 183.173659] nix-daemon[831]: reaped child process 1901, status = succeeded worker1 # [ 183.175741] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.789Z level=INFO msg="All 1 paths already cached" worker1 # [ 183.188326] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 183.197442] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 183.318740] nix-daemon[831]: accepted connection from pid 1928, user nix-grpc-daemon (trusted) worker1 # [ 183.337260] nix-daemon[831]: reaped child process 1931, status = succeeded lb # [ 183.341398] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.350875] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.967Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.352519] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.967Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 183.372211] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 183.381843] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.998Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.383793] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 183.396212] niks3-server[1105]: 2026/09/22 17:01:27 INFO Signed narinfos id=72 count=1 lb # [ 183.398893] niks3-server[1105]: 2026/09/22 17:01:28 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst lb # [ 183.411981] niks3-server[1105]: 2026/09/22 17:01:28 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 183.416759] niks3-server[1105]: 2026/09/22 17:01:28 INFO Completed upload id=72 worker1 # [ 183.416950] nix-grpc-daemon[1594]: time=2026-09-22T17:01:28.033Z level=INFO msg="Upload complete. (214ms)" lb # [ 183.419361] niks3-server[1105]: 2026/09/22 17:01:28 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 183.424224] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 183.436896] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 183.448798] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 183.454069] nix-daemon[831]: reaped child process 1926, status = succeeded worker1 # [ 183.467055] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 184.977273] nix-daemon[831]: reaped child process 1935, status = succeeded worker1 # [ 185.048855] nix-daemon[831]: accepted connection from pid 1951, user nix-grpc-daemon (trusted) worker1 # [ 185.069766] nix-daemon[831]: reaped child process 1954, status = succeeded lb # [ 185.074505] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.084468] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.699Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 185.086256] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.700Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 185.116555] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 185.128297] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.744Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.132329] niks3-server[1105]: 2026/09/22 17:01:29 INFO Signed narinfos id=73 count=1 lb # [ 185.134806] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 185.148146] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 185.150916] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 185.156747] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 185.163120] niks3-server[1105]: 2026/09/22 17:01:29 INFO Completed upload id=73 worker1 # [ 185.161999] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.778Z level=INFO msg="Upload complete. (284ms)" lb # [ 185.165102] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 185.181161] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=1 worker1 # [ 185.197213] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 185.208586] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 185.214068] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 185.228237] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 185.231346] nix-daemon[831]: reaped child process 1933, status = succeeded worker1 # [ 185.237664] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 185.243182] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.829Z level=INFO msg="All 2 paths already cached" worker1 # [ 185.247582] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 185.365627] nix-daemon[831]: accepted connection from pid 1960, user nix-grpc-daemon (trusted) worker1 # [ 185.385232] nix-daemon[831]: reaped child process 1963, status = succeeded lb # [ 185.382174] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.391836] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 185.393817] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.008Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 185.409016] niks3-server[1105]: 2026/09/22 17:01:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign worker1 # [ 185.415418] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.032Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.421329] niks3-server[1105]: 2026/09/22 17:01:30 INFO Signed narinfos id=74 count=1 lb # [ 185.429961] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 185.442324] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 185.451080] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.067Z level=INFO msg="Upload complete. (214ms)" lb # [ 185.450742] niks3-server[1105]: 2026/09/22 17:01:30 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 185.454563] niks3-server[1105]: 2026/09/22 17:01:30 INFO Completed upload id=74 lb # [ 185.459646] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 185.461027] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 185.472845] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 185.481622] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 185.484764] nix-daemon[831]: reaped child process 1958, status = succeeded worker1 # [ 185.496201] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 186.601404] nix-daemon[831]: reaped child process 1967, status = succeeded worker1 # [ 186.669665] nix-daemon[831]: accepted connection from pid 1983, user nix-grpc-daemon (trusted) worker1 # [ 186.690948] nix-daemon[831]: reaped child process 1986, status = succeeded lb # [ 186.695973] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.710162] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.325Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 186.711887] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.325Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 186.745052] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls worker1 # [ 186.752278] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.368Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.755529] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 186.765274] niks3-server[1105]: 2026/09/22 17:01:31 INFO Signed narinfos id=75 count=1 worker1 # [ 186.766319] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.383Z level=INFO msg="Upload complete. (260ms)" lb # [ 186.767265] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 186.777434] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete worker1 # [ 186.776332] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=1 lb # [ 186.781160] niks3-server[1105]: 2026/09/22 17:01:31 INFO Completed upload id=75 lb # [ 186.783953] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo lb # [ 186.786706] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker1 # [ 186.794087] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 186.798762] nix-daemon[831]: reaped child process 1965, status = succeeded worker1 # [ 186.821443] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 186.836115] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 186.850691] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.467Z level=INFO msg="All 2 paths already cached" worker1 # [ 186.860367] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 186.873857] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 187.017867] nix-daemon[831]: accepted connection from pid 1990, user nix-grpc-daemon (trusted) worker1 # [ 187.036453] nix-daemon[831]: reaped child process 1993, status = succeeded lb # [ 187.043524] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.054269] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 187.055777] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.670Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 187.077335] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 187.084094] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.700Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.091393] niks3-server[1105]: 2026/09/22 17:01:31 INFO Signed narinfos id=76 count=1 lb # [ 187.099196] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 187.113110] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.729Z level=INFO msg="Upload complete. (225ms)" lb # [ 187.112437] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 187.120364] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 187.122211] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 187.126991] niks3-server[1105]: 2026/09/22 17:01:31 INFO Completed upload id=76 lb # [ 187.129672] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 187.132503] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 187.143985] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 187.149229] nix-daemon[831]: reaped child process 1988, status = succeeded worker1 # [ 187.160139] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 188.637003] nix-daemon[831]: reaped child process 1997, status = succeeded worker1 # [ 188.748138] nix-daemon[831]: accepted connection from pid 2013, user nix-grpc-daemon (trusted) worker1 # [ 188.765788] nix-daemon[831]: reaped child process 2016, status = succeeded lb # [ 188.773130] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.784543] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.399Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 188.786881] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.399Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 188.808794] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 188.830243] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker1 # [ 188.837021] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.453Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.843000] niks3-server[1105]: 2026/09/22 17:01:33 INFO Signed narinfos id=77 count=1 lb # [ 188.847637] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 188.855461] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 188.861629] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 188.866998] niks3-server[1105]: 2026/09/22 17:01:33 INFO Completed upload id=77 worker1 # [ 188.866137] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.482Z level=INFO msg="Upload complete. (292ms)" worker1 # [ 188.881222] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=1 worker1 # [ 188.896293] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 188.905524] nix-daemon[831]: reaped child process 1995, status = succeeded worker1 # [ 188.921291] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 188.936153] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 188.949882] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.566Z level=INFO msg="All 2 paths already cached" worker1 # [ 188.961076] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 188.972959] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 189.124869] nix-daemon[831]: accepted connection from pid 2020, user nix-grpc-daemon (trusted) worker1 # [ 189.141895] nix-daemon[831]: reaped child process 2023, status = succeeded lb # [ 189.148828] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.159779] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.776Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 189.161503] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.776Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 189.179435] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 189.187995] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.804Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.193895] niks3-server[1105]: 2026/09/22 17:01:33 INFO Signed narinfos id=78 count=1 lb # [ 189.201140] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 189.218036] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete worker1 # [ 189.224596] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.841Z level=INFO msg="Upload complete. (237ms)" lb # [ 189.226607] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 189.233816] niks3-server[1105]: 2026/09/22 17:01:33 INFO Completed upload id=78 worker1 # [ 189.234155] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 189.236459] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 189.250510] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 189.258141] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 189.263883] nix-daemon[831]: reaped child process 2018, status = succeeded worker1 # [ 189.276299] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted) worker1 # [ 190.787487] nix-daemon[831]: reaped child process 2027, status = succeeded worker1 # [ 190.862741] nix-daemon[831]: accepted connection from pid 2043, user nix-grpc-daemon (trusted) worker1 # [ 190.880302] nix-daemon[831]: reaped child process 2046, status = succeeded lb # [ 190.885306] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.898506] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.514Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 190.901036] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.514Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 190.927227] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign worker1 # [ 190.935647] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.552Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.942877] niks3-server[1105]: 2026/09/22 17:01:35 INFO Signed narinfos id=79 count=1 lb # [ 190.951435] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 190.961814] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 190.972160] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 190.976523] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 190.981796] niks3-server[1105]: 2026/09/22 17:01:35 INFO Completed upload id=79 worker1 # [ 190.981805] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.598Z level=INFO msg="Upload complete. (291ms)" lb # [ 190.984842] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 190.997252] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=1 worker1 # [ 191.015569] nix-daemon[831]: reaped child process 2025, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.18 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 191.230616] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 05864bdb2e3a4213838aa445ceeb0932 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 191.429507] nix[1576]: don't know how to build these paths: client # [ 191.430481] nix[1576]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 191.357139] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47258 worker1 # [ 191.393233] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 191.414674] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:36Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 191.458453] nix[1576]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 199.955997] envoy[919]: conn peer=[2001:db8:1::1]:53712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27153 tx=12353 ms=17900 client # [ 221.463673] nix[1576]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 221.476151] nix[1576]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 221.514726] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 221.522988] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 221.526822] systemd[1]: gpu.service: Consumed 189ms CPU time over 30.289s wall clock time, 35.4M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.08 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.01 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 223.187100] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:07Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.15 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 worker1 # [ 224.439345] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42994 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 224.454587] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42994 worker1 # [ 224.492267] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 224.499539] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 224.518349] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 224.536341] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47258 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 224.553557] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 224.569809] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 224.600627] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=info event=niks3_push_started pid=2024 worker2 # [ 224.753714] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.377Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 224.865215] nix-daemon[859]: accepted connection from pid 2037, user nix-grpc-daemon (trusted) worker2 # [ 224.883983] nix-daemon[859]: reaped child process 2040, status = succeeded lb # [ 224.922697] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 224.905092] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 224.906603] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.527Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 224.970235] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign worker2 # [ 224.952078] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.575Z level=INFO msg="Uploading 1 narinfos" lb # [ 224.985815] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 224.996219] niks3-server[1105]: 2026/09/22 17:02:09 INFO Signed narinfos id=80 count=1 lb # [ 224.998461] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 225.010277] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 225.015079] niks3-server[1105]: 2026/09/22 17:02:09 INFO Completed upload id=80 worker2 # [ 224.986375] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.610Z level=INFO msg="Upload complete. (228ms)" lb # [ 225.017292] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 225.000199] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 225.012079] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 225.017799] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 225.022778] nix-daemon[859]: reaped child process 2022, status = succeeded worker2 # [ 225.032725] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 226.112826] nix-daemon[859]: reaped child process 2044, status = succeeded worker2 # [ 226.215564] nix-daemon[859]: accepted connection from pid 2060, user nix-grpc-daemon (trusted) worker2 # [ 226.236984] nix-daemon[859]: reaped child process 2063, status = succeeded lb # [ 226.265034] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 226.248554] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 226.250257] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.871Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 226.317668] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst worker2 # [ 226.299662] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.923Z level=INFO msg="Uploading 1 narinfos" lb # [ 226.329971] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 226.340427] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 226.343460] niks3-server[1105]: 2026/09/22 17:02:10 INFO Signed narinfos id=81 count=1 lb # [ 226.349443] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 226.357579] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete worker2 # [ 226.327874] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.951Z level=INFO msg="Upload complete. (283ms)" lb # [ 226.360629] niks3-server[1105]: 2026/09/22 17:02:10 INFO Completed upload id=81 lb # [ 226.362951] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 226.341187] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=1 worker1 # [ 226.377136] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:10Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 226.382588] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:10Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 226.361230] nix-daemon[859]: reaped child process 2042, status = succeeded worker2 # [ 226.392137] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 226.407569] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47258 client # copying 1 paths to 'lb:50051'... worker2 # [ 226.422520] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.046Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 226.428925] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 226.442979] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 226.605478] nix-daemon[859]: accepted connection from pid 2067, user nix-grpc-daemon (trusted) worker2 # [ 226.625797] nix-daemon[859]: reaped child process 2070, status = succeeded lb # [ 226.652568] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 226.632940] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.256Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 226.634389] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.256Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 226.679474] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 226.665870] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.289Z level=INFO msg="Uploading 1 narinfos" lb # [ 226.693926] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 226.706273] niks3-server[1105]: 2026/09/22 17:02:11 INFO Signed narinfos id=82 count=1 lb # [ 226.713393] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 226.727300] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete worker2 # [ 226.706504] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.330Z level=INFO msg="Upload complete. (246ms)" lb # [ 226.735825] niks3-server[1105]: 2026/09/22 17:02:11 INFO Completed upload id=82 lb # [ 226.739638] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 226.716268] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 226.731571] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 226.740331] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 226.745210] nix-daemon[859]: reaped child process 2065, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 226.758282] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 228.185538] nix-daemon[859]: reaped child process 2074, status = succeeded worker2 # [ 228.282697] nix-daemon[859]: accepted connection from pid 2090, user nix-grpc-daemon (trusted) worker2 # [ 228.301110] nix-daemon[859]: reaped child process 2093, status = succeeded lb # [ 228.338354] niks3-server[1105]: 2026/09/22 17:02:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.321148] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.944Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 228.322628] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.944Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 228.382371] niks3-server[1105]: 2026/09/22 17:02:12 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker2 # [ 228.365223] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.988Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.396480] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 228.404766] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 228.414350] niks3-server[1105]: 2026/09/22 17:02:13 INFO Signed narinfos id=83 count=1 worker2 # [ 228.388097] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.012Z level=INFO msg="Upload complete. (216ms)" lb # [ 228.418816] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 228.422959] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 228.426195] niks3-server[1105]: 2026/09/22 17:02:13 INFO Completed upload id=83 lb # [ 228.427608] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 228.396931] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=1 worker1 # [ 228.436192] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:13Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 228.415330] nix-daemon[859]: reaped child process 2072, status = succeeded worker1 # [ 228.441741] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:13Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 228.452190] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 228.468128] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 228.484087] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.107Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 228.495568] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 228.509329] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 228.670239] nix-daemon[859]: accepted connection from pid 2097, user nix-grpc-daemon (trusted) worker2 # [ 228.692274] nix-daemon[859]: reaped child process 2100, status = succeeded lb # [ 228.719724] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.701295] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 228.702753] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.325Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" worker2 # [ 228.723865] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.347Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.752084] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 228.765803] niks3-server[1105]: 2026/09/22 17:02:13 INFO Signed narinfos id=84 count=1 lb # [ 228.773573] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst lb # [ 228.786778] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 228.763345] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.387Z level=INFO msg="Upload complete. (237ms)" lb # [ 228.795745] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 228.800917] niks3-server[1105]: 2026/09/22 17:02:13 INFO Completed upload id=84 worker2 # [ 228.772345] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 228.804565] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 228.788756] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 worker2 # [ 228.795714] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 228.801505] nix-daemon[859]: reaped child process 2095, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 228.814888] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain lb # [ 229.956659] envoy[919]: conn peer=[2001:db8:1::1]:51160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=579 ms=30065 worker2 # [ 230.100278] nix-daemon[859]: reaped child process 2104, status = succeeded worker2 # [ 230.191121] nix-daemon[859]: accepted connection from pid 2120, user nix-grpc-daemon (trusted) worker2 # [ 230.210648] nix-daemon[859]: reaped child process 2123, status = succeeded lb # [ 230.239345] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 230.220535] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.843Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 230.222059] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.843Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 230.281127] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker2 # [ 230.259852] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.883Z level=INFO msg="Uploading 1 narinfos" lb # [ 230.297538] niks3-server[1105]: 2026/09/22 17:02:14 INFO Signed narinfos id=85 count=1 lb # [ 230.308470] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 230.317591] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 230.300624] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.924Z level=INFO msg="Upload complete. (275ms)" lb # [ 230.330189] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 230.335788] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 230.340896] niks3-server[1105]: 2026/09/22 17:02:14 INFO Completed upload id=85 worker2 # [ 230.313144] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=1 lb # [ 230.343431] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 230.330850] nix-daemon[859]: reaped child process 2102, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.19 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 231.190516] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:15Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 10.03 seconds) (finished: run the VM test script, in 232.81 seconds) test script finished in 233.06s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.11 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh