tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker1 # Disk image does not exist, creating the virtualisation disk image... worker1 # Formatting '/build/vm-state-worker1/tmp.TK9Rze6tJd', fmt=raw size=1073741824 worker1 # mke2fs 1.47.4 (6-Mar-2025) client # Disk image does not exist, creating the virtualisation disk image... worker1 # Discarding device blocks: 0/262144 done client: QEMU running (pid 49) client # Formatting '/build/vm-state-client/tmp.25bTFcZ9qK', fmt=raw size=1073741824 worker1: QEMU running (pid 51) lb # Disk image does not exist, creating the virtualisation disk image... worker2 # Disk image does not exist, creating the virtualisation disk image... worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes lb: QEMU running (pid 50) client # mke2fs 1.47.4 (6-Mar-2025) worker2: QEMU running (pid 52) lb # Formatting '/build/vm-state-lb/tmp.IYbqLiibSh', fmt=raw size=1073741824 client # Discarding device blocks: 0/262144 done worker2 # Formatting '/build/vm-state-worker2/tmp.CdW84R8xjt', fmt=raw size=1073741824 worker1 # Filesystem UUID: 07f97f2e-1a27-4afe-86d2-1a582037d5c9 (finished: start all VMs, in 0.50 seconds) worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # Superblock backups stored on blocks: client: waiting for unit network-addresses-eth1.service worker2 # Discarding device blocks: 0/262144 done worker1 # 32768, 98304, 163840, 229376 client: waiting for the VM to finish booting worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # Filesystem UUID: 28eb8080-5cf1-4418-abb6-e32a14f1f74c worker1 # Allocating group tables: 0/8 done client # Filesystem UUID: c916ece1-5f76-4c92-b04d-2db56ad83b45 lb # Discarding device blocks: 0/262144 done worker2 # Superblock backups stored on blocks: worker1 # Writing inode tables: 0/8 done client # Superblock backups stored on blocks: lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # 32768, 98304, 163840, 229376 worker1 # Creating journal (8192 blocks): done client # 32768, 98304, 163840, 229376 lb # Filesystem UUID: 59281576-d49d-44d1-811e-7aef8f48f1ce worker2 # worker1 # Writing superblocks and filesystem accounting information: 0/8 done client # lb # Superblock backups stored on blocks: worker2 # Allocating group tables: 0/8 done worker1 # client # Allocating group tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # Writing inode tables: 0/8 done worker1 # Virtualisation disk image created. client # Writing inode tables: 0/8 done lb # worker2 # Creating journal (8192 blocks): done worker1 # Starting virtiofs daemons... client # Creating journal (8192 blocks): done lb # Allocating group tables: 0/8 done worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # worker2 # worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Writing inode tables: 0/8 done client # Virtualisation disk image created. worker2 # Virtualisation disk image created. worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Creating journal (8192 blocks): done client # Starting virtiofs daemons... worker2 # Starting virtiofs daemons... worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Writing superblocks and filesystem accounting information: 0/8 done client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Virtualisation disk image created. client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Starting virtiofs daemons... client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] random: crng init done worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Normal empty worker1 # [ 0.000000] Device empty worker1 # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test/init regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000033] arm-pv: using stolen time PV worker1 # [ 0.000645] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000802] Console: colour dummy device 80x25 worker1 # [ 0.000809] printk: legacy console [tty0] enabled worker1 # [ 0.001022] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.001029] pid_max: default: 32768 minimum: 301 worker1 # [ 0.001107] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.001280] landlock: Up and running. worker1 # [ 0.001283] Yama: becoming mindful. worker1 # [ 0.001872] LSM support for eBPF active worker1 # [ 0.002026] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.002059] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.003931] rcu: Hierarchical SRCU implementation. worker1 # [ 0.003935] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.005168] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.005257] EFI services will not be available. worker1 # [ 0.005334] smp: Bringing up secondary CPUs ... worker1 # [ 0.005342] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled worker1 # [ 0.005345] SMP: Total of 1 processors activated. worker2 # [ 0.000000] random: crng init done worker1 # [ 0.005350] CPU: All CPU(s) started at EL1 worker2 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.005360] CPU features: detected: Branch Target Identification worker2 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.005365] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005368] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: worker1 # [ 0.005372] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005376] CPU features: detected: Common not Private translations lb # [ 0.000000] KASLR enabled worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.005379] CPU features: detected: CRC32 instructions lb # [ 0.000000] random: crng init done worker2 # [ 0.000000] Normal empty worker2 # [ 0.000000] Device empty lb # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.005382] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Early memory node ranges worker1 # [ 0.005385] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] KASLR enabled worker1 # [ 0.005388] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] random: crng init done worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005391] CPU features: detected: E0PD client # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] Zone ranges: worker1 # [ 0.005394] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] DMA32 empty worker1 # [ 0.005397] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.000000] Normal empty worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005400] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] Device empty worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] Movable zone start for each node worker1 # [ 0.005403] CPU features: detected: Fine Grained Traps client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] Early memory node ranges worker1 # [ 0.005406] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.005411] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.000000] Normal empty lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Device empty worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.005414] CPU features: detected: LSE atomic instructions lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.005417] CPU features: detected: Privileged Access Never client # [ 0.000000] Early memory node ranges lb # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.005419] CPU features: detected: PMUv3 worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005422] CPU features: detected: RAS Extension Support worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005425] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.005428] CPU features: detected: Random Number Generator lb # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.005430] CPU features: detected: Speculation barrier (SB) lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.005433] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.005436] CPU features: detected: TLB range maintenance instructions lb # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.005441] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.005480] alternatives: applying system-wide alternatives lb # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.008358] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test/init regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.008519] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.008889] devtmpfs: initialized client # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.010640] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration", will be passed to user space. client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.010662] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.010840] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test/init regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space. lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.010844] 0 pages in range for non-PLT usage worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.010845] 508272 pages in range for PLT usage client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.010968] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.011739] DMI not present or invalid. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.015398] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] Policy zone: DMA client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.017551] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.017728] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.017926] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.017947] audit: initializing netlink subsys (disabled) client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.018504] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.018506] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.018509] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.018512] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.018517] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.018543] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.018552] cpuidle: using governor ladder client # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.018558] cpuidle: using governor menu client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.018753] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.018768] ASID allocator initialised with 65536 entries worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.020021] Serial: AMBA PL011 UART driver worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.025384] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.025534] printk: console [ttyAMA0] enabled client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.147082] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.147098] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.147103] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.147107] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.147111] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.147116] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.147120] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.147124] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker2 # [ 0.000029] arm-pv: using stolen time PV client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 worker2 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.154225] fbcon: Taking over console lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000558] Console: colour dummy device 80x25 worker1 # [ 0.154236] ACPI: Interpreter disabled. client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000566] printk: legacy console [tty0] enabled worker1 # [ 0.156078] iommu: Default domain type: Translated lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.156087] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000761] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000767] pid_max: default: 32768 minimum: 301 client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.157695] SCSI subsystem initialized client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000838] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000975] landlock: Up and running. worker2 # [ 0.000978] Yama: becoming mindful. lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.001414] LSM support for eBPF active lb # [ 0.000027] arm-pv: using stolen time PV client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.001547] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000400] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000028] arm-pv: using stolen time PV worker2 # [ 0.001579] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000595] Console: colour dummy device 80x25 worker2 # [ 0.003300] rcu: Hierarchical SRCU implementation. lb # [ 0.000602] printk: legacy console [tty0] enabled client # [ 0.000361] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.003305] rcu: Max phase no-delay instances is 1000. client # [ 0.000524] Console: colour dummy device 80x25 worker2 # [ 0.004518] fsl-mc MSI: its@8080000 domain created client # [ 0.000531] printk: legacy console [tty0] enabled lb # [ 0.000786] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.004607] EFI services will not be available. lb # [ 0.000793] pid_max: default: 32768 minimum: 301 worker2 # [ 0.004670] smp: Bringing up secondary CPUs ... client # [ 0.000723] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.004678] smp: Brought up 1 node, 1 CPU lb # [ 0.001006] landlock: Up and running. client # [ 0.000729] pid_max: default: 32768 minimum: 301 worker2 # [ 0.004681] SMP: Total of 1 processors activated. lb # [ 0.001008] Yama: becoming mindful. client # [ 0.000810] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.004686] CPU: All CPU(s) started at EL1 lb # [ 0.001446] LSM support for eBPF active worker1 # [ 0.165264] usbcore: registered new interface driver usbfs client # [ 0.000934] landlock: Up and running. worker2 # [ 0.004696] CPU features: detected: Branch Target Identification client # [ 0.000937] Yama: becoming mindful. worker1 # [ 0.165297] usbcore: registered new interface driver hub lb # [ 0.001601] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.001344] LSM support for eBPF active worker2 # [ 0.004699] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.165313] usbcore: registered new device driver usb lb # [ 0.001633] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.165577] pps_core: LinuxPPS API ver. 1 registered client # [ 0.001480] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.003412] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004702] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.001513] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.003416] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.165583] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.165593] PTP clock support registered worker2 # [ 0.004706] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.003315] rcu: Hierarchical SRCU implementation. lb # [ 0.004624] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.165659] EDAC MC: Ver: 3.0.0 client # [ 0.003319] rcu: Max phase no-delay instances is 1000. lb # [ 0.004720] EFI services will not be available. worker2 # [ 0.004710] CPU features: detected: Common not Private translations client # [ 0.004590] fsl-mc MSI: its@8080000 domain created lb # [ 0.004795] smp: Bringing up secondary CPUs ... worker2 # [ 0.004713] CPU features: detected: CRC32 instructions worker1 # [ 0.170086] scmi_core: SCMI protocol bus registered lb # [ 0.004803] smp: Brought up 1 node, 1 CPU client # [ 0.004678] EFI services will not be available. worker2 # [ 0.004716] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.004806] SMP: Total of 1 processors activated. client # [ 0.004748] smp: Bringing up secondary CPUs ... worker1 # [ 0.171027] FPGA manager framework lb # [ 0.004811] CPU: All CPU(s) started at EL1 client # [ 0.004756] smp: Brought up 1 node, 1 CPU worker1 # [ 0.171974] vgaarb: loaded worker2 # [ 0.004720] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.004760] SMP: Total of 1 processors activated. lb # [ 0.004821] CPU features: detected: Branch Target Identification client # [ 0.004765] CPU: All CPU(s) started at EL1 worker2 # [ 0.004723] CPU features: detected: Data independent timing control (DIT) lb # [ 0.004824] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004726] CPU features: detected: E0PD worker1 # [ 0.172564] clocksource: Switched to clocksource arch_sys_counter client # [ 0.004776] CPU features: detected: Branch Target Identification worker1 # [ 0.173126] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.004728] CPU features: detected: Enhanced Counter Virtualization client # [ 0.004781] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.004828] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.173153] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.004731] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.004831] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.004784] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004735] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.175394] netfs: FS-Cache loaded worker1 # [ 0.175502] pnp: PnP ACPI: disabled lb # [ 0.004835] CPU features: detected: Common not Private translations worker2 # [ 0.004738] CPU features: detected: Fine Grained Traps client # [ 0.004788] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.004839] CPU features: detected: CRC32 instructions client # [ 0.004792] CPU features: detected: Common not Private translations worker2 # [ 0.004742] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.004842] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.004795] CPU features: detected: CRC32 instructions worker2 # [ 0.004747] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.004845] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.004750] CPU features: detected: LSE atomic instructions client # [ 0.004798] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004753] CPU features: detected: Privileged Access Never lb # [ 0.004848] CPU features: detected: Data independent timing control (DIT) client # [ 0.004801] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.004756] CPU features: detected: PMUv3 lb # [ 0.004852] CPU features: detected: E0PD client # [ 0.004804] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004759] CPU features: detected: RAS Extension Support lb # [ 0.004854] CPU features: detected: Enhanced Counter Virtualization client # [ 0.004807] CPU features: detected: E0PD worker2 # [ 0.004761] CPU features: detected: RASv1p1 Extension Support lb # [ 0.004857] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.004810] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004764] CPU features: detected: Random Number Generator lb # [ 0.004861] CPU features: detected: Enhanced Virtualization Traps client # [ 0.004813] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004767] CPU features: detected: Speculation barrier (SB) lb # [ 0.004864] CPU features: detected: Fine Grained Traps client # [ 0.004816] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.004769] CPU features: detected: Stage-2 Force Write-Back client # [ 0.004820] CPU features: detected: Fine Grained Traps lb # [ 0.004867] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.004772] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.004777] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.004823] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.181411] NET: Registered PF_INET protocol family worker2 # [ 0.004814] alternatives: applying system-wide alternatives client # [ 0.004828] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.004831] CPU features: detected: LSE atomic instructions worker1 # [ 0.181667] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.007660] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.004834] CPU features: detected: Privileged Access Never client # [ 0.004836] CPU features: detected: PMUv3 worker2 # [ 0.007867] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) client # [ 0.004839] CPU features: detected: RAS Extension Support worker2 # [ 0.008264] devtmpfs: initialized client # [ 0.004842] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.010022] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.004844] CPU features: detected: Random Number Generator client # [ 0.004847] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.010043] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.004850] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.010212] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.010216] 0 pages in range for non-PLT usage client # [ 0.004852] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.010217] 508272 pages in range for PLT usage client # [ 0.004857] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.004872] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.010304] pinctrl core: initialized pinctrl subsystem client # [ 0.004891] alternatives: applying system-wide alternatives worker2 # [ 0.011012] DMI not present or invalid. lb # [ 0.004875] CPU features: detected: LSE atomic instructions client # [ 0.007762] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.013975] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.004878] CPU features: detected: Privileged Access Never lb # [ 0.004881] CPU features: detected: PMUv3 worker2 # [ 0.016348] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.004884] CPU features: detected: RAS Extension Support client # [ 0.007961] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker2 # [ 0.016535] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.004887] CPU features: detected: RASv1p1 Extension Support client # [ 0.008385] devtmpfs: initialized lb # [ 0.004889] CPU features: detected: Random Number Generator worker2 # [ 0.016728] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.010159] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.016747] audit: initializing netlink subsys (disabled) lb # [ 0.004892] CPU features: detected: Speculation barrier (SB) client # [ 0.010181] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.004894] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.017283] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.010349] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.004897] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.017285] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.010353] 0 pages in range for non-PLT usage worker2 # [ 0.017288] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.004902] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.010354] 508272 pages in range for PLT usage worker2 # [ 0.017291] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.004935] alternatives: applying system-wide alternatives client # [ 0.010448] pinctrl core: initialized pinctrl subsystem client # [ 0.011204] DMI not present or invalid. worker2 # [ 0.017296] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.007824] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.014291] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.017325] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.017335] cpuidle: using governor ladder client # [ 0.016432] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.008086] Memory: 1405696K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) worker2 # [ 0.017341] cpuidle: using governor menu lb # [ 0.008457] devtmpfs: initialized client # [ 0.016624] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.017515] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.010263] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.016821] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.017530] ASID allocator initialised with 65536 entries worker2 # [ 0.018723] Serial: AMBA PL011 UART driver client # [ 0.016842] audit: initializing netlink subsys (disabled) lb # [ 0.010290] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.017360] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.010463] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.017362] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.010467] 0 pages in range for non-PLT usage worker2 # [ 0.023949] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.010469] 508272 pages in range for PLT usage worker2 # [ 0.024120] printk: console [ttyAMA0] enabled client # [ 0.017366] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.010576] pinctrl core: initialized pinctrl subsystem client # [ 0.017368] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.011355] DMI not present or invalid. client # [ 0.017373] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.014466] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.017396] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.016665] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.017407] cpuidle: using governor ladder client # [ 0.017412] cpuidle: using governor menu lb # [ 0.016838] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.017596] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.017041] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.017610] ASID allocator initialised with 65536 entries lb # [ 0.017062] audit: initializing netlink subsys (disabled) client # [ 0.018781] Serial: AMBA PL011 UART driver lb # [ 0.017586] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.023908] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.017588] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.024039] printk: console [ttyAMA0] enabled worker2 # [ 0.140753] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.017592] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.140767] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.017594] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.140773] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.017600] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.140777] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.017622] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.140782] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.017632] cpuidle: using governor ladder lb # [ 0.017638] cpuidle: using governor menu worker2 # [ 0.140786] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.209860] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.017820] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.140790] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.209895] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.017833] ASID allocator initialised with 65536 entries worker2 # [ 0.140794] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.019038] Serial: AMBA PL011 UART driver worker1 # [ 0.209919] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.024307] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.209988] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.024452] printk: console [ttyAMA0] enabled client # [ 0.140151] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.210131] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.148110] fbcon: Taking over console client # [ 0.140166] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.148123] ACPI: Interpreter disabled. worker1 # [ 0.210201] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.144565] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.140172] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.210233] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.144583] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.140176] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.210268] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.144589] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.140181] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.210364] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.144593] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.140185] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.210383] NET: Registered PF_XDP protocol family lb # [ 0.144597] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.140190] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.210398] PCI: CLS 0 bytes, default 64 lb # [ 0.144602] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.210610] Trying to unpack rootfs image as initramfs... client # [ 0.140194] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.144606] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.144611] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.147345] fbcon: Taking over console client # [ 0.147358] ACPI: Interpreter disabled. lb # [ 0.151885] fbcon: Taking over console worker2 # [ 0.156966] iommu: Default domain type: Translated lb # [ 0.151897] ACPI: Interpreter disabled. worker2 # [ 0.156976] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.157357] SCSI subsystem initialized worker2 # [ 0.159334] usbcore: registered new interface driver usbfs worker2 # [ 0.159365] usbcore: registered new interface driver hub worker1 # [ 0.226025] kvm [1]: HYP mode not available worker2 # [ 0.159380] usbcore: registered new device driver usb worker2 # [ 0.159652] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.159658] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.159668] PTP clock support registered worker2 # [ 0.159722] EDAC MC: Ver: 3.0.0 lb # [ 0.158918] iommu: Default domain type: Translated lb # [ 0.158926] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.156340] iommu: Default domain type: Translated worker2 # [ 0.164294] scmi_core: SCMI protocol bus registered lb # [ 0.160567] SCSI subsystem initialized client # [ 0.156349] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.165237] FPGA manager framework client # [ 0.156677] SCSI subsystem initialized worker2 # [ 0.166207] vgaarb: loaded lb # [ 0.161238] usbcore: registered new interface driver usbfs worker2 # [ 0.166822] clocksource: Switched to clocksource arch_sys_counter client # [ 0.158642] usbcore: registered new interface driver usbfs lb # [ 0.161270] usbcore: registered new interface driver hub client # [ 0.158673] usbcore: registered new interface driver hub lb # [ 0.161286] usbcore: registered new device driver usb client # [ 0.158688] usbcore: registered new device driver usb lb # [ 0.161561] pps_core: LinuxPPS API ver. 1 registered client # [ 0.158960] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.161567] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.161578] PTP clock support registered client # [ 0.158966] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.161630] EDAC MC: Ver: 3.0.0 client # [ 0.158976] PTP clock support registered worker2 # [ 0.170434] VFS: Disk quotas dquot_6.6.0 client # [ 0.159033] EDAC MC: Ver: 3.0.0 lb # [ 0.166272] scmi_core: SCMI protocol bus registered worker2 # [ 0.170468] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.163649] scmi_core: SCMI protocol bus registered lb # [ 0.167236] FPGA manager framework lb # [ 0.168230] vgaarb: loaded client # [ 0.164610] FPGA manager framework client # [ 0.165543] vgaarb: loaded lb # [ 0.168842] clocksource: Switched to clocksource arch_sys_counter client # [ 0.166167] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.174200] netfs: FS-Cache loaded worker2 # [ 0.174315] pnp: PnP ACPI: disabled lb # [ 0.172121] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.178205] NET: Registered PF_INET protocol family lb # [ 0.172156] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.178463] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.172327] VFS: Disk quotas dquot_6.6.0 lb # [ 0.175747] netfs: FS-Cache loaded lb # [ 0.175878] pnp: PnP ACPI: disabled client # [ 0.172371] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.174027] netfs: FS-Cache loaded client # [ 0.174150] pnp: PnP ACPI: disabled lb # [ 0.179640] NET: Registered PF_INET protocol family client # [ 0.177915] NET: Registered PF_INET protocol family lb # [ 0.179890] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.178170] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.208388] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.208429] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.207136] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.208454] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.207176] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.208523] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.207201] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.208666] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.207270] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.208744] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.207413] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.208777] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207482] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.208812] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207514] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207549] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207648] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.206974] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.207668] NET: Registered PF_XDP protocol family worker2 # [ 0.207684] PCI: CLS 0 bytes, default 64 client # [ 0.207014] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.207915] Trying to unpack rootfs image as initramfs... client # [ 0.207038] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.207109] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.207251] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.207334] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.207370] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.207405] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.207499] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.216242] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.207524] NET: Registered PF_XDP protocol family lb # [ 0.216266] NET: Registered PF_XDP protocol family client # [ 0.207540] PCI: CLS 0 bytes, default 64 lb # [ 0.216283] PCI: CLS 0 bytes, default 64 worker2 # [ 0.224483] kvm [1]: HYP mode not available client # [ 0.207777] Trying to unpack rootfs image as initramfs... lb # [ 0.216512] Trying to unpack rootfs image as initramfs... lb # [ 0.227297] kvm [1]: HYP mode not available worker1 # [ 0.309016] Initialise system trusted keyrings client # [ 0.223871] kvm [1]: HYP mode not available worker1 # [ 0.309718] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.316847] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.317652] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.337697] Key type asymmetric registered worker1 # [ 0.337715] Asymmetric key parser 'x509' registered worker1 # [ 0.337770] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.339860] io scheduler mq-deadline registered worker1 # [ 0.339870] io scheduler kyber registered worker1 # [ 0.353001] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.354349] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.354749] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.354765] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.354777] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.354784] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.354810] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.354831] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.354908] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.354918] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.354923] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.354928] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.354933] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.355017] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.355442] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.355636] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.355653] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.355682] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.355697] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.356141] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.356319] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.356335] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.356363] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.375823] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.376005] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.376021] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.376050] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.376493] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.315395] Initialise system trusted keyrings worker2 # [ 0.316145] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.317375] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.318160] 9p: Installing v9fs 9p2000 file system support client # [ 0.314694] Initialise system trusted keyrings client # [ 0.315444] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.316752] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.388764] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.388797] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.317533] 9p: Installing v9fs 9p2000 file system support lb # [ 0.321412] Initialise system trusted keyrings worker1 # [ 0.388827] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.322154] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.388845] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.323477] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.389338] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.389528] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.324252] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.389558] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.390006] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.390188] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.390216] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.390592] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.390771] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.391063] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.391248] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391277] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.391731] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.391912] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.338813] Key type asymmetric registered worker1 # [ 0.391941] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392378] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.392559] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.392600] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.393044] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.393310] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.393327] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.393355] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.393819] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.338133] Key type asymmetric registered worker1 # [ 0.393997] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.338156] Asymmetric key parser 'x509' registered worker2 # [ 0.346871] Asymmetric key parser 'x509' registered worker1 # [ 0.394012] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.346956] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.394041] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.348607] io scheduler mq-deadline registered worker1 # [ 0.394612] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.348619] io scheduler kyber registered worker1 # [ 0.394623] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.394628] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.394673] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.394719] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.348259] Key type asymmetric registered worker1 # [ 0.394766] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.348284] Asymmetric key parser 'x509' registered client # [ 0.346252] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.394813] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.348348] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.347306] io scheduler mq-deadline registered worker1 # [ 0.394857] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.347317] io scheduler kyber registered worker1 # [ 0.394902] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.394948] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.394993] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.395038] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.358960] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.395103] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.395147] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.395168] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.395189] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.360367] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.395210] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.360745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.395231] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.360762] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.395252] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.356895] io scheduler mq-deadline registered lb # [ 0.356914] io scheduler kyber registered worker2 # [ 0.360783] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.395273] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.395297] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.358313] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.360792] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.362032] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.395319] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.360819] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.395340] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.360843] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.395362] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.363304] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.360914] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.359697] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.395383] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.363679] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.360923] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.395404] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.360126] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.360929] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.395425] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.363696] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.360143] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.360935] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.395445] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.363708] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.395466] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.360940] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.360156] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.363716] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.395487] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.361000] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.360165] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.395508] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.363736] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.361439] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.395532] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.360185] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.361625] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.363763] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.395541] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.360212] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.361641] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.395546] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.363832] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.360287] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.361671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363841] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.396358] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.360296] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.361687] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.363847] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.360302] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.362153] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.363852] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.360307] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.362335] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.363858] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.360312] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.362351] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.363910] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.360366] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.362380] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364345] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.360806] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.364531] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.360994] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.452694] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55019 usecs lb # [ 0.364547] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.361011] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.381357] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.364577] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.381547] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.361041] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364594] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.381562] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.361057] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.381592] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.361519] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.361705] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.382052] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.361721] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.382236] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.382251] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.361750] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.382281] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.382299] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.382762] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.380557] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.380744] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.380759] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.380789] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.381247] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.381430] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.381446] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.381475] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.381494] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.381952] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.385137] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.382139] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.385339] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.385357] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.385387] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.385851] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.386034] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.386050] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.394996] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.386079] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.395037] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.386531] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.395524] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.386715] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.473666] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.395715] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.386731] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.475875] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.394203] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.395745] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.386760] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.394724] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.386779] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.396150] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.394927] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.396331] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.387243] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.394957] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.387430] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.396619] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.396806] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.395401] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.387460] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.395585] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.396835] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.387911] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.395833] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.397283] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.388098] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.396050] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.397469] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388127] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.396089] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.397499] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388519] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.396565] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.397949] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.388700] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.396752] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.398136] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.389007] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.396782] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.398166] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.389195] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.397233] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.398622] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.389224] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.397419] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.398906] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.486870] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.389672] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.398922] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.397449] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.389857] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.398951] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.397926] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.389894] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.399404] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.390353] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399586] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.390540] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.399601] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.399631] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.390569] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400209] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.391023] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.391307] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.400220] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.391324] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400225] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.391354] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400268] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.391811] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.492741] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.400314] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.391993] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.494695] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.392009] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400361] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.392038] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400408] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.392618] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.400454] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.392630] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.400500] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.392635] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.400546] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.418295] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.392680] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.400591] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.418325] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.392725] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.418355] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400637] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.392771] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.418852] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.400714] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.419047] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.392817] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.400765] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.419064] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400787] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.419094] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400808] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.419698] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.400829] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.504749] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.419709] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.400850] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.506747] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.419714] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.400871] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.508514] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.400892] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.419761] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.400916] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.419809] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.400938] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.419856] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.400959] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.419905] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.400981] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.419953] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.401003] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.401024] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.420011] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.401045] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.420060] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.401066] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.420107] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.401086] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.401107] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.420158] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.401128] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.420288] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.401153] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.518642] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.420357] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.401162] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.440955] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.420379] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.401167] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.420403] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.441023] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.401965] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.420424] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.441074] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.420446] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.441125] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.420470] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.441173] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.420492] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.441318] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.420520] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.441369] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.420542] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.441393] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.420565] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.441416] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.420587] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.524669] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.441438] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.420610] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.527034] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.420632] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.441461] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned client # [ 0.420653] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.441484] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.420674] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.441506] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.420695] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.441534] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.420716] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.441555] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.420737] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.441577] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.420765] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.441599] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.420774] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.441622] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.420778] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.441643] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.421600] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.441664] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.441685] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.441706] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.441727] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.441748] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.441782] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.441792] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.441797] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.442637] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.545865] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.478466] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.548686] msm_serial: driver initialized worker1 # [ 0.548823] SuperH (H)SCI(F) driver initialized worker1 # [ 0.548878] STM32 USART driver initialized client # [ 0.477224] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.488298] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.487007] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.492847] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.494793] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.487631] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.497111] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.500779] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.508536] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.581806] loop: module loaded worker1 # [ 0.582016] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.510676] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.582814] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.498844] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.500925] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.514098] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.503021] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.505237] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.510314] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.516832] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.589271] megasas: 07.734.00.00-rc1 lb # [ 0.514961] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.590004] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.592395] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.592465] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.515463] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.517325] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.527048] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.521026] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.529002] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.522848] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.601632] Using buffer write method lb # [ 0.524734] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.601720] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.528463] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.531243] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.533579] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.612621] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.539090] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.612657] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.614315] Using buffer write method worker1 # [ 0.614350] Concatenating MTD devices: worker1 # [ 0.614355] (0): "0.flash" worker1 # [ 0.614365] (1): "0.flash" worker1 # [ 0.614369] into device "0.flash" lb # [ 0.538417] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.540346] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.552152] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.546457] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.552179] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.554951] msm_serial: driver initialized worker2 # [ 0.555099] SuperH (H)SCI(F) driver initialized worker2 # [ 0.555152] STM32 USART driver initialized client # [ 0.555604] msm_serial: driver initialized client # [ 0.555768] SuperH (H)SCI(F) driver initialized client # [ 0.555822] STM32 USART driver initialized lb # [ 0.562102] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.564953] msm_serial: driver initialized lb # [ 0.565108] SuperH (H)SCI(F) driver initialized lb # [ 0.565165] STM32 USART driver initialized worker2 # [ 0.587446] loop: module loaded worker2 # [ 0.587641] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.588422] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.585011] loop: module loaded client # [ 0.585201] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.586036] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.595511] megasas: 07.734.00.00-rc1 worker2 # [ 0.596459] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.598690] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.598716] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.599607] loop: module loaded worker2 # [ 0.600411] Using buffer write method lb # [ 0.599795] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.600483] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.600605] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.602216] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.602240] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.598879] megasas: 07.734.00.00-rc1 client # [ 0.599589] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.601860] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.601887] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.603435] Using buffer write method client # [ 0.603498] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.605283] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.620872] Using buffer write method client # [ 0.605307] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.620908] Concatenating MTD devices: worker2 # [ 0.620912] (0): "0.flash" worker2 # [ 0.620916] (1): "0.flash" worker2 # [ 0.620924] into device "0.flash" lb # [ 0.613516] megasas: 07.734.00.00-rc1 lb # [ 0.614234] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.618771] Using buffer write method client # [ 0.618797] Concatenating MTD devices: client # [ 0.618801] (0): "0.flash" lb # [ 0.616430] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.618805] (1): "0.flash" client # [ 0.618812] into device "0.flash" lb # [ 0.616454] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.618022] Using buffer write method lb # [ 0.618087] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.619722] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.619745] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.638469] Using buffer write method lb # [ 0.638505] Concatenating MTD devices: lb # [ 0.638510] (0): "0.flash" lb # [ 0.638514] (1): "0.flash" lb # [ 0.638524] into device "0.flash" worker1 # [ 0.882601] Freeing initrd memory: 26976K worker1 # [ 0.888899] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.892737] thunder_xcv, ver 1.0 worker1 # [ 0.892782] thunder_bgx, ver 1.0 worker1 # [ 0.892805] nicpf, ver 1.0 worker1 # [ 0.893369] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.893376] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.893402] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.893411] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.893440] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.893446] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.893472] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.893478] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.893614] sky2: driver version 1.30 worker1 # [ 0.895230] usbcore: registered new interface driver usb-storage worker1 # [ 0.895314] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.895330] usbserial: USB Serial support registered for generic worker1 # [ 0.896009] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.897338] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.897369] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.897597] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.909739] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.910079] hub 1-0:1.0: USB hub found worker1 # [ 0.910100] hub 1-0:1.0: 6 ports detected worker1 # [ 0.912416] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.912444] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520) worker1 # [ 0.914491] i2c_dev: i2c /dev entries driver worker1 # [ 0.918146] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.918158] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.918418] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.918793] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.920329] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.923850] usbcore: registered new interface driver usbhid worker1 # [ 0.923860] usbhid: USB HID core driver worker1 # [ 0.926131] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.927654] drop_monitor: Initializing network drop monitor service worker1 # [ 0.927831] NET: Registered PF_INET6 protocol family worker1 # [ 0.930851] Segment Routing with IPv6 worker1 # [ 0.930867] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.930895] NET: Registered PF_PACKET protocol family worker1 # [ 0.932504] 9pnet: Installing 9P2000 support worker1 # [ 0.932550] Key type dns_resolver registered worker1 # [ 0.939687] registered taskstats version 1 worker1 # [ 0.939873] Loading compiled-in X.509 certificates worker2 # [ 0.871288] Freeing initrd memory: 26944K worker1 # [ 0.948495] Demotion targets for Node 0: null worker2 # [ 0.877324] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.949333] Key type .fscrypt registered worker1 # [ 0.949344] Key type fscrypt-provisioning registered worker1 # [ 0.949445] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.949465] ima: Allocated hash algorithm: sha1 worker1 # [ 0.949487] ima: No architecture policies found worker1 # [ 0.952962] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.881008] thunder_xcv, ver 1.0 worker2 # [ 0.881047] thunder_bgx, ver 1.0 worker2 # [ 0.881070] nicpf, ver 1.0 worker2 # [ 0.881621] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.881627] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.877325] Freeing initrd memory: 26972K worker2 # [ 0.881651] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.881660] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.881706] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.881712] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.881748] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.881754] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.881890] sky2: driver version 1.30 lb # [ 0.886284] Freeing initrd memory: 26948K worker2 # [ 0.890390] usbcore: registered new interface driver usb-storage client # [ 0.883433] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.890482] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.890496] usbserial: USB Serial support registered for generic worker2 # [ 0.893760] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.893788] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.894053] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.892349] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.896764] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.898271] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.887259] thunder_xcv, ver 1.0 client # [ 0.887301] thunder_bgx, ver 1.0 client # [ 0.887322] nicpf, ver 1.0 worker2 # [ 0.898296] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520) client # [ 0.887873] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.898597] i2c_dev: i2c /dev entries driver worker1 # [ 0.971552] clk: Disabling unused clocks client # [ 0.887880] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.971595] PM: genpd: Disabling unused power domains lb # [ 0.896066] thunder_xcv, ver 1.0 client # [ 0.887904] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.896110] thunder_bgx, ver 1.0 client # [ 0.887912] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.896133] nicpf, ver 1.0 lb # [ 0.896689] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.887940] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.896696] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.887945] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.896721] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.888004] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.975751] Freeing unused kernel memory: 4736K lb # [ 0.896729] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.975969] Run /init as init process client # [ 0.888010] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.896760] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.888148] sky2: driver version 1.30 lb # [ 0.896765] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.889738] usbcore: registered new interface driver usb-storage lb # [ 0.896804] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.889785] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.904033] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.904044] sdhci: Copyright(c) Pierre Ossman lb # [ 0.896810] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.889798] usbserial: USB Serial support registered for generic worker2 # [ 0.904306] Synopsys Designware Multimedia Card Interface Driver client # [ 0.890883] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.904674] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.890907] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.903601] sky2: driver version 1.30 worker2 # [ 0.906163] hid: raw HID events driver (C) Jiri Kosina client # [ 0.891186] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.906410] usbcore: registered new interface driver usbhid worker2 # [ 0.906416] usbhid: USB HID core driver lb # [ 0.905940] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.902688] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.902993] hub 1-0:1.0: USB hub found lb # [ 0.905968] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.911294] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.903019] hub 1-0:1.0: 6 ports detected worker2 # [ 0.911608] hub 1-0:1.0: USB hub found lb # [ 0.906193] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.911627] hub 1-0:1.0: 6 ports detected client # [ 0.905247] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.908925] usbcore: registered new interface driver usb-storage worker2 # [ 0.914278] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.906820] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.909016] usbcore: registered new interface driver usbserial_generic lb # [ 0.909031] usbserial: USB Serial support registered for generic client # [ 0.906844] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520) lb # [ 0.909636] hv_vmbus: registering driver hyperv_keyboard client # [ 0.907144] i2c_dev: i2c /dev entries driver worker2 # [ 0.916919] drop_monitor: Initializing network drop monitor service worker2 # [ 0.917064] NET: Registered PF_INET6 protocol family worker2 # [ 0.918952] Segment Routing with IPv6 worker1 # [ 0.990795] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.913398] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.918980] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.919007] NET: Registered PF_PACKET protocol family lb # [ 0.913422] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520) lb # [ 0.913736] i2c_dev: i2c /dev entries driver client # [ 0.912378] sdhci: Secure Digital Host Controller Interface driver client # [ 0.912388] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.920626] 9pnet: Installing 9P2000 support worker2 # [ 0.920675] Key type dns_resolver registered client # [ 0.912652] Synopsys Designware Multimedia Card Interface Driver client # [ 0.913031] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.919148] sdhci: Secure Digital Host Controller Interface driver client # [ 0.917270] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.919160] sdhci: Copyright(c) Pierre Ossman client # [ 0.917506] usbcore: registered new interface driver usbhid lb # [ 0.919431] Synopsys Designware Multimedia Card Interface Driver client # [ 0.917512] usbhid: USB HID core driver lb # [ 0.919807] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.923556] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.924578] hub 1-0:1.0: USB hub found lb # [ 0.925092] hub 1-0:1.0: 6 ports detected client # [ 0.920418] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.927551] registered taskstats version 1 client # [ 0.921939] drop_monitor: Initializing network drop monitor service worker2 # [ 0.927701] Loading compiled-in X.509 certificates client # [ 0.922079] NET: Registered PF_INET6 protocol family lb # [ 0.926490] hid: raw HID events driver (C) Jiri Kosina client # [ 0.925208] Segment Routing with IPv6 lb # [ 0.926751] usbcore: registered new interface driver usbhid client # [ 0.925226] In-situ OAM (IOAM) with IPv6 lb # [ 0.926757] usbhid: USB HID core driver client # [ 0.925253] NET: Registered PF_PACKET protocol family lb # [ 0.929725] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.936453] Demotion targets for Node 0: null worker2 # [ 0.936557] Key type .fscrypt registered lb # [ 0.931310] drop_monitor: Initializing network drop monitor service client # [ 0.927100] 9pnet: Installing 9P2000 support worker2 # [ 0.936564] Key type fscrypt-provisioning registered lb # [ 0.931444] NET: Registered PF_INET6 protocol family client # [ 0.927146] Key type dns_resolver registered worker2 # [ 0.936654] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.934458] Segment Routing with IPv6 worker2 # [ 0.936673] ima: Allocated hash algorithm: sha1 lb # [ 0.934488] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.936693] ima: No architecture policies found lb # [ 0.934521] NET: Registered PF_PACKET protocol family worker2 # [ 0.940694] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.936174] 9pnet: Installing 9P2000 support client # [ 0.933603] registered taskstats version 1 lb # [ 0.936217] Key type dns_resolver registered client # [ 0.933746] Loading compiled-in X.509 certificates lb # [ 0.942974] registered taskstats version 1 lb # [ 0.943134] Loading compiled-in X.509 certificates client # [ 0.942340] Demotion targets for Node 0: null client # [ 0.942446] Key type .fscrypt registered client # [ 0.942452] Key type fscrypt-provisioning registered client # [ 0.942547] ima: No TPM chip found, activating TPM-bypass! client # [ 0.942566] ima: Allocated hash algorithm: sha1 client # [ 0.942591] ima: No architecture policies found client # [ 0.946717] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.951876] Demotion targets for Node 0: null lb # [ 0.951992] Key type .fscrypt registered lb # [ 0.951997] Key type fscrypt-provisioning registered lb # [ 0.952094] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.958553] clk: Disabling unused clocks lb # [ 0.952114] ima: Allocated hash algorithm: sha1 worker2 # [ 0.958579] PM: genpd: Disabling unused power domains lb # [ 0.952135] ima: No architecture policies found lb # [ 0.956184] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.962731] Freeing unused kernel memory: 4736K worker2 # [ 0.963529] Run /init as init process client # [ 0.964967] clk: Disabling unused clocks client # [ 0.964993] PM: genpd: Disabling unused power domains worker2 # [ 0.976998] systemd[1]: Successfully made /usr/ read-only. client # [ 0.969117] Freeing unused kernel memory: 4736K client # [ 0.969332] Run /init as init process lb # [ 0.974403] clk: Disabling unused clocks lb # [ 0.974429] PM: genpd: Disabling unused power domains lb # [ 0.978590] Freeing unused kernel memory: 4736K lb # [ 0.978787] Run /init as init process client # [ 0.984333] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.994997] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 1.156643] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.150253] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.158894] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.172913] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.309195] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.325604] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.337611] systemd[1]: Detected virtualization qemu. worker1 # [ 1.339584] systemd[1]: Detected architecture arm64. worker1 # [ 1.341520] systemd[1]: Running in initrd. worker1 # [ 1.344071] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.347121] systemd[1]: Hostname set to . client # [ 1.302699] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.311490] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.396850] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.317667] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.330121] systemd[1]: Detected virtualization qemu. worker2 # [ 1.332305] systemd[1]: Detected architecture arm64. lb # [ 1.325455] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.334211] systemd[1]: Running in initrd. client # [ 1.318973] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.336851] systemd[1]: Initializing machine ID from random generator. client # [ 1.331093] systemd[1]: Detected virtualization qemu. worker2 # [ 1.339726] systemd[1]: Hostname set to . client # [ 1.333096] systemd[1]: Detected architecture arm64. client # [ 1.335050] systemd[1]: Running in initrd. client # [ 1.337617] systemd[1]: Initializing machine ID from random generator. lb # [ 1.331572] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.340523] systemd[1]: Hostname set to . lb # [ 1.343997] systemd[1]: Detected virtualization qemu. lb # [ 1.344090] systemd[1]: Detected architecture arm64. lb # [ 1.344114] systemd[1]: Running in initrd. lb # [ 1.350580] systemd[1]: Initializing machine ID from random generator. lb # [ 1.353346] systemd[1]: Hostname set to . client # [ 1.386453] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.395068] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.405092] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.516650] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.514867] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.510214] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.524886] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.645852] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.677460] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.683073] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.640438] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.643763] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.655330] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.671655] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.676769] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.670995] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.678493] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.764448] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.685640] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.691282] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.775022] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.776256] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.777723] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.778821] systemd[1]: Reached target Path Units. worker1 # [ 1.779669] systemd[1]: Reached target Slice Units. worker1 # [ 1.780523] systemd[1]: Reached target Swaps. worker1 # [ 1.781364] systemd[1]: Reached target Timer Units. worker1 # [ 1.782398] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.783681] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.784987] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.785130] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.785246] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.785269] systemd[1]: Reached target Socket Units. worker1 # [ 1.790353] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.791532] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.800749] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.808892] systemd[1]: Starting Journal Service... worker2 # [ 1.754068] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.832724] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.833683] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.765315] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.766526] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.759049] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.768014] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.769108] systemd[1]: Reached target Path Units. worker2 # [ 1.769944] systemd[1]: Reached target Slice Units. worker2 # [ 1.770795] systemd[1]: Reached target Swaps. worker2 # [ 1.771639] systemd[1]: Reached target Timer Units. worker2 # [ 1.772668] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.773923] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.768610] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.775113] systemd[1]: Listening on Journal Sockets. lb # [ 1.772401] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.775256] systemd[1]: Listening on udev Control Socket. client # [ 1.769831] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.775375] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.775398] systemd[1]: Reached target Socket Units. client # [ 1.771279] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.772394] systemd[1]: Reached target Path Units. worker2 # [ 1.780519] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.852838] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.773245] systemd[1]: Reached target Slice Units. client # [ 1.774100] systemd[1]: Reached target Swaps. worker2 # [ 1.781715] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.774949] systemd[1]: Reached target Timer Units. lb # [ 1.778983] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.775999] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.780105] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.777283] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.857740] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.778521] systemd[1]: Listening on Journal Sockets. lb # [ 1.781455] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.778664] systemd[1]: Listening on udev Control Socket. lb # [ 1.781489] systemd[1]: Reached target Path Units. lb # [ 1.781507] systemd[1]: Reached target Slice Units. client # [ 1.778780] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.781527] systemd[1]: Reached target Swaps. client # [ 1.778805] systemd[1]: Reached target Socket Units. lb # [ 1.781546] systemd[1]: Reached target Timer Units. lb # [ 1.781732] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.783900] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.781870] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.790979] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.781998] systemd[1]: Listening on Journal Sockets. client # [ 1.785105] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.782116] systemd[1]: Listening on udev Control Socket. lb # [ 1.782229] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.782249] systemd[1]: Reached target Socket Units. lb # [ 1.792316] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.868718] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.793429] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.799123] systemd[1]: Starting Journal Service... worker1 # [ 1.869377] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.794332] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.803924] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.802525] systemd[1]: Starting Journal Service... worker1 # [ 1.884961] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.811337] systemd[1]: Starting Journal Service... worker2 # [ 1.824270] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.824369] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.828155] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.825468] systemd[1]: Starting Load Kernel Modules... lb # [ 1.825578] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.826785] systemd[1]: Starting Load Kernel Modules... client # [ 1.827783] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.833175] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.847037] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.921135] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.923496] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.848966] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.849770] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.850833] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.860015] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.934063] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.855497] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.872151] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.874627] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.869247] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.870277] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.944642] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.870993] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.880740] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.950557] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.950784] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.950795] [drm] features: -context_init worker1 # [ 1.951466] [drm] number of scanouts: 1 worker1 # [ 1.951484] [drm] number of cap sets: 0 client # [ 1.878317] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.891910] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.891389] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.897422] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.897644] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.897654] [drm] features: -context_init worker2 # [ 1.898326] [drm] number of scanouts: 1 worker2 # [ 1.898344] [drm] number of cap sets: 0 lb # [ 1.898898] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.977161] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.977461] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.977487] systemd[1]: Reached target Local File Systems. client # [ 1.899641] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.981328] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.909376] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.915447] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.985842] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.911691] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.917791] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.985860] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 1.914275] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.918982] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.916944] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.919292] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.929980] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.930207] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.931038] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.930216] [drm] features: -context_init worker2 # [ 1.931054] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 2.005257] Console: switching to colour frame buffer device 160x50 lb # [ 1.930912] [drm] number of scanouts: 1 lb # [ 1.930930] [drm] number of cap sets: 0 client # [ 1.935255] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.935487] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.935497] [drm] features: -context_init client # [ 1.936219] [drm] number of scanouts: 1 client # [ 1.936236] [drm] number of cap sets: 0 worker2 # [ 1.951187] Console: switching to colour frame buffer device 160x50 lb # [ 1.953367] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.953533] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.953558] systemd[1]: Reached target Local File Systems. worker1 # [ 2.033175] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.957295] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.950508] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.950524] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 2.038951] systemd[1]: Started Journal Service. lb # [ 1.962192] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.962210] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.971374] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.971526] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.971549] systemd[1]: Reached target Local File Systems. worker2 # [ 1.975273] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 2.036440] systemd-modules-load[74]: Inserted module 'dm_mod' worker1 # [ 2.037664] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker1 # [ 2.038709] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 1.974750] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 2.039758] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 1.975064] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.983434] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.975090] systemd[1]: Reached target Local File Systems. client # [ 1.981080] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.981141] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.995363] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.052591] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.053616] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.998305] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.054638] systemd[1]: Finished Load Kernel Modules. client # [ 1.986565] Console: switching to colour frame buffer device 160x50 client # [ 1.993016] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.067387] systemd[1]: Starting Apply Kernel Variables... lb # [ 2.016062] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.018469] systemd[1]: Finished Load Kernel Modules. client # [ 2.021427] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.087248] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 2.033419] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.096201] systemd[1]: Finished Create System Files and Directories. lb # [ 2.037191] systemd[1]: Starting Apply Kernel Variables... client # [ 2.034354] systemd[1]: Started Journal Service. client # [ 2.023073] systemd-modules-load[74]: Inserted module 'dm_mod' worker1 # [ 2.101274] systemd[1]: Finished Apply Kernel Variables. client # [ 2.024158] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 2.051445] systemd[1]: Finished Apply Kernel Variables. client # [ 2.036317] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 2.037470] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.057604] systemd[1]: Started Journal Service. client # [ 2.039597] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.045102] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.123022] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.055829] systemd-modules-load[73]: Inserted module 'dm_mod' worker2 # [ 2.057128] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker2 # [ 2.060908] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.067690] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.064900] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.065156] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.073160] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.082800] systemd[1]: Started Journal Service. lb # [ 2.069046] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.070150] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.077534] systemd[1]: Starting Create System Files and Directories... lb # [ 2.071227] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 2.074523] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker2 # [ 2.084645] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 2.081080] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.082369] systemd-udevd[78]: Using default interface naming scheme 'v261'. lb # [ 2.083430] systemd[1]: Starting Create System Files and Directories... lb # [ 2.088379] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.096124] systemd[1]: Finished Create System Files and Directories. client # [ 2.092995] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.107689] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.183558] systemd[1]: Starting Virtual Console Setup... lb # [ 2.108984] systemd[1]: Finished Create System Files and Directories. client # [ 2.116691] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.119813] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.224504] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.227806] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.163709] systemd[1]: Starting Virtual Console Setup... client # [ 2.164864] systemd[1]: Starting Virtual Console Setup... lb # [ 2.178043] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.208643] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.211621] systemd[1]: Finished Virtual Console Setup. client # [ 2.216408] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.219688] systemd[1]: Finished Virtual Console Setup. lb # [ 2.224457] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.227619] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.835738] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.840185] systemd[1]: Reached target System Initialization. worker1 # [ 2.840988] systemd[1]: Reached target Basic System. worker2 # [ 2.817125] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.820117] systemd[1]: Reached target System Initialization. worker2 # [ 2.820998] systemd[1]: Reached target Basic System. lb # [ 2.818641] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.824182] systemd[1]: Reached target System Initialization. lb # [ 2.825200] systemd[1]: Reached target Basic System. client # [ 2.824854] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.825802] systemd[1]: Reached target System Initialization. client # [ 2.826630] systemd[1]: Reached target Basic System. worker1 # [ 2.976291] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.003167] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.948637] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.021166] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.975963] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.984256] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.988384] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.983516] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.082333] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.085278] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.088704] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.012099] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.014537] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.017409] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.024145] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.049997] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.051833] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.056077] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.136192] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.144109] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.148209] systemd[1]: Mounting /sysroot... client # [ 3.081715] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.086691] systemd[1]: Reached target Initrd Root Device. client # [ 3.092557] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.101046] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.110828] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.104858] systemd[1]: Reached target Initrd Root Device. lb # [ 3.107668] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.202975] EXT4-fs (vda): mounted filesystem 07f97f2e-1a27-4afe-86d2-1a582037d5c9 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.118483] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.191572] systemd[1]: Mounted /sysroot. worker1 # [ 3.192535] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.124127] systemd[1]: Mounting /sysroot... worker1 # [ 3.195473] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.220382] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.143885] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.226364] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.150978] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.172504] EXT4-fs (vda): mounted filesystem 28eb8080-5cf1-4418-abb6-e32a14f1f74c r/w with ordered data mode. Quota mode: none. client # [ 3.153600] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.232500] systemd[1]: Reloading... worker2 # [ 3.163124] systemd[1]: Mounted /sysroot. client # [ 3.157039] systemd[1]: Mounting /sysroot... lb # [ 3.157970] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.165233] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.169895] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.163830] systemd[1]: Mounting /sysroot... worker2 # [ 3.192357] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.200116] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.205953] EXT4-fs (vda): mounted filesystem c916ece1-5f76-4c92-b04d-2db56ad83b45 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.204787] systemd[1]: Reloading... client # [ 3.198725] systemd[1]: Mounted /sysroot. lb # [ 3.214919] EXT4-fs (vda): mounted filesystem 59281576-d49d-44d1-811e-7aef8f48f1ce r/w with ordered data mode. Quota mode: none. client # [ 3.200693] systemd[1]: Reached target Initrd Root File System. lb # [ 3.203148] systemd[1]: Mounted /sysroot. lb # [ 3.205481] systemd[1]: Reached target Initrd Root File System. client # [ 3.205448] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.208993] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.231891] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.235372] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.239188] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.240899] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.245572] systemd[1]: Reloading... client # [ 3.245134] systemd[1]: Reloading... worker1 # [ 3.442598] systemd[1]: Reloading finished in 214 ms. worker1 # [ 3.473863] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.406468] systemd[1]: Reloading finished in 203 ms. worker1 # [ 3.476448] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker1 # [ 3.481846] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.484210] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.486323] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.439259] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.445352] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.439930] systemd[1]: Reloading finished in 201 ms. worker2 # [ 3.450847] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.453071] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.447037] systemd[1]: Reloading finished in 203 ms. worker2 # [ 3.454693] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.474983] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.474871] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... client # [ 3.476938] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... lb # [ 3.478918] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.482864] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.484782] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.484465] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.485945] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.486764] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.486916] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.853722] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.862998] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.873927] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.819125] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.900524] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.834742] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.839035] systemd[1]: Mounting /sysroot/run... lb # [ 3.832834] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.910753] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.842957] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.851153] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.859653] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.854689] systemd[1]: Mounting /sysroot/run... client # [ 3.859204] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.864125] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.942520] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.870915] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.974089] fuse: init (API version 7.45) worker2 # [ 3.890051] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.887996] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.890108] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.983820] virtiofs virtio6: discovered new tag: nix-store client # [ 3.898516] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.992676] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.921954] fuse: init (API version 7.45) worker1 # [ 3.987136] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.937720] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 4.010400] virtiofs virtio7: discovered new tag: shared worker2 # [ 3.938536] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.011173] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.922256] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 4.016222] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.931295] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 4.021624] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 4.007905] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.952766] virtiofs virtio7: discovered new tag: shared lb # [ 3.933408] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.953570] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.960202] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.960917] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.941381] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.961916] fuse: init (API version 7.45) worker1 # [ 4.027508] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 3.969766] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.970553] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.033614] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.037300] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.967718] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.974950] fuse: init (API version 7.45) worker1 # [ 4.039920] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.041404] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.986461] virtiofs virtio7: discovered new tag: shared lb # [ 3.987205] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.980087] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.986498] virtiofs virtio6: discovered new tag: nix-store client # [ 3.987266] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.983021] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.983884] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.994033] virtiofs virtio8: discovered new tag: xchg lb # [ 3.994764] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.982043] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.993237] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.990960] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.003844] virtiofs virtio7: discovered new tag: shared client # [ 4.004611] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.997608] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.010882] virtiofs virtio8: discovered new tag: xchg client # [ 4.011615] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.008834] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.010171] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.007506] systemd[1]: Mounted /sysroot/run. lb # [ 4.008306] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.013972] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.024534] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.025602] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.025601] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.027775] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 4.030134] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.032773] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.035491] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.041542] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.080351] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.082429] systemd[1]: Reached target Initrd File Systems. client # [ 4.084112] systemd[1]: Starting Find NixOS closure... client # [ 4.100862] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.131399] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.135547] systemd[1]: Finished Find NixOS closure. client # [ 4.138321] systemd[1]: Reached target Initrd Default Target. client # [ 4.147820] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.170426] systemd[1]: Stopped target Initrd Default Target. client # [ 4.173940] systemd[1]: Stopped target Basic System. client # [ 4.176449] systemd[1]: Stopped target Initrd Root Device. client # [ 4.178411] systemd[1]: Stopped target Path Units. client # [ 4.180278] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.183614] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.189685] systemd[1]: Stopped target Slice Units. client # [ 4.190971] systemd[1]: Stopped target Socket Units. client # [ 4.192635] systemd[1]: Stopped target System Initialization. client # [ 4.196644] systemd[1]: Stopped target Swaps. client # [ 4.198832] systemd[1]: Stopped target Timer Units. client # [ 4.199591] systemd[1]: dbus.socket: Deactivated successfully. client # [ 4.202425] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 4.207793] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 4.209155] systemd[1]: Stopped Find NixOS closure. client # [ 4.211333] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.213570] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 4.222426] systemd[1]: Stopped Apply Kernel Variables. client # [ 4.223264] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 4.227203] systemd[1]: Stopped Load Kernel Modules. client # [ 4.229614] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 4.232731] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 4.236267] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 4.238151] systemd[1]: Stopped Create System Files and Directories. client # [ 4.239059] systemd[1]: Stopped target Local File Systems. client # [ 4.239816] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 4.240893] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 4.241866] systemd[1]: Stopped Coldplug All udev Devices. client # [ 4.242617] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 4.243613] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.247277] systemd[1]: Stopped Virtual Console Setup. client # [ 4.249526] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 4.251240] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 4.255620] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.256928] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.257742] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 4.258663] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 4.259634] systemd[1]: systemd-udevd.service: Consumed 1.125s CPU time over 2.279s wall clock time, 17.9M memory peak. client # [ 4.261444] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 4.262448] systemd[1]: Closed udev Control Socket. client # [ 4.263153] systemd[1]: Starting Cleanup udev Database... client # [ 4.263907] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 4.265084] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 4.265954] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 4.267059] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 4.276113] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 4.277176] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 4.301890] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 4.302993] systemd[1]: Finished Cleanup udev Database. client # [ 4.307762] systemd[1]: Reached target Switch Root. client # [ 4.309034] systemd[1]: Starting NixOS Activation... worker2 # [ 4.322382] (udev-worker)[94]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.327688] (udev-worker)[94]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.402926] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.409048] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.353724] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.356368] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.360730] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.361554] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.434164] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.436588] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.440830] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.375568] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.446816] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.377154] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.383385] systemd[1]: Starting Virtual Console Setup... client # [ 4.382397] initrd-nixos-activation-start[171]: booting system configuration /nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test worker1 # [ 4.462084] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.463529] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.468272] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.409598] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.411161] systemd[1]: Finished Virtual Console Setup. client # [ 4.413298] initrd-nixos-activation-start[171]: running activation script... worker1 # [ 4.492457] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.416592] (udev-worker)[90]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.495634] systemd[1]: Finished Virtual Console Setup. lb # [ 4.422192] (udev-worker)[90]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.462065] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.464110] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.465331] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.468192] systemd[1]: Starting Virtual Console Setup... lb # [ 4.492328] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.495564] systemd[1]: Finished Virtual Console Setup. client # [ 4.633048] initrd-nixos-activation-start[194]: setting up /etc... client # [ 4.744309] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 4.746988] systemd[1]: Finished NixOS Activation. client # [ 4.748248] systemd[1]: Starting Switch Root... client # [ 4.770745] systemd[1]: Switching root. worker1 # [ 4.854992] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.820515] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.835415] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.917644] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.922678] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.925062] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.936337] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 4.889786] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.893076] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.897406] systemd[1]: Starting Find NixOS closure... lb # [ 4.897215] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.905116] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.900116] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.978298] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.904154] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.984352] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.909283] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.994331] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.996588] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.999683] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.952249] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 4.949116] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.957458] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 5.030173] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.032210] systemd[1]: Stopped target Basic System. lb # [ 4.956159] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 5.036545] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.037736] systemd[1]: Stopped target Path Units. worker2 # [ 4.967492] systemd[1]: Finished Find NixOS closure. lb # [ 4.959357] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.969586] systemd[1]: Reached target Initrd Default Target. worker1 # [ 5.039879] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 4.973063] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.044253] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.045667] systemd[1]: Stopped target Slice Units. worker1 # [ 5.046574] systemd[1]: Stopped target Socket Units. lb # [ 4.969781] systemd[1]: Finished Find NixOS closure. worker1 # [ 5.047519] systemd[1]: Stopped target System Initialization. lb # [ 4.972322] systemd[1]: Reached target Initrd Default Target. lb # [ 4.975248] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.055277] systemd[1]: Stopped target Swaps. worker1 # [ 5.056949] systemd[1]: Stopped target Timer Units. worker1 # [ 5.059028] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.068243] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.070387] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.002746] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.074599] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.008576] systemd[1]: Stopped target Basic System. worker1 # [ 5.079507] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.009702] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.011008] systemd[1]: Stopped target Path Units. lb # [ 5.004472] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.081954] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.006138] systemd[1]: Stopped target Basic System. worker2 # [ 5.012605] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.084879] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.008245] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.086790] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.017399] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.088995] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.012437] systemd[1]: Stopped target Path Units. lb # [ 5.013485] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.091339] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.092736] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.015852] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.025161] systemd[1]: Stopped target Slice Units. worker1 # [ 5.096248] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.027038] systemd[1]: Stopped target Socket Units. worker1 # [ 5.097315] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.020984] systemd[1]: Stopped target Slice Units. lb # [ 5.022154] systemd[1]: Stopped target Socket Units. worker2 # [ 5.029039] systemd[1]: Stopped target System Initialization. lb # [ 5.023098] systemd[1]: Stopped target System Initialization. worker1 # [ 5.100407] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.101239] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.031702] systemd[1]: Stopped target Swaps. worker1 # [ 5.102237] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 5.103339] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.104214] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.105224] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.106208] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.106911] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.030772] systemd[1]: Stopped target Swaps. worker1 # [ 5.107863] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.040587] systemd[1]: Stopped target Timer Units. lb # [ 5.033925] systemd[1]: Stopped target Timer Units. worker2 # [ 5.041535] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.112240] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.042643] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.113756] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.043641] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.116270] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.040291] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.120224] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.044619] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.121279] systemd[1]: systemd-udevd.service: Consumed 1.435s CPU time over 3.123s wall clock time, 22.3M memory peak. worker1 # [ 5.122694] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.045728] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.055146] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.057166] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.128187] systemd[1]: Closed udev Control Socket. worker1 # [ 5.128936] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.058838] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.129737] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.130770] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.060878] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.054894] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.132120] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.057148] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.064358] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.065429] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.066154] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.136474] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.060386] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.137479] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.138448] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.062772] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.064917] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.071591] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.072863] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.073863] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.067632] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.074714] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.075462] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.069079] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.077157] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.078143] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.071736] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.078914] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.079924] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.081048] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.081765] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.082663] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.076401] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.077464] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.083657] systemd[1]: systemd-udevd.service: Consumed 1.376s CPU time over 3.094s wall clock time, 22.3M memory peak. lb # [ 5.078383] systemd[1]: Stopped target Local File Systems. lb # [ 5.079133] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.081370] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.082515] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.083293] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.084416] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.085441] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.092273] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.086184] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.163001] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.093257] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.164120] systemd[1]: Finished Cleanup udev Database. lb # [ 5.087297] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.096104] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.097126] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.168551] systemd[1]: Reached target Switch Root. worker1 # [ 5.169317] systemd[1]: Starting NixOS Activation... lb # [ 5.092197] systemd[1]: systemd-udevd.service: Consumed 1.470s CPU time over 3.121s wall clock time, 22.2M memory peak. worker2 # [ 5.100163] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.093678] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 5.101187] systemd[1]: Closed udev Control Socket. worker2 # [ 5.101870] systemd[1]: Starting Cleanup udev Database... lb # [ 5.096401] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.104219] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.100210] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.101256] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.108324] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.102087] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.109243] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.110332] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.104139] systemd[1]: Closed udev Control Socket. worker2 # [ 5.112269] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.108176] systemd[1]: Starting Cleanup udev Database... lb # [ 5.108971] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.116340] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.110040] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.112161] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.116364] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.117385] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.118355] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.134961] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.136116] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.140268] systemd[1]: Reached target Switch Root. worker2 # [ 5.141050] systemd[1]: Starting NixOS Activation... lb # [ 5.140416] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.142258] systemd[1]: Finished Cleanup udev Database. lb # [ 5.144491] systemd[1]: Reached target Switch Root. lb # [ 5.147937] systemd[1]: Starting NixOS Activation... worker1 # [ 5.245948] initrd-nixos-activation-start[177]: booting system configuration /nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test worker1 # [ 5.277461] initrd-nixos-activation-start[177]: running activation script... worker2 # [ 5.217884] initrd-nixos-activation-start[176]: booting system configuration /nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test lb # [ 5.224514] initrd-nixos-activation-start[175]: booting system configuration /nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test worker2 # [ 5.249362] initrd-nixos-activation-start[176]: running activation script... lb # [ 5.255319] initrd-nixos-activation-start[175]: running activation script... worker1 # [ 5.502349] initrd-nixos-activation-start[200]: setting up /etc... worker2 # [ 5.492485] initrd-nixos-activation-start[199]: setting up /etc... client # [ 5.499612] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 5.512053] systemd[1]: Detected virtualization qemu. client # [ 5.515091] systemd[1]: Detected architecture arm64. client # [ 5.519035] systemd[1]: Detected first boot. lb # [ 5.509914] initrd-nixos-activation-start[198]: setting up /etc... client # [ 5.524597] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.618608] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.621336] systemd[1]: Finished NixOS Activation. worker1 # [ 5.622490] systemd[1]: Starting Switch Root... worker1 # [ 5.643453] systemd[1]: Switching root. worker2 # [ 5.607151] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.612193] systemd[1]: Finished NixOS Activation. worker2 # [ 5.616249] systemd[1]: Starting Switch Root... lb # [ 5.623768] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.626664] systemd[1]: Finished NixOS Activation. lb # [ 5.627834] systemd[1]: Starting Switch Root... worker2 # [ 5.635200] systemd[1]: Switching root. lb # [ 5.646780] systemd[1]: Switching root. worker1 # [ 5.830226] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.816057] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.837851] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.844398] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.035144] systemd[1]: Applying preset policy. client # [ 6.272053] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.353261] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.365122] systemd[1]: Detected virtualization qemu. worker1 # [ 6.368660] systemd[1]: Detected architecture arm64. worker1 # [ 6.370692] systemd[1]: Detected first boot. worker1 # [ 6.375272] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.341434] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.354001] systemd[1]: Detected virtualization qemu. worker2 # [ 6.357017] systemd[1]: Detected architecture arm64. worker2 # [ 6.360585] systemd[1]: Detected first boot. worker2 # [ 6.365934] systemd[1]: Initializing machine ID from random generator. lb # [ 6.354700] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.367289] systemd[1]: Detected virtualization qemu. lb # [ 6.370367] systemd[1]: Detected architecture arm64. lb # [ 6.374386] systemd[1]: Detected first boot. lb # [ 6.379888] systemd[1]: Initializing machine ID from random generator. client # [ 6.487684] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 6.488931] systemd[1]: Stopped initrd-switch-root.service. client # [ 6.493135] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 6.497588] systemd[1]: Created slice Slice /system/getty. client # [ 6.499578] systemd[1]: Created slice User and Session Slice. client # [ 6.500828] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 6.503611] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 6.505662] systemd[1]: Expecting device /dev/hvc0... client # [ 6.507495] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 6.509263] systemd[1]: Reached target Local Encrypted Volumes. client # [ 6.511160] systemd[1]: Stopped target initrd-fs.target. client # [ 6.512179] systemd[1]: Stopped target initrd-root-fs.target. client # [ 6.513625] systemd[1]: Stopped target initrd-switch-root.target. client # [ 6.515252] systemd[1]: Reached target Virtual Machines and Containers. client # [ 6.517463] systemd[1]: Reached target Path Units. client # [ 6.518454] systemd[1]: Reached target Remote File Systems. client # [ 6.520703] systemd[1]: Reached target Slice Units. client # [ 6.521654] systemd[1]: Reached target Swaps. client # [ 6.524332] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 6.527198] systemd[1]: Listening on Process Core Dump Socket. client # [ 6.529355] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 6.531678] systemd[1]: Listening on Factory Reset Management. client # [ 6.532858] systemd[1]: Listening on Hostname Service Socket. client # [ 6.536516] systemd[1]: Starting Journal Log Access Socket... client # [ 6.538565] systemd[1]: Listening on Journal Audit Socket. client # [ 6.541219] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.542780] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.544894] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.546492] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 6.552530] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 6.553855] systemd[1]: Listening on udev Control Socket. client # [ 6.555351] systemd[1]: Listening on udev Varlink Socket. client # [ 6.559156] systemd[1]: Mounting Huge Pages File System... client # [ 6.567862] systemd[1]: Mounting POSIX Message Queue File System... client # [ 6.577587] systemd[1]: Mounting Kernel Debug File System... client # [ 6.587792] systemd[1]: Mounting Kernel Trace File System... client # [ 6.599439] systemd[1]: Starting Create List of Static Device Nodes... client # [ 6.601452] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 6.693216] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.618523] systemd[1]: Mounting Kernel Configuration File System... client # [ 6.618874] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 6.619131] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.619400] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.646014] systemd[1]: Mounting FUSE Control File System... client # [ 6.647204] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.670869] systemd[1]: Starting Journal Service... worker2 # [ 6.683351] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.686039] systemd[1]: Starting Load Kernel Modules... lb # [ 6.697693] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.704049] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 6.713007] systemd-journald[265]: Collecting audit messages is enabled. client # [ 6.721857] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 6.723275] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.715882] systemd[1]: Queued start job for default target Multi-User System. client # [ 6.718205] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 6.734946] systemd[1]: Starting Coldplug All udev Devices... client # [ 6.765833] systemd[1]: Started Journal Service. client # [ 6.753287] systemd-modules-load[267]: Module 'atkbd' is built in client # [ 6.755095] systemd-modules-load[267]: Module 'loop' is built in client # [ 6.761640] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.763674] systemd[1]: Mounted Huge Pages File System. client # [ 6.765694] systemd[1]: Mounted POSIX Message Queue File System. client # [ 6.766584] systemd[1]: Mounted Kernel Debug File System. client # [ 6.767462] systemd[1]: Mounted Kernel Trace File System. client # [ 6.777536] systemd[1]: Finished Create List of Static Device Nodes. client # [ 6.778601] systemd[1]: Finished Load Kernel Modules. worker1 # [ 6.878437] systemd[1]: Applying preset policy. client # [ 6.794148] systemd[1]: Mounted Kernel Configuration File System. client # [ 6.806143] systemd[1]: Starting Apply Kernel Variables... client # [ 6.811272] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 6.864103] systemd[1]: Applying preset policy. client # [ 6.858239] EXT4-fs (vda): re-mounted c916ece1-5f76-4c92-b04d-2db56ad83b45. lb # [ 6.876577] systemd[1]: Applying preset policy. client # [ 6.863355] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 6.864586] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 6.868293] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 6.869272] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.883181] systemd[1]: Starting Load/Save OS Random Seed... client # [ 6.892300] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.896790] systemd[1]: Mounted FUSE Control File System. client # [ 6.910488] systemd-oomd[268]: No swap; memory pressure usage will be degraded client # [ 6.925328] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 6.970589] systemd-journald[265]: Received client request to flush runtime journal. worker1 # [ 7.097619] systemd[1]: Populated /etc with preset unit settings. client # [ 7.012491] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.013359] systemd[1]: Reached target First Boot Complete. client # [ 7.014154] systemd[1]: Finished Apply Kernel Variables. client # [ 7.021144] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.104213] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.138256] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.319317] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.320686] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.323431] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.327264] systemd[1]: Created slice Slice /system/getty. client # [ 7.234048] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 7.235901] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 7.329387] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.331395] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.333736] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.335833] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.337752] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.338782] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.340316] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.342465] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.343525] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.345951] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.347139] systemd[1]: Reached target Path Units. worker1 # [ 7.348629] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.350796] systemd[1]: Reached target Slice Units. worker1 # [ 7.352502] systemd[1]: Reached target Swaps. worker1 # [ 7.355526] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.358451] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.360691] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.363008] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.364218] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.368197] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.370420] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.373132] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.374642] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.376057] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.378585] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.384283] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.385622] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.387019] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.390337] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.326781] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.397266] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.328515] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.332512] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.407038] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.336697] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.338636] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.339973] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.341669] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.343421] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.345522] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.347425] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.420275] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.348521] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.350662] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.351769] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.353283] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.354903] systemd[1]: Reached target Path Units. worker2 # [ 7.356919] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.358657] systemd[1]: Reached target Slice Units. worker1 # [ 7.431798] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.360391] systemd[1]: Reached target Swaps. worker2 # [ 7.363377] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.366205] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.436759] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.368427] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.370699] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.372644] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.444322] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.445500] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.370342] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.376191] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.371652] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.378207] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.451992] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.380888] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.375805] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.382393] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.454678] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.380110] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.383958] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.382078] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.386741] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.383496] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.385301] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.392351] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.387621] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.393657] systemd[1]: Listening on udev Control Socket. lb # [ 7.389496] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 7.390539] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.395137] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.392067] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.399129] systemd[1]: Mounting Huge Pages File System... lb # [ 7.393575] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.395775] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 7.396930] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.399213] systemd[1]: Reached target Path Units. lb # [ 7.400977] systemd[1]: Reached target Remote File Systems. lb # [ 7.402788] systemd[1]: Reached target Slice Units. worker2 # [ 7.407161] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.404591] systemd[1]: Reached target Swaps. worker2 # [ 7.411062] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.407801] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.410741] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.484756] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.485132] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.412970] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.415333] systemd[1]: Listening on Factory Reset Management. lb # [ 7.416515] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.423262] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.420390] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.422490] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.501515] systemd[1]: Starting Journal Service... lb # [ 7.426014] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.427595] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.428998] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.431273] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.435181] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.435537] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.436803] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.438202] systemd[1]: Listening on udev Control Socket. lb # [ 7.439661] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.517293] systemd[1]: Starting Load Kernel Modules... lb # [ 7.442902] systemd[1]: Mounting Huge Pages File System... lb # [ 7.445825] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.451876] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.452223] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.452480] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.452743] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.532429] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.457991] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.463652] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.466982] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.541293] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.467355] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.543018] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.553111] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.479274] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.558640] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.484957] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.562348] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.489811] systemd[1]: Starting Journal Service... worker1 # [ 7.567717] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.493437] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.568188] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.568568] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.571980] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.495633] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.574275] systemd[1]: Mounted FUSE Control File System. lb # [ 7.504915] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.507200] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.518276] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.594842] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.600263] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 7.531567] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.528732] systemd[1]: Mounting FUSE Control File System... lb # [ 7.530675] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.545958] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.546300] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.552000] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.559126] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.559575] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.559978] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.560347] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.560714] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.561078] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.561437] systemd[1]: Mounted FUSE Control File System. lb # [ 7.565276] systemd[1]: Starting Journal Service... worker1 # [ 7.647049] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.652443] systemd[1]: Starting Firewall... lb # [ 7.579105] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.659681] systemd[1]: Starting Apply Kernel Variables... client # [ 7.574290] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 7.596128] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.669464] systemd-journald[270]: Collecting audit messages is enabled. lb # [ 7.596410] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.601371] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.582498] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 7.584818] systemd[1]: Reached target Preparation for Local File Systems. client # [ 7.588108] systemd[1]: Mounting /run/wrappers... lb # [ 7.608758] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.592774] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.613026] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.694057] EXT4-fs (vda): re-mounted 07f97f2e-1a27-4afe-86d2-1a582037d5c9. lb # [ 7.629682] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.704135] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.704883] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.705247] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.714263] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 7.716477] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.644403] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.634389] systemd[1]: Mounted /run/wrappers. client # [ 7.635143] systemd[1]: Reached target Local File Systems. worker2 # [ 7.658596] systemd[1]: Finished Load Kernel Modules. client # [ 7.639195] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 7.642561] systemd[1]: Starting register-nix-paths.service... lb # [ 7.654662] systemd-journald[268]: Collecting audit messages is enabled. worker2 # [ 7.664076] systemd[1]: Starting Firewall... lb # [ 7.662235] systemd[1]: Mounted Huge Pages File System. client # [ 7.647960] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 7.670945] systemd-journald[269]: Collecting audit messages is enabled. client # [ 7.649142] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.728263] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.745931] systemd[1]: Started Journal Service. lb # [ 7.670529] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.731665] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.671860] systemd[1]: Mounted Kernel Debug File System. client # [ 7.655825] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 7.677146] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.673627] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.681569] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.749665] systemd-modules-load[271]: Module 'atkbd' is built in lb # [ 7.674682] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.677730] systemd[1]: Starting Create System Files and Directories... worker2 # [ 7.699284] EXT4-fs (vda): re-mounted 28eb8080-5cf1-4418-abb6-e32a14f1f74c. lb # [ 7.680505] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.681841] systemd-modules-load[269]: Module 'atkbd' is built in worker1 # [ 7.758981] systemd-modules-load[271]: Module 'loop' is built in lb # [ 7.682858] systemd-modules-load[269]: Module 'loop' is built in lb # [ 7.701837] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.703254] systemd[1]: Started Journal Service. worker2 # [ 7.708225] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 7.708788] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.709080] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.768922] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.697773] systemd[1]: Finished Load Kernel Modules. lb # [ 7.698603] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.719807] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.721911] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.732131] systemd[1]: Started Journal Service. worker2 # [ 7.719280] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.720565] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.736203] systemd-modules-load[270]: Module 'atkbd' is built in lb # [ 7.749028] EXT4-fs (vda): re-mounted 59281576-d49d-44d1-811e-7aef8f48f1ce. lb # [ 7.741232] systemd[1]: Starting Firewall... worker2 # [ 7.749844] systemd-modules-load[270]: Module 'loop' is built in lb # [ 7.743940] systemd-oomd[270]: No swap; memory pressure usage will be degraded worker2 # [ 7.755914] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.750442] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 7.752940] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.753751] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.832119] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.757823] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.758812] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 7.761128] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 7.764236] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.839517] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.846002] systemd[1]: Reached target First Boot Complete. lb # [ 7.773884] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 7.851154] systemd-oomd[273]: No swap; memory pressure usage will be degraded lb # [ 7.774877] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.873027] systemd-journald[270]: Received client request to flush runtime journal. client # [ 7.782169] systemd-udevd[295]: Using default interface naming scheme 'v261'. lb # [ 7.784312] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.785144] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.829262] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.822139] systemd[1]: Finished Create System Files and Directories. worker2 # [ 7.830137] systemd[1]: Reached target First Boot Complete. client # [ 7.830153] systemd[1]: Starting Rebuild Journal Catalog... client # [ 7.834325] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 7.841397] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker1 # [ 7.912358] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.851913] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.922444] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.925296] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.857917] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.928392] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.873942] systemd-journald[269]: Received client request to flush runtime journal. lb # [ 7.882223] systemd-journald[268]: Received client request to flush runtime journal. worker2 # [ 7.908671] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 7.909745] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.918414] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 7.921035] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.921967] systemd[1]: Reached target First Boot Complete. lb # [ 7.929083] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.937811] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 7.938879] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 7.944532] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.046791] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.053123] systemd[1]: Reached target Preparation for Local File Systems. client # [ 7.976894] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.056522] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 7.986053] systemd[1]: Starting Update is Completed... lb # [ 7.998147] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.000642] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.043776] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.046729] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.047152] systemd[1]: Finished Update is Completed. worker2 # [ 8.056265] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.161260] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker2 # [ 8.160758] systemd-udevd[307]: Using default interface naming scheme 'v261'. worker1 # [ 8.315426] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.357574] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.358460] systemd[1]: Reached target Local File Systems. worker1 # [ 8.366191] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.367202] systemd[1]: Starting register-nix-paths.service... lb # [ 8.298573] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.299583] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.376659] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.309258] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.316701] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.386385] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.400174] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.405173] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.408429] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 8.340216] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.361919] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.397817] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.400280] systemd[1]: Reached target Local File Systems. worker2 # [ 8.401103] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.406091] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.417726] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.494986] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.425228] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.502903] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.439960] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.441520] systemd[1]: Starting Create System Files and Directories... lb # [ 8.440658] systemd[1]: Mounted /run/wrappers. lb # [ 8.444570] systemd[1]: Reached target Local File Systems. client # [ 8.449582] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 8.452206] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.452781] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 8.459091] systemd[1]: Starting register-nix-paths.service... lb # [ 8.465728] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.466833] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.472869] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.480774] systemd-udevd[305]: Using default interface naming scheme 'v261'. lb # [ 8.489180] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.521233] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.528959] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.610400] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.617895] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.622710] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.597886] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 8.624109] systemd[1]: Finished Create System Files and Directories. lb # [ 8.621558] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.633857] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 8.634726] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.642146] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 8.734371] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.696331] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.724134] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.802326] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.805028] systemd[1]: Starting Update is Completed... worker2 # [ 8.742343] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.738208] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 8.752199] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.811879] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.814148] systemd[1]: Starting Update is Completed... worker1 # [ 8.889321] systemd[1]: Finished Update is Completed. lb # [ 8.877941] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.891607] systemd[1]: Finished Update is Completed. lb # [ 8.950964] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 8.961331] systemd[1]: Starting Update is Completed... client # [ 9.010823] systemd[1]: Starting Virtual Console Setup... client # [ 9.023818] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.061236] systemd[1]: Finished Update is Completed. client # [ 9.062996] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 9.067035] systemd[1]: Stopped Virtual Console Setup. client # [ 9.075147] systemd[1]: Starting Virtual Console Setup... lb # [ 9.133089] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 9.243862] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.248585] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.200840] systemd[1]: Finished register-nix-paths.service. client # [ 9.234428] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.251512] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 9.326113] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.328700] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.345707] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.361177] (udev-worker)[352]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 9.366357] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.368249] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.412073] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 9.514297] systemd[1]: Finished Firewall. client # [ 9.510603] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 9.514743] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 9.519477] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 9.521640] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.522904] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 9.536406] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 9.537996] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 9.539246] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 9.556666] systemd[1]: Finished Firewall. worker2 # [ 9.621916] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 9.717592] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.654463] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.659351] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.662871] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.681494] mousedev: PS/2 mouse device common for all mice worker2 # [ 9.699599] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.703779] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 9.779027] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 9.719458] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.724448] systemd[1]: Finished register-nix-paths.service. lb # [ 9.729237] systemd[1]: Reached target System Initialization. lb # [ 9.730093] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.731085] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.731907] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 9.738812] systemd-vconsole-setup[387]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 9.818947] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.745449] systemd[1]: Reached target Timer Units. lb # [ 9.746243] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 9.746658] systemd[1]: Finished Virtual Console Setup. lb # [ 9.751538] systemd[1]: Listening on niks3 server socket. lb # [ 9.756760] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.757579] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.758760] systemd[1]: Reached target Socket Units. lb # [ 9.759464] systemd[1]: Reached target Basic System. lb # [ 9.772658] systemd[1]: Started backdoor.service. lb # [ 9.773389] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 9.774392] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.800094] systemd[1]: Starting Post-Boot Actions... lb # [ 9.818667] systemd[1]: Started Reset console on configuration changes. lb # [ 9.842957] systemd[1]: Starting resolvconf update... lb # [ 9.856456] systemd[1]: Started rustfs.service. worker1 # [ 9.950632] systemd[1]: Finished register-nix-paths.service. worker1 # [ 9.953378] systemd[1]: Reached target System Initialization. worker1 # [ 9.956674] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 9.958953] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 9.890102] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 9.961424] systemd[1]: Reached target Timer Units. worker1 # [ 9.967306] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 9.976489] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 9.985653] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 9.911091] nsncd[451]: Sep 23 05:32:09.512 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # connecting to host... lb # [ 9.919906] systemd[1]: Starting rustfs-bucket.service... worker1 # [ 10.000692] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.927106] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.007884] systemd[1]: Reached target Socket Units. worker1 # [ 10.011072] systemd[1]: Reached target Basic System. worker1 # [ 10.014851] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 9.949874] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 10.020239] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.029269] systemd[1]: Starting Post-Boot Actions... lb # [ 9.955087] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.036569] systemd[1]: Started Reset console on configuration changes. worker2 # [ 9.967111] systemd[1]: Finished register-nix-paths.service. worker2 # [ 9.970654] systemd[1]: Reached target System Initialization. worker2 # [ 9.973159] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 9.975188] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 9.979831] systemd[1]: Reached target Timer Units. worker1 # [ 10.050672] systemd[1]: Starting resolvconf update... worker2 # [ 9.986894] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.061117] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 9.994873] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 9.995668] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 9.993309] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.994279] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.019339] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.027105] systemd[1]: Reached target Socket Units. client # [ 10.021165] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 10.024321] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.024765] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 10.032313] systemd[1]: Reached target Basic System. client # [ 10.028885] systemd[1]: Reached target System Initialization. worker2 # [ 10.037201] systemd[1]: Started backdoor.service. client # [ 10.030829] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.034829] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.112728] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.042948] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.035943] (udev-worker)[392]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.037207] systemd[1]: Reached target Timer Units. client # [ 10.040377] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.042649] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.121742] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.054699] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.047340] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.049363] systemd[1]: Reached target Socket Units. lb # [ 10.050830] (udev-worker)[403]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.052818] systemd[1]: Reached target Basic System. client # [ 10.056196] systemd[1]: Started backdoor.service. client # [ 10.058119] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.060551] systemd[1]: Starting nftables firewall... worker2 # [ 10.069030] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.140933] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.065668] (udev-worker)[403]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.066843] systemd[1]: Starting User Login Management... worker2 # [ 10.076591] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.147739] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.073220] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.085742] systemd[1]: Starting resolvconf update... worker1 # [ 10.158966] nsncd[488]: Sep 23 05:32:09.740 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.094352] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.172690] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.099788] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 10.101018] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.179698] systemd[1]: Starting User Login Management... worker1 # [ 10.193815] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.136668] systemd[1]: Finished Post-Boot Actions. client # [ 10.129345] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.209787] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.219388] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' client # [ 10.147142] systemd[1]: Starting resolvconf update... worker1 # [ 10.226307] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.168175] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.242005] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.176918] nsncd[489]: Sep 23 05:32:09.783 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.251635] systemd[1]: Started backdoor.service. worker1 # [ 10.254704] systemd[1]: Started D-Bus System Message Bus. worker2 # connecting to host... worker2 # [ 10.196091] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.197060] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.197895] systemd[1]: Starting User Login Management... client # [ 10.195627] systemd[1]: Starting D-Bus System Message Bus... client # connecting to host... worker2 # [ 10.230545] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.308419] dbus-broker-launch[493]: Ready client # [ 10.241802] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.250636] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.265534] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' client # [ 10.258463] nsncd[458]: Sep 23 05:32:09.862 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.266562] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client: Guest shell says: b'Spawning backdoor root shell...\n' worker1 # connecting to host... worker2 # [ 10.282511] systemd[1]: Started D-Bus System Message Bus. client # [ 10.273655] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.304350] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.305234] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.381346] systemd-logind[510]: New seat seat0. worker2 # [ 10.323690] dbus-broker-launch[493]: Ready client # [ 10.331885] systemd[1]: Starting User Login Management... worker1 # [ 10.389834] systemd[1]: Started User Login Management. worker2 # [ 10.364400] (udev-worker)[343]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.335639] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 10.363819] systemd-logind[486]: New seat seat0. client: connected to guest root shell worker1 # [ 10.394473] systemd[1]: Starting linger-users.service... client: (connecting took 10.76 seconds) worker1 # [ 10.418136] systemd[1]: Stopped target Host and Network Name Lookups. client: (finished: waiting for the VM to finish booting, in 10.76 seconds) worker1 # [ 10.419049] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.432211] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.433151] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.433934] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.434728] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.435499] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.377537] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.378952] systemd-logind[503]: New seat seat0. lb # [ 10.372275] systemd[1]: Started User Login Management. worker2 # [ 10.379672] systemd[1]: Started User Login Management. worker1 # [ 10.452791] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.379236] dbus-broker-launch[485]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.389224] systemd[1]: Starting linger-users.service... lb # [ 10.386631] systemd[1]: Starting linger-users.service... client # [ 10.394346] dbus-broker-launch[467]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.406296] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.400138] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.407276] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.401093] dbus-broker-launch[485]: NSS returned no entry for 'systemd-timesync' lb # [ 10.402066] dbus-broker-launch[485]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.483884] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.418546] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.419482] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.491113] systemd[1]: Finished linger-users.service. lb # [ 10.418054] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.418978] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.419815] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.420446] dbus-broker-launch[467]: NSS returned no entry for 'systemd-timesync' client # [ 10.421506] dbus-broker-launch[467]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.425883] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.431359] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.434800] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.435626] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.433707] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 10.434654] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.435575] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.444610] (udev-worker)[338]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.519788] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.523216] nsncd[563]: Sep 23 05:32:10.123 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.531044] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.531967] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.540468] (udev-worker)[463]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.542573] systemd[1]: Finished resolvconf update. worker1 # [ 10.543272] systemd[1]: Reached target Preparation for Network. client # [ 10.469143] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.478205] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.479367] systemd[1]: Finished linger-users.service. lb # [ 10.475378] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.481930] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.560108] systemd[1]: Starting DHCP Client... worker1 # [ 10.560888] systemd[1]: Starting Extra networking commands.... worker1 # [ 10.568207] (udev-worker)[349]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.569383] (udev-worker)[463]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.501853] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.502761] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.503620] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.511366] nsncd[561]: Sep 23 05:32:10.121 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 10.533399] dbus-broker-launch[467]: Ready lb # [ 10.533703] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.542608] systemd[1]: Finished resolvconf update. worker2 # [ 10.543325] systemd[1]: Reached target Preparation for Network. lb # [ 10.540488] systemd[1]: Finished linger-users.service. worker2 # [ 10.550087] systemd[1]: Starting DHCP Client... worker2 # [ 10.556336] systemd[1]: Starting Extra networking commands.... lb # [ 10.557653] dbus-broker-launch[485]: Ready client # [ 10.590282] systemd-logind[488]: New seat seat0. client # [ 10.602561] systemd-logind[488]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 10.603926] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.608999] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.615568] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.616477] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.617791] nsncd[562]: Sep 23 05:32:10.214 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 10.619152] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.619723] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.622941] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.623806] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 10.631284] systemd-logind[488]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 10.633608] systemd[1]: Finished resolvconf update. lb # [ 10.637034] systemd[1]: Starting DHCP Client... client # [ 10.640120] systemd[1]: Started User Login Management. client # [ 10.640871] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.641686] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 10.652645] systemd[1]: Starting linger-users.service... client # [ 10.668120] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.779166] dhcpcd[597]: dhcpcd-10.3.2 starting worker1 # [ 10.790331] dhcpcd[637]: dev: loaded udev lb # [ 10.742367] systemd-logind[486]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 10.845164] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 10.762005] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.768132] systemd[1]: Starting Address configuration of eth1... client # [ 10.761934] systemd[1]: Finished resolvconf update. client # [ 10.766104] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.776115] dhcpcd[598]: dhcpcd-10.3.2 starting client # [ 10.772272] systemd[1]: Finished linger-users.service. client # [ 10.773338] nsncd[549]: Sep 23 05:32:10.377 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 10.775255] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.784424] dhcpcd[639]: dev: loaded udev client # [ 10.785470] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.786384] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.787222] systemd[1]: Starting DHCP Client... lb # [ 10.799063] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.823350] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 10.898499] systemd[1]: Finished Extra networking commands.. worker1 # [ 10.902680] systemd[1]: Reached target Network. worker1 # [ 10.913636] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.914375] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 10.933230] systemd[1]: Starting Address configuration of eth1... worker1 # [ 10.953578] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 10.872140] systemd[1]: Finished Extra networking commands.. worker2 # [ 10.873216] systemd[1]: Reached target Network. worker2 # [ 10.878094] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.980252] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 10.981561] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 10.983570] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 10.983913] cfg80211: failed to load regulatory.db worker2 # [ 10.925622] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 10.913681] dhcpcd[611]: dhcpcd-10.3.2 starting worker2 # [ 10.928556] systemd-logind[503]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 10.928957] dhcpcd[625]: dev: loaded udev worker1 # [ 11.009235] systemd[1]: Finished Permit User Sessions. worker2 # [ 10.958094] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.014790] systemd[1]: Started Getty on tty1. worker2 # [ 10.958582] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.015564] systemd[1]: Reached target Login Prompts. worker2 # [ 10.961456] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 10.961781] cfg80211: failed to load regulatory.db worker2 # [ 10.973842] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 10.967095] systemd[1]: Finished Firewall. worker2 # [ 10.975221] systemd[1]: Finished Permit User Sessions. lb # [ 10.970345] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.064524] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 10.982415] systemd[1]: Started Getty on tty1. worker2 # [ 10.983104] systemd[1]: Reached target Login Prompts. client # [ 10.977989] dhcpcd[583]: dhcpcd-10.3.2 starting lb # [ 10.980736] systemd[1]: Starting Address configuration of eth1... worker2 # [ 10.993719] network-addresses-eth1-start[636]: adding address 192.168.1.4/24... done client # [ 10.987521] dhcpcd[602]: dev: loaded udev lb # [ 10.989862] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.068522] network-addresses-eth1-start[652]: adding address 192.168.1.3/24... done worker1 # [ 11.092349] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.017938] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.079935] dhcpcd[637]: eth0: waiting for carrier worker1 # [ 11.084181] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.014562] network-addresses-eth1-start[636]: adding address 2001:db8:1::4/64... done worker1 # [ 11.087605] network-addresses-eth1-start[652]: adding address 2001:db8:1::3/64... done worker1 # [ 11.091275] dhcpcd[637]: eth0: carrier acquired client # [ 11.040572] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.104935] dhcpcd[637]: DUID 00:01:00:01:32:46:20:da:52:54:00:12:34:56 worker2 # [ 11.035031] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.105936] dhcpcd[637]: eth0: IAID 00:12:34:56 worker1 # [ 11.106583] dhcpcd[637]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.112806] systemd[1]: Finished Address configuration of eth1. worker2 # [ 11.063717] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.052117] dhcpcd[639]: eth0: waiting for carrier worker2 # [ 11.053872] dhcpcd[639]: eth0: waiting for carrier worker2 # [ 11.055307] dhcpcd[639]: eth0: carrier acquired worker1 # [ 11.134618] systemd-logind[510]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 11.065608] dhcpcd[639]: DUID 00:01:00:01:32:46:20:da:52:54:00:12:34:56 worker2 # [ 11.066587] dhcpcd[639]: eth0: IAID 00:12:34:56 worker2 # [ 11.067222] dhcpcd[639]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.191319] mousedev: PS/2 mouse device common for all mice worker2 # [ 11.130966] mousedev: PS/2 mouse device common for all mice client # [ 11.144657] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.160139] systemd-logind[503]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.176800] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.193044] mousedev: PS/2 mouse device common for all mice client # [ 11.191119] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.191596] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.195015] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.195326] cfg80211: failed to load regulatory.db worker1 # [ 11.285609] systemd-logind[510]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.237463] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.268459] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.270908] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.274406] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.274721] cfg80211: failed to load regulatory.db lb # [ 11.262377] network-addresses-eth1-start[632]: adding address 192.168.1.2/24... done client # [ 11.272215] systemd[1]: Finished nftables firewall. client # [ 11.274899] systemd[1]: Reached target Preparation for Network. client # [ 11.283330] systemd[1]: Starting Address configuration of eth1... client # [ 11.302221] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.290819] network-addresses-eth1-start[632]: adding address 2001:db8:1::2/64... done client # [ 11.290773] systemd[1]: Starting Extra networking commands.... client # [ 11.294255] dhcpcd[602]: eth0: waiting for carrier client # [ 11.295066] dhcpcd[602]: libudev: received NULL device client # [ 11.295837] dhcpcd[602]: libudev: received NULL device client # [ 11.303509] dhcpcd[602]: eth0: carrier acquired client # [ 11.319099] dhcpcd[602]: DUID 00:01:00:01:32:46:20:da:52:54:00:12:34:56 client # [ 11.322054] dhcpcd[602]: eth0: IAID 00:12:34:56 client # [ 11.322746] dhcpcd[602]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.333661] systemd[1]: Finished Address configuration of eth1. worker2 # [ 11.385777] dhcpcd[639]: eth0: soliciting a DHCP lease lb # [ 11.379496] dhcpcd[668]: /nix/store/1ki7bgdr0niilkg9jbkpcdaggbv0kb8a-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (637) - Operation not permitted worker2 # [ 11.392953] dhcpcd[639]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 11.402614] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.389865] dhcpcd[668]: .resolvconf-wrapped: clearing stale lock pid 637 worker2 # [ 11.400210] dhcpcd[639]: eth0: probing address 10.0.2.15/24 client # [ 11.412889] network-addresses-eth1-start[625]: adding address 192.168.1.1/24... done client # [ 11.436773] network-addresses-eth1-start[625]: adding address 2001:db8:1::1/64... done worker1 # [ 11.521801] dhcpcd[637]: eth0: soliciting a DHCP lease worker1 # [ 11.528569] dhcpcd[637]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.454429] systemd-logind[486]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.474886] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.536738] dhcpcd[637]: eth0: probing address 10.0.2.15/24 client # [ 11.461677] systemd[1]: Finished Address configuration of eth1. lb # [ 11.462395] dhcpcd[625]: eth0: waiting for carrier lb # [ 11.463163] dhcpcd[625]: libudev: received NULL device lb # [ 11.463894] dhcpcd[625]: libudev: received NULL device lb # [ 11.466133] dhcpcd[625]: eth0: carrier acquired lb # [ 11.490317] dhcpcd[625]: DUID 00:01:00:01:32:46:20:db:52:54:00:12:34:56 lb # [ 11.491257] dhcpcd[625]: eth0: IAID 00:12:34:56 lb # [ 11.491894] dhcpcd[625]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.519491] systemd[1]: Finished Extra networking commands.. lb # [ 11.528455] systemd[1]: Reached target Network. lb # [ 11.532082] systemd[1]: Starting PostgreSQL Server... lb # [ 11.536954] systemd[1]: Starting Permit User Sessions... client # [ 11.546641] systemd[1]: Finished Extra networking commands.. client # [ 11.551258] systemd[1]: Reached target Network. client # [ 11.553623] systemd[1]: Starting Permit User Sessions... client # [ 11.599966] systemd[1]: Finished Permit User Sessions. client # [ 11.605914] systemd[1]: Started Getty on tty1. client # [ 11.608480] systemd[1]: Reached target Login Prompts. lb # [ 11.632320] systemd[1]: Finished Permit User Sessions. lb # [ 11.645212] systemd[1]: Started Getty on tty1. lb # [ 11.645994] systemd[1]: Reached target Login Prompts. worker2 # [ 11.841809] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 11.824749] postgresql-pre-start[727]: The files belonging to this database system will be owned by user "postgres". lb # [ 11.826156] postgresql-pre-start[727]: This user must also own the server process. client: (finished: waiting for unit network-addresses-eth1.service, in 12.23 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 11.841468] postgresql-pre-start[727]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 11.844650] postgresql-pre-start[727]: The default database encoding has accordingly been set to "UTF8". lb # [ 11.845906] postgresql-pre-start[727]: The default text search configuration will be set to "english". lb # [ 11.847093] postgresql-pre-start[727]: Data page checksums are enabled. lb # [ 11.847950] postgresql-pre-start[727]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 11.854572] postgresql-pre-start[727]: creating subdirectories ... ok lb # [ 11.855445] postgresql-pre-start[727]: selecting dynamic shared memory implementation ... posix worker1 # [ 11.960514] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.08 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 11.992656] postgresql-pre-start[727]: selecting default "max_connections" ... 100 client # [ 12.023281] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.025701] systemd[1]: Stopped Virtual Console Setup. client # [ 12.026463] systemd[1]: Stopping Virtual Console Setup... lb: (finished: waiting for unit network-addresses-eth1.service, in 0.11 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } client # [ 12.035863] systemd[1]: Starting Virtual Console Setup... client # [ 12.043848] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.049682] systemd[1]: Stopped Virtual Console Setup. client # [ 12.052811] systemd[1]: Starting Virtual Console Setup... client # [ 12.102999] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 12.114282] systemd-logind[488]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.09 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 12.137699] postgresql-pre-start[727]: selecting default "shared_buffers" ... 128MB worker2 # [ 12.159094] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 12.165082] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.193826] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.194875] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.197216] systemd[1]: Starting Virtual Console Setup... worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } client # [ 12.195423] systemd-vconsole-setup[702]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.199047] systemd[1]: Finished Virtual Console Setup. worker1 # [ 12.291178] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.246543] systemd-logind[503]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.318437] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.07 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 12.356459] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 12.359874] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.364482] systemd[1]: Starting Virtual Console Setup... worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.03 seconds) lb: waiting for unit niks3.service worker1 # [ 12.427119] systemd-logind[510]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 12.372670] systemd-vconsole-setup[692]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 12.377203] systemd[1]: Finished Virtual Console Setup. worker1 # [ 12.507140] systemd-vconsole-setup[704]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 12.509847] systemd[1]: Finished Virtual Console Setup. client # [ 12.514568] dhcpcd[602]: eth0: soliciting a DHCP lease client # [ 12.521030] dhcpcd[602]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.524482] dhcpcd[602]: eth0: probing address 10.0.2.15/24 worker1 # [ 12.713018] dhcpcd[637]: eth0: soliciting an IPv6 router worker1 # [ 12.716527] dhcpcd[637]: eth0: Router Advertisement from fe80::2 worker1 # [ 12.719090] dhcpcd[637]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 12.722057] dhcpcd[637]: eth0: adding route to fec0::/64 lb # [ 12.662768] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1 # [ 12.724334] dhcpcd[637]: eth0: adding default route via fe80::2 lb # [ 12.878968] dhcpcd[625]: eth0: soliciting an IPv6 router lb # [ 12.879463] dhcpcd[625]: eth0: Router Advertisement from fe80::2 lb # [ 12.879562] dhcpcd[625]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 12.879607] dhcpcd[625]: eth0: adding route to fec0::/64 lb # [ 12.879647] dhcpcd[625]: eth0: adding default route via fe80::2 lb # [ 12.971388] systemd[1]: Starting Virtual Console Setup... lb # [ 12.989842] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 12.990907] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.012588] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.018467] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.039482] dhcpcd[639]: eth0: soliciting an IPv6 router worker2 # [ 13.042855] dhcpcd[639]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.045648] dhcpcd[639]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.048394] dhcpcd[639]: eth0: adding route to fec0::/64 worker2 # [ 13.050584] dhcpcd[639]: eth0: adding default route via fe80::2 lb # [ 13.070470] systemd-logind[486]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 13.235360] dhcpcd[602]: eth0: soliciting an IPv6 router client # [ 13.238878] dhcpcd[602]: eth0: Router Advertisement from fe80::2 lb # [ 13.241739] systemd-vconsole-setup[757]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.241765] dhcpcd[602]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 13.245434] systemd[1]: Finished Virtual Console Setup. client # [ 13.244564] dhcpcd[602]: eth0: adding route to fec0::/64 client # [ 13.246763] dhcpcd[602]: eth0: adding default route via fe80::2 lb # [ 13.485451] dhcpcd[625]: eth0: soliciting a DHCP lease lb # [ 13.492587] dhcpcd[625]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.500275] dhcpcd[625]: eth0: probing address 10.0.2.15/24 lb # [ 13.659505] postgresql-pre-start[727]: selecting default time zone ... UTC lb # [ 13.662279] postgresql-pre-start[727]: creating configuration files ... ok lb # [ 13.874537] postgresql-pre-start[727]: running bootstrap script ... ok lb # [ 14.370345] postgresql-pre-start[727]: performing post-bootstrap initialization ... ok lb # [ 14.526376] postgresql-pre-start[727]: syncing data to disk ... ok lb # [ 14.527330] postgresql-pre-start[727]: initdb: warning: enabling "trust" authentication for local connections lb # [ 14.528729] postgresql-pre-start[727]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 14.530830] postgresql-pre-start[727]: Success. You can now start the database server using: lb # [ 14.531892] postgresql-pre-start[727]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 14.695489] postgres[781]: [781] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 14.699755] postgres[781]: [781] LOG: listening on IPv6 address "::1", port 5432 lb # [ 14.701133] postgres[781]: [781] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 14.703601] postgres[781]: [781] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 14.713680] postgres[800]: [800] LOG: database system was shut down at 2026-09-23 05:32:13 GMT lb # [ 14.720320] postgres[781]: [781] LOG: database system is ready to accept connections lb # [ 14.724399] systemd[1]: Started PostgreSQL Server. lb # [ 14.728333] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 14.887500] postgresql-setup-start[811]: CREATE DATABASE lb # [ 14.922794] postgresql-setup-start[816]: CREATE ROLE lb # [ 14.940410] postgresql-setup-start[818]: ALTER DATABASE lb # [ 14.943716] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 14.945753] systemd[1]: Reached target PostgreSQL. worker1 # [ 15.839607] dhcpcd[637]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 15.843262] dhcpcd[637]: eth0: adding route to 10.0.2.0/24 worker1 # [ 15.845926] dhcpcd[637]: eth0: adding default route via 10.0.2.2 worker1 # [ 15.998889] systemd[1]: Started DHCP Client. worker1 # [ 16.001236] systemd[1]: Reached target Multi-User System. worker1 # [ 16.002167] systemd[1]: Startup finished in 963ms (kernel) + 4.952s (initrd) + 10.083s (userspace) = 16.000s. client # [ 17.207374] dhcpcd[602]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.211141] dhcpcd[602]: eth0: adding route to 10.0.2.0/24 client # [ 17.216381] dhcpcd[602]: eth0: adding default route via 10.0.2.2 worker2 # [ 17.231453] dhcpcd[639]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 17.235913] dhcpcd[639]: eth0: adding route to 10.0.2.0/24 worker2 # [ 17.238379] dhcpcd[639]: eth0: adding default route via 10.0.2.2 worker2 # [ 17.393005] systemd[1]: Started DHCP Client. worker2 # [ 17.394987] systemd[1]: Reached target Multi-User System. worker2 # [ 17.395815] systemd[1]: Startup finished in 952ms (kernel) + 4.952s (initrd) + 11.489s (userspace) = 17.394s. client # [ 17.404396] systemd[1]: Started DHCP Client. client # [ 17.407940] systemd[1]: Reached target Multi-User System. client # [ 17.410087] systemd[1]: Startup finished in 959ms (kernel) + 4.107s (initrd) + 12.342s (userspace) = 17.409s. lb # [ 18.814168] dhcpcd[625]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 18.823097] dhcpcd[625]: eth0: adding route to 10.0.2.0/24 lb # [ 18.827210] dhcpcd[625]: eth0: adding default route via 10.0.2.2 lb # [ 18.990318] systemd[1]: Started DHCP Client. lb # [ 18.993318] systemd[1]: Reached target Network is Online. lb # [ 18.999132] systemd[1]: Started Envoy reverse proxy. lb # [ 19.012830] systemd[1]: Started mock-oidc.service. lb # [ 19.173060] mock-oidc-server[905]: Mock OIDC Server running lb # [ 19.176763] mock-oidc-server[905]: OIDC Address: 192.168.1.2:8080 lb # [ 19.178886] mock-oidc-server[905]: Issue Address: 0.0.0.0:8081 lb # [ 19.181147] mock-oidc-server[905]: Issuer: http://192.168.1.2:8080/oidc lb # [ 19.182059] mock-oidc-server[905]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 19.183143] mock-oidc-server[905]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 19.189278] mock-oidc-server[905]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 19.382376] envoy[904]: [2026-09-23 05:32:18.985][904][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 19.385237] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 19.386707] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 19.392157] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 19.394629] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 19.397385] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 19.399207] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 19.402558] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 19.409985] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 19.412612] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 19.416290] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 19.419098] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 19.424255] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 19.451501] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 19.454761] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 19.458123] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 19.461274] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 19.464122] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 19.466261] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 19.469646] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 19.477427] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 19.484080] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 19.486263] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 19.487960] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 19.491976] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 19.493738] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 19.495599] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 19.499251] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 19.502736] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 19.504525] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 19.506418] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 19.508377] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 19.510359] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 19.513241] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 19.515065] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 19.520220] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 19.552144] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 19.554562] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 19.556349] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 19.558174] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 19.561463] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 19.564412] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 19.566871] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 19.569540] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 19.574140] envoy[904]: [2026-09-23 05:32:18.987][904][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 19.577740] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 19.580026] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 19.581968] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 19.589400] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 19.591670] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 19.594169] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 19.597124] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 19.601258] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 19.603377] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 19.605565] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 19.612571] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 19.615102] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 19.617925] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 19.619798] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle worker2 # [ 19.630151] systemd[1]: Reached target Network is Online. lb # [ 19.623435] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 19.628546] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 19.631347] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 19.633102] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.635603] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router worker2 # [ 19.645786] systemd[1]: Starting scheduler-token.service... worker1 # [ 19.714839] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 19.637423] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 19.645581] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.648063] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 19.653266] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 19.654907] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 19.657059] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 19.658572] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 19.660318] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 19.662830] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 19.664480] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 19.666225] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 19.671366] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 19.674942] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 19.678213] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.680523] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 19.684833] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 19.687004] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 19.688925] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 19.690874] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 19.693535] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 19.696064] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 19.697594] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 19.699496] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 19.712914] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 19.714547] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 19.719060] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 19.724036] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 19.728684] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 19.735711] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 19.737806] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 19.740776] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 19.745048] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 19.747674] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 19.751341] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 19.753938] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 worker2 # [ 19.766827] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 19.755791] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params worker2 # [ 19.772689] systemd[1]: Finished scheduler-token.service. lb # [ 19.766755] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 19.768680] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name worker2 # [ 19.774393] systemd[1]: scheduler-token.service: Consumed 80ms CPU time over 125ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 19.771116] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.773184] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 19.775696] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 19.778351] envoy[904]: [2026-09-23 05:32:18.988][904][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 19.784623] envoy[904]: [2026-09-23 05:32:19.012][904][info][main] [source/server/server.cc:531] HTTP header map info: worker2 # [ 19.796446] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 19.785958] envoy[904]: [2026-09-23 05:32:19.013][904][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 19.801509] envoy[904]: [2026-09-23 05:32:19.013][904][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 19.803029] envoy[904]: [2026-09-23 05:32:19.013][904][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 19.812065] envoy[904]: [2026-09-23 05:32:19.013][904][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 19.813821] envoy[904]: [2026-09-23 05:32:19.053][904][info][main] [source/server/server.cc:968] runtime: {} lb # [ 19.815058] envoy[904]: [2026-09-23 05:32:19.057][904][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 19.816823] envoy[904]: [2026-09-23 05:32:19.057][904][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 19.818440] envoy[904]: [2026-09-23 05:32:19.057][904][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 19.819914] envoy[904]: [2026-09-23 05:32:19.058][904][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 19.821598] envoy[904]: [2026-09-23 05:32:19.058][904][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 19.823165] envoy[904]: [2026-09-23 05:32:19.058][904][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 19.824755] envoy[904]: [2026-09-23 05:32:19.214][904][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 19.826269] envoy[904]: [2026-09-23 05:32:19.230][904][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 19.827845] envoy[904]: [2026-09-23 05:32:19.231][904][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 19.831763] envoy[904]: [2026-09-23 05:32:19.231][904][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 19.970105] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 19.982346] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 19.984457] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 19.998931] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 20.001934] nix-grpc-daemon[769]: E0000 00:00:1790141539.602424 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 20.007261] nix-grpc-daemon[769]: E0000 00:00:1790141539.611305 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 20.027171] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 20.033471] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 20.034444] nix-grpc-daemon[769]: E0000 00:00:1790141539.635556 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.970589] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 19.969667] envoy[904]: [2026-09-23 05:32:19.572][904][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization worker2 # [ 19.981596] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 19.983226] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 19.979232] envoy[904]: [2026-09-23 05:32:19.578][904][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker2 # [ 19.997296] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 20.000413] nix-grpc-daemon[769]: E0000 00:00:1790141539.613120 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 20.072068] nix-grpc-daemon[769]: E0000 00:00:1790141539.671009 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 20.074456] nix-grpc-daemon[769]: E0000 00:00:1790141539.671996 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.006033] nix-grpc-daemon[769]: E0000 00:00:1790141539.622140 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.021311] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 20.024245] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 20.097786] nix-grpc-daemon[769]: E0000 00:00:1790141539.700265 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 20.101992] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.036175] nix-grpc-daemon[769]: E0000 00:00:1790141539.642144 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.050112] nix-grpc-daemon[769]: E0000 00:00:1790141539.666390 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.064266] nix-grpc-daemon[769]: E0000 00:00:1790141539.669036 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.078614] nix-grpc-daemon[769]: E0000 00:00:1790141539.693860 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.082606] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 20.303521] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.283712] nix-grpc-daemon[769]: ts=2026-09-23T05:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 20.704817] nix-grpc-daemon[769]: ts=2026-09-23T05:32:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.684881] nix-grpc-daemon[769]: ts=2026-09-23T05:32:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 21.506514] nix-grpc-daemon[769]: ts=2026-09-23T05:32:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 21.486614] nix-grpc-daemon[769]: ts=2026-09-23T05:32:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 23.117066] nix-grpc-daemon[769]: ts=2026-09-23T05:32:22Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 23.086910] nix-grpc-daemon[769]: ts=2026-09-23T05:32:22Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 24.327183] rustfs-bucket-start[965]: mb s3://farm lb # [ 24.336870] systemd[1]: Finished rustfs-bucket.service. lb # [ 24.348981] systemd[1]: Starting niks3 server... lb # [ 24.485648] postgres[980]: [980] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 24.486898] postgres[980]: [980] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 24.512615] niks3-server[975]: 2026/09/23 05:32:24 OK 20241026095416_initial_model.sql (14.84ms) lb # [ 24.516220] niks3-server[975]: 2026/09/23 05:32:24 OK 20251210153512_drop_unused_gin_index.sql (3.85ms) lb # [ 24.521811] niks3-server[975]: 2026/09/23 05:32:24 OK 20251218171726_add_pins.sql (5.53ms) lb # [ 24.526400] niks3-server[975]: 2026/09/23 05:32:24 OK 20260628120000_add_object_size_and_stats.sql (4.56ms) lb # [ 24.531061] niks3-server[975]: 2026/09/23 05:32:24 OK 20260905000000_add_claims.sql (4.63ms) lb # [ 24.534741] niks3-server[975]: 2026/09/23 05:32:24 OK 20260920000000_drop_claims.sql (3.58ms) lb # [ 24.536746] niks3-server[975]: 2026/09/23 05:32:24 goose: successfully migrated database to version: 20260920000000 lb # [ 24.540582] niks3-server[975]: 2026/09/23 05:32:24 OK 1_commit_pending_closure.sql (5.82ms) lb # [ 24.543096] niks3-server[975]: 2026/09/23 05:32:24 OK 2_object_stats_trigger.sql (2.38ms) lb # [ 24.546209] niks3-server[975]: 2026/09/23 05:32:24 goose: up to current file version: 2 lb # [ 24.548695] niks3-server[975]: 2026/09/23 05:32:24 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 24.582522] niks3-server[975]: 2026/09/23 05:32:24 INFO Read proxy enabled — serving cache objects from S3 lb # [ 24.590849] niks3-server[975]: 2026/09/23 05:32:24 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 24.593709] niks3-server[975]: 2026/09/23 05:32:24 INFO systemd watchdog enabled interval=15s lb # [ 24.594865] niks3-server[975]: 2026/09/23 05:32:24 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 24.597934] systemd[1]: Started niks3 server. lb # [ 24.599196] systemd[1]: Reached target Multi-User System. lb # [ 24.601193] systemd[1]: Startup finished in 966ms (kernel) + 4.956s (initrd) + 18.669s (userspace) = 24.592s. lb: (finished: waiting for unit niks3.service, in 12.45 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.04 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 26.317694] nix-grpc-daemon[769]: ts=2026-09-23T05:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 26.287573] nix-grpc-daemon[769]: ts=2026-09-23T05:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 31.318258] nix-grpc-daemon[769]: ts=2026-09-23T05:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 31.288191] nix-grpc-daemon[769]: ts=2026-09-23T05:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 34.595736] niks3-server[975]: 2026/09/23 05:32:34 INFO lead: acquired remote=192.168.1.3:53876 worker1 # [ 34.673239] nix-grpc-daemon[769]: ts=2026-09-23T05:32:34Z level=info event=scheduler_take_over lb # [ 34.969025] envoy[904]: [2026-09-23 05:32:34.570][904][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 34.978669] envoy[904]: [2026-09-23 05:32:34.581][904][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 34.984590] envoy[904]: [2026-09-23 05:32:34.581][904][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 34.995031] envoy[904]: [2026-09-23 05:32:34.596][904][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 35.409211] envoy[904]: conn peer=[::1]:53444 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=7 lb: (finished: waiting for TCP port 50051 on localhost, in 10.54 seconds) worker2 # [ 35.595705] systemd[1]: Starting scheduler-token.service... worker2 # [ 35.676997] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 35.687286] systemd[1]: Finished scheduler-token.service. worker2 # [ 35.690958] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 36.319525] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.337229] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53214 worker1 # [ 36.342942] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 36.289812] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.368128] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 36.373344] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53214 worker1 # [ 36.378705] nix-grpc-daemon[769]: ts=2026-09-23T05:32:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker1 # [ 38.969168] nix-grpc-daemon[769]: ts=2026-09-23T05:32:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:53226 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 3.31 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.71 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 40.006146] nix-grpc-daemon[769]: ts=2026-09-23T05:32:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 40.022434] systemd[1]: Started Nix Daemon. worker2 # [ 40.212756] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 40.233636] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5) worker2 # [ 40.323296] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 2/5) worker2 # [ 40.386139] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 3/5) worker2 # [ 40.532828] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 485 ms (attempt 4/5) worker2 # [ 41.021542] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 41.026058] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 41.038905] nix-grpc-daemon[769]: ts=2026-09-23T05:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 41.047575] nix-grpc-daemon[769]: ts=2026-09-23T05:32:40Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 41.149281] nix-grpc-daemon[769]: ts=2026-09-23T05:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 41.161176] systemd[1]: Started Nix Daemon. worker1 # [ 41.347041] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 41.369019] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 1/5) worker1 # [ 41.469500] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5) worker1 # [ 41.569297] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 394 ms (attempt 3/5) worker1 # [ 41.965207] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 679 ms (attempt 4/5) worker1 # [ 42.648744] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 42.652909] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 42.666682] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker2 # [ 42.925263] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 42.931633] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34358 worker2 # [ 42.944138] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34358 worker2 # [ 42.955624] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 42.969971] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34358 worker2 # [ 42.977857] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 43.215637] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 43.376122] nix-grpc-daemon[769]: ts=2026-09-23T05:32:42Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.69 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 43.615274] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 43.555808] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 43.665961] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 43.672145] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 43.709446] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 43.726606] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 client # copying 1 paths to 'lb:50051'... worker1 # [ 43.742588] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 43.765413] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 43.809193] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=info event=niks3_push_started pid=867 worker1 # [ 43.994427] nix-grpc-daemon[867]: time=2026-09-23T05:32:43.597Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 44.104097] nix-daemon[854]: accepted connection from pid 880, user nix-grpc-daemon (trusted) worker1 # [ 44.124570] nix-daemon[854]: reaped child process 883, status = succeeded lb # [ 44.053634] niks3-server[975]: 2026/09/23 05:32:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 44.152873] nix-grpc-daemon[867]: time=2026-09-23T05:32:43.755Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 44.157137] nix-grpc-daemon[867]: time=2026-09-23T05:32:43.755Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 44.125962] niks3-server[975]: 2026/09/23 05:32:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 44.145078] niks3-server[975]: 2026/09/23 05:32:43 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 44.154098] niks3-server[975]: 2026/09/23 05:32:43 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 44.172953] niks3-server[975]: 2026/09/23 05:32:43 INFO Signed narinfos id=1 count=1 worker1 # [ 44.254778] nix-grpc-daemon[867]: time=2026-09-23T05:32:43.856Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.187937] niks3-server[975]: 2026/09/23 05:32:43 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 44.202712] niks3-server[975]: 2026/09/23 05:32:43 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo lb # [ 44.210880] niks3-server[975]: 2026/09/23 05:32:43 INFO Completed upload id=1 worker1 # [ 44.288701] nix-grpc-daemon[867]: time=2026-09-23T05:32:43.891Z level=INFO msg="Upload complete. (288ms)" worker1 # [ 44.300243] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 44.317990] nix-grpc-daemon[769]: ts=2026-09-23T05:32:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 44.339116] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 44.345166] nix-daemon[854]: reaped child process 865, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 44.365205] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 45.598997] nix-daemon[854]: reaped child process 887, status = succeeded worker1 # [ 45.695670] nix-daemon[854]: accepted connection from pid 904, user nix-grpc-daemon (trusted) worker1 # [ 45.711434] nix-daemon[854]: reaped child process 907, status = succeeded lb # [ 45.641925] niks3-server[975]: 2026/09/23 05:32:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.732371] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 45.733828] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.334Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 45.689383] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 45.700888] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 45.786558] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.390Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.713410] niks3-server[975]: 2026/09/23 05:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 45.719640] niks3-server[975]: 2026/09/23 05:32:45 INFO Signed narinfos id=2 count=1 lb # [ 45.725141] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 45.734123] niks3-server[975]: 2026/09/23 05:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 45.739213] niks3-server[975]: 2026/09/23 05:32:45 INFO Completed upload id=2 worker1 # [ 45.816282] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.420Z level=INFO msg="Upload complete. (235ms)" lb # [ 45.741757] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 45.836817] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 45.851354] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 45.856686] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 45.861079] nix-daemon[854]: reaped child process 885, status = succeeded worker1 # [ 45.881272] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 45.895136] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 45.909097] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.513Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 45.919634] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 45.945649] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 46.101816] nix-daemon[854]: accepted connection from pid 913, user nix-grpc-daemon (trusted) worker1 # [ 46.120727] nix-daemon[854]: reaped child process 916, status = succeeded lb # [ 46.042043] niks3-server[975]: 2026/09/23 05:32:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 46.129210] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.732Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 46.130706] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.732Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 46.073541] niks3-server[975]: 2026/09/23 05:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign worker1 # [ 46.159824] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.763Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.090744] niks3-server[975]: 2026/09/23 05:32:45 INFO Signed narinfos id=3 count=1 lb # [ 46.097792] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 46.111535] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker1 # [ 46.193611] nix-grpc-daemon[867]: time=2026-09-23T05:32:45.797Z level=INFO msg="Upload complete. (230ms)" lb # [ 46.120052] niks3-server[975]: 2026/09/23 05:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 46.126009] niks3-server[975]: 2026/09/23 05:32:45 INFO Completed upload id=3 worker1 # [ 46.204266] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 46.129944] niks3-server[975]: 2026/09/23 05:32:45 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 46.219858] nix-grpc-daemon[769]: ts=2026-09-23T05:32:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 46.228593] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 46.236145] nix-daemon[854]: reaped child process 911, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 46.244135] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 47.320153] nix-daemon[854]: reaped child process 920, status = succeeded worker1 # [ 47.416871] nix-daemon[854]: accepted connection from pid 936, user nix-grpc-daemon (trusted) worker1 # [ 47.437160] nix-daemon[854]: reaped child process 939, status = succeeded lb # [ 47.359543] niks3-server[975]: 2026/09/23 05:32:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 47.449282] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.052Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 47.451040] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.052Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 47.400007] niks3-server[975]: 2026/09/23 05:32:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker1 # [ 47.486083] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.089Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.416838] niks3-server[975]: 2026/09/23 05:32:47 INFO Signed narinfos id=4 count=1 lb # [ 47.424151] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 47.435916] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 47.445181] niks3-server[975]: 2026/09/23 05:32:47 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 47.448738] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 47.453659] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo lb # [ 47.459601] niks3-server[975]: 2026/09/23 05:32:47 INFO Completed upload id=4 worker1 # [ 47.536384] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.140Z level=INFO msg="Upload complete. (282ms)" worker1 # [ 47.556971] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 47.571868] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 47.576559] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 47.581106] nix-daemon[854]: reaped child process 918, status = succeeded worker1 # [ 47.597029] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 47.611964] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 47.625507] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.229Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 47.636410] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 47.653408] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 47.803629] nix-daemon[854]: accepted connection from pid 945, user nix-grpc-daemon (trusted) worker1 # [ 47.820505] nix-daemon[854]: reaped child process 948, status = succeeded lb # [ 47.748189] niks3-server[975]: 2026/09/23 05:32:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 47.834484] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 47.835957] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.438Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 47.779249] niks3-server[975]: 2026/09/23 05:32:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker1 # [ 47.862977] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.466Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.793515] niks3-server[975]: 2026/09/23 05:32:47 INFO Signed narinfos id=5 count=1 lb # [ 47.801377] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 47.817243] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 47.821619] niks3-server[975]: 2026/09/23 05:32:47 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete worker1 # [ 47.903747] nix-grpc-daemon[867]: time=2026-09-23T05:32:47.507Z level=INFO msg="Upload complete. (233ms)" lb # [ 47.827105] niks3-server[975]: 2026/09/23 05:32:47 INFO Completed upload id=5 lb # [ 47.834414] niks3-server[975]: 2026/09/23 05:32:47 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 47.916282] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 47.928303] nix-grpc-daemon[769]: ts=2026-09-23T05:32:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 47.936190] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 47.944387] nix-daemon[854]: reaped child process 943, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker1 # [ 47.952743] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 49.219140] nix-daemon[854]: reaped child process 952, status = succeeded worker1 # [ 49.333421] nix-daemon[854]: accepted connection from pid 968, user nix-grpc-daemon (trusted) worker1 # [ 49.352650] nix-daemon[854]: reaped child process 971, status = succeeded lb # [ 49.284724] niks3-server[975]: 2026/09/23 05:32:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 49.374587] nix-grpc-daemon[867]: time=2026-09-23T05:32:48.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 49.376365] nix-grpc-daemon[867]: time=2026-09-23T05:32:48.976Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 49.330496] niks3-server[975]: 2026/09/23 05:32:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 49.414732] nix-grpc-daemon[867]: time=2026-09-23T05:32:49.018Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.344000] niks3-server[975]: 2026/09/23 05:32:48 INFO Signed narinfos id=6 count=1 lb # [ 49.352359] niks3-server[975]: 2026/09/23 05:32:48 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 49.362149] niks3-server[975]: 2026/09/23 05:32:48 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 49.369557] niks3-server[975]: 2026/09/23 05:32:48 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 49.379327] niks3-server[975]: 2026/09/23 05:32:48 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 49.385523] niks3-server[975]: 2026/09/23 05:32:48 INFO Completed upload id=6 worker1 # [ 49.462554] nix-grpc-daemon[867]: time=2026-09-23T05:32:49.066Z level=INFO msg="Upload complete. (298ms)" lb # [ 49.389955] niks3-server[975]: 2026/09/23 05:32:48 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 49.477332] nix-grpc-daemon[769]: ts=2026-09-23T05:32:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 49.498073] nix-daemon[854]: reaped child process 950, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.05 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath lb # [ 49.616408] envoy[904]: conn peer=[2001:db8:1::1]:35894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=2632 lb # [ 49.621369] envoy[904]: conn peer=[2001:db8:1::1]:33862 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=75 lb # [ 49.625912] envoy[904]: conn peer=[2001:db8:1::1]:33866 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=9 client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds) lb # [ 49.630403] envoy[904]: conn peer=[2001:db8:1::1]:33880 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=16 lb # [ 49.635299] envoy[904]: conn peer=[2001:db8:1::1]:33884 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9506 tx=4386 ms=5879 client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... worker1 # [ 50.036110] nix-grpc-daemon[769]: E0000 00:00:1790141569.637615 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.58 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # [ 50.085442] nix-grpc-daemon[769]: E0000 00:00:1790141569.665658 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 50.087950] nix-grpc-daemon[769]: E0000 00:00:1790141569.665889 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 50.096152] nix-grpc-daemon[769]: E0000 00:00:1790141569.686760 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 50.036813] nix-grpc-daemon[769]: E0000 00:00:1790141569.646439 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 50.068078] nix-grpc-daemon[769]: E0000 00:00:1790141569.683832 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 50.092564] nix-grpc-daemon[769]: E0000 00:00:1790141569.688976 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 50.095487] nix-grpc-daemon[769]: E0000 00:00:1790141569.707901 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.20 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 50.702720] nix-grpc-daemon[769]: ts=2026-09-23T05:32:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # [ 50.723361] nix-grpc-daemon[769]: ts=2026-09-23T05:32:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 50.735621] nix-grpc-daemon[867]: time=2026-09-23T05:32:50.338Z level=INFO msg="All 2 paths already cached" worker1 # [ 50.755357] nix-grpc-daemon[769]: ts=2026-09-23T05:32:50Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.31 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.17 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 51.487743] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 51.560616] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 51.571880] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.175Z level=INFO msg="All 2 paths already cached" worker1 # [ 51.587712] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 51.594778] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 51.633041] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 51.649490] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 51.664818] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.268Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 51.675910] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 51.690780] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 51.849218] nix-daemon[854]: accepted connection from pid 1000, user nix-grpc-daemon (trusted) worker1 # [ 51.869434] nix-daemon[854]: reaped child process 1003, status = succeeded lb # [ 51.791732] niks3-server[975]: 2026/09/23 05:32:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 51.879733] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.482Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 51.881920] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.483Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 51.826886] niks3-server[975]: 2026/09/23 05:32:51 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 51.913748] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.517Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.843382] niks3-server[975]: 2026/09/23 05:32:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 51.853252] niks3-server[975]: 2026/09/23 05:32:51 INFO Signed narinfos id=7 count=1 lb # [ 51.856003] niks3-server[975]: 2026/09/23 05:32:51 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls worker1 # [ 51.945138] nix-grpc-daemon[867]: time=2026-09-23T05:32:51.549Z level=INFO msg="Upload complete. (237ms)" lb # [ 51.871090] niks3-server[975]: 2026/09/23 05:32:51 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 51.876354] niks3-server[975]: 2026/09/23 05:32:51 INFO Completed upload id=7 lb # [ 51.880083] niks3-server[975]: 2026/09/23 05:32:51 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 51.957985] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 51.972900] nix-grpc-daemon[769]: ts=2026-09-23T05:32:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 51.982842] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 51.986575] nix-daemon[854]: reaped child process 998, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 52.001856] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 53.253952] nix-daemon[854]: reaped child process 1007, status = succeeded worker1 # [ 53.352386] nix-daemon[854]: accepted connection from pid 1023, user nix-grpc-daemon (trusted) worker1 # [ 53.373297] nix-daemon[854]: reaped child process 1026, status = succeeded lb # [ 53.301724] niks3-server[975]: 2026/09/23 05:32:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.390814] nix-grpc-daemon[867]: time=2026-09-23T05:32:52.992Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 53.392507] nix-grpc-daemon[867]: time=2026-09-23T05:32:52.992Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 53.346578] niks3-server[975]: 2026/09/23 05:32:52 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 53.435379] nix-grpc-daemon[867]: time=2026-09-23T05:32:53.039Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.355710] niks3-server[975]: 2026/09/23 05:32:52 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 53.370947] niks3-server[975]: 2026/09/23 05:32:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 53.378810] niks3-server[975]: 2026/09/23 05:32:52 INFO Signed narinfos id=8 count=1 lb # [ 53.383121] niks3-server[975]: 2026/09/23 05:32:52 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls worker1 # [ 53.470420] nix-grpc-daemon[867]: time=2026-09-23T05:32:53.074Z level=INFO msg="Upload complete. (255ms)" lb # [ 53.393599] niks3-server[975]: 2026/09/23 05:32:52 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 53.398987] niks3-server[975]: 2026/09/23 05:32:52 INFO Completed upload id=8 lb # [ 53.402769] niks3-server[975]: 2026/09/23 05:32:53 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 53.485768] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 53.502843] nix-daemon[854]: reaped child process 1005, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.14 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.66 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker1 # [ 54.088837] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 worker2 # [ 54.025309] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 54.118273] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker1 # [ 54.124761] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 54.057842] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 54.073583] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 54.089681] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 54.105229] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 54.135838] nix-grpc-daemon[769]: ts=2026-09-23T05:32:53Z level=info event=niks3_push_started pid=866 worker2 # [ 54.305445] nix-grpc-daemon[866]: time=2026-09-23T05:32:53.920Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 54.410518] nix-daemon[839]: accepted connection from pid 879, user nix-grpc-daemon (trusted) worker2 # [ 54.431084] nix-daemon[839]: reaped child process 882, status = succeeded lb # [ 54.430812] niks3-server[975]: 2026/09/23 05:32:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 54.447982] nix-grpc-daemon[866]: time=2026-09-23T05:32:54.063Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 54.449487] nix-grpc-daemon[866]: time=2026-09-23T05:32:54.063Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" worker2 # [ 54.487573] nix-grpc-daemon[866]: time=2026-09-23T05:32:54.103Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.469271] niks3-server[975]: 2026/09/23 05:32:54 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 54.483452] niks3-server[975]: 2026/09/23 05:32:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 54.494109] niks3-server[975]: 2026/09/23 05:32:54 INFO Signed narinfos id=9 count=1 lb # [ 54.503260] niks3-server[975]: 2026/09/23 05:32:54 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls worker2 # [ 54.516709] nix-grpc-daemon[866]: time=2026-09-23T05:32:54.133Z level=INFO msg="Upload complete. (212ms)" lb # [ 54.512224] niks3-server[975]: 2026/09/23 05:32:54 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 54.518509] niks3-server[975]: 2026/09/23 05:32:54 INFO Completed upload id=9 worker2 # [ 54.524338] nix-grpc-daemon[769]: ts=2026-09-23T05:32:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 54.522703] niks3-server[975]: 2026/09/23 05:32:54 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 54.535659] nix-grpc-daemon[769]: ts=2026-09-23T05:32:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 54.542173] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 54.547576] nix-daemon[839]: reaped child process 864, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker2 # [ 54.559827] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 55.993489] nix-daemon[839]: reaped child process 886, status = succeeded worker2 # [ 56.105418] nix-daemon[839]: accepted connection from pid 902, user nix-grpc-daemon (trusted) worker2 # [ 56.128184] nix-daemon[839]: reaped child process 905, status = succeeded lb # [ 56.121753] niks3-server[975]: 2026/09/23 05:32:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 56.143352] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.757Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 56.146136] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.757Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 56.171191] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 56.187189] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 56.204916] niks3-server[975]: 2026/09/23 05:32:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker2 # [ 56.217299] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.832Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.211315] niks3-server[975]: 2026/09/23 05:32:55 INFO Signed narinfos id=10 count=1 lb # [ 56.217990] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 56.232145] niks3-server[975]: 2026/09/23 05:32:55 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 56.238835] niks3-server[975]: 2026/09/23 05:32:55 INFO Completed upload id=10 worker2 # [ 56.245338] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.861Z level=INFO msg="Upload complete. (275ms)" lb # [ 56.245222] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker2 # [ 56.261608] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker1 # [ 56.339717] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 56.346539] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 56.283188] nix-daemon[839]: reaped child process 884, status = succeeded worker2 # [ 56.317249] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 56.341164] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 56.358634] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.974Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 56.371479] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 56.386545] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 56.552465] nix-daemon[839]: accepted connection from pid 909, user nix-grpc-daemon (trusted) worker2 # [ 56.575597] nix-daemon[839]: reaped child process 912, status = succeeded lb # [ 56.573609] niks3-server[975]: 2026/09/23 05:32:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 56.592518] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.208Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 56.594375] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.208Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 56.613323] niks3-server[975]: 2026/09/23 05:32:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker2 # [ 56.634382] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.250Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.634785] niks3-server[975]: 2026/09/23 05:32:56 INFO Signed narinfos id=11 count=1 lb # [ 56.643380] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 56.654243] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 56.667442] niks3-server[975]: 2026/09/23 05:32:56 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 56.676380] niks3-server[975]: 2026/09/23 05:32:56 INFO Completed upload id=11 worker2 # [ 56.682340] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.298Z level=INFO msg="Upload complete. (276ms)" lb # [ 56.679381] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 56.694611] nix-grpc-daemon[769]: ts=2026-09-23T05:32:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 56.708739] nix-grpc-daemon[769]: ts=2026-09-23T05:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 56.722852] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 56.731407] nix-daemon[839]: reaped child process 907, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker2 # [ 56.747182] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 58.054253] nix-daemon[839]: reaped child process 916, status = succeeded worker2 # [ 58.140994] nix-daemon[839]: accepted connection from pid 932, user nix-grpc-daemon (trusted) worker2 # [ 58.163293] nix-daemon[839]: reaped child process 935, status = succeeded lb # [ 58.163245] niks3-server[975]: 2026/09/23 05:32:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 58.180534] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.796Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 58.182173] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.796Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 58.222255] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker2 # [ 58.239927] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.855Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.234336] niks3-server[975]: 2026/09/23 05:32:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 58.244333] niks3-server[975]: 2026/09/23 05:32:57 INFO Signed narinfos id=12 count=1 lb # [ 58.246945] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 58.260177] niks3-server[975]: 2026/09/23 05:32:57 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 58.273506] niks3-server[975]: 2026/09/23 05:32:57 INFO Completed upload id=12 lb # [ 58.275296] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker2 # [ 58.280561] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.896Z level=INFO msg="Upload complete. (317ms)" lb # [ 58.281090] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker1 # [ 58.378076] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 58.293541] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker1 # [ 58.378530] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 58.317562] nix-daemon[839]: reaped child process 914, status = succeeded worker2 # [ 58.352839] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 58.373364] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 58.390983] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.007Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 58.403781] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 58.418079] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 58.609987] nix-daemon[839]: accepted connection from pid 939, user nix-grpc-daemon (trusted) worker2 # [ 58.630544] nix-daemon[839]: reaped child process 942, status = succeeded lb # [ 58.630218] niks3-server[975]: 2026/09/23 05:32:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 58.649379] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.264Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 58.650887] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.264Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 58.674536] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls worker2 # [ 58.691953] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.307Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.687067] niks3-server[975]: 2026/09/23 05:32:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 58.698624] niks3-server[975]: 2026/09/23 05:32:58 INFO Signed narinfos id=13 count=1 lb # [ 58.704919] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst lb # [ 58.719645] niks3-server[975]: 2026/09/23 05:32:58 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete worker2 # [ 58.734794] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.351Z level=INFO msg="Upload complete. (296ms)" lb # [ 58.730493] niks3-server[975]: 2026/09/23 05:32:58 INFO Completed upload id=13 lb # [ 58.733107] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 58.748299] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 58.760748] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 58.772970] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 58.785452] nix-daemon[839]: reaped child process 937, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 58.795151] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 59.615864] envoy[904]: conn peer=[2001:db8:1::1]:33886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=77 client # farm-top> LOG-farm-top-sig worker2 # [ 59.859124] nix-daemon[839]: reaped child process 946, status = succeeded worker2 # [ 59.966642] nix-daemon[839]: accepted connection from pid 962, user nix-grpc-daemon (trusted) worker2 # [ 59.982928] nix-daemon[839]: reaped child process 965, status = succeeded lb # [ 59.621112] envoy[904]: conn peer=[2001:db8:1::1]:44036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1918 ms=1946 lb # [ 59.985837] niks3-server[975]: 2026/09/23 05:32:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 60.007607] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.621Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 60.009720] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.621Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 60.028605] niks3-server[975]: 2026/09/23 05:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 60.042969] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.659Z level=INFO msg="Uploading 1 narinfos" lb # [ 60.041396] niks3-server[975]: 2026/09/23 05:32:59 INFO Signed narinfos id=14 count=1 lb # [ 60.050648] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 60.062569] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 60.074490] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker2 # [ 60.089262] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.705Z level=INFO msg="Upload complete. (282ms)" lb # [ 60.083841] niks3-server[975]: 2026/09/23 05:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 60.087735] niks3-server[975]: 2026/09/23 05:32:59 INFO Completed upload id=14 lb # [ 60.091071] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 60.101418] nix-grpc-daemon[769]: ts=2026-09-23T05:32:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 60.121683] nix-daemon[839]: reaped child process 944, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.35 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 60.556330] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:49718 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 60.594541] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:49718 worker1 # [ 60.603861] nix-grpc-daemon[769]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 60.625973] nix-grpc-daemon[769]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 60.640892] nix-grpc-daemon[769]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 60.787096] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:37340 worker2 # [ 60.829707] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:37340 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/870" client # removing stale temporary roots file "/nix/var/nix/temproots/924" client # removing stale temporary roots file "/nix/var/nix/temproots/891" client # removing stale temporary roots file "/nix/var/nix/temproots/903" client # removing stale temporary roots file "/nix/var/nix/temproots/932" client # removing stale temporary roots file "/nix/var/nix/temproots/911" client # removing stale temporary roots file "/nix/var/nix/temproots/854" client # removing stale temporary roots file "/nix/var/nix/temproots/947" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 61.303593] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 61.310539] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 61.324332] nix-grpc-daemon[867]: time=2026-09-23T05:33:00.927Z level=INFO msg="All 1 paths already cached" worker1 # [ 61.332515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker2 # [ 61.273305] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying 3 paths... worker2 # [ 61.289291] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 61.319432] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 61.327347] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 61.409285] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 61.417551] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 61.362023] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 61.368733] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.57 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.67 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 61.518700] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: f6d7edc45aaf445c920eb10718d0e6db client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker2 # [ 61.716715] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34358 client # [ 61.717662] nix[999]: don't know how to build these paths: client # [ 61.719679] nix[999]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 61.809076] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 61.814514] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 61.856762] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 61.872116] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 client # [ 61.808258] nix[999]: copying 1 paths to 'lb:50051'... worker1 # [ 61.886641] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 client # [ 61.812158] nix[999]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 61.903866] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 62.080060] nix-daemon[854]: accepted connection from pid 1046, user nix-grpc-daemon (trusted) worker1 # [ 62.101928] nix-daemon[854]: reaped child process 1049, status = succeeded lb # [ 62.030133] niks3-server[975]: 2026/09/23 05:33:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 62.117660] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.720Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 62.119301] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.720Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 62.061093] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls worker1 # [ 62.149975] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.752Z level=INFO msg="Uploading 1 narinfos" lb # [ 62.076585] niks3-server[975]: 2026/09/23 05:33:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 62.088558] niks3-server[975]: 2026/09/23 05:33:01 INFO Signed narinfos id=15 count=1 lb # [ 62.095127] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker1 # [ 62.183546] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.787Z level=INFO msg="Upload complete. (259ms)" lb # [ 62.107151] niks3-server[975]: 2026/09/23 05:33:01 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 62.115182] niks3-server[975]: 2026/09/23 05:33:01 INFO Completed upload id=15 lb # [ 62.118872] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 62.196497] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 62.208149] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 62.219289] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 62.221986] nix-daemon[854]: reaped child process 1044, status = succeeded client # [ 62.156302] nix[999]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 62.236723] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # [ 62.614302] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 1b8859bce55f4915af7ce096bdd14586 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) worker2 # [ 62.851468] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34358 client # [ 62.852506] nix[1015]: don't know how to build these paths: client # [ 62.853513] nix[1015]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 62.941906] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 62.872916] nix[1015]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 62.949437] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.06 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 66.284325] nix-daemon[854]: reaped child process 1053, status = succeeded worker1 # [ 66.407898] nix-daemon[854]: accepted connection from pid 1085, user nix-grpc-daemon (trusted) worker1 # [ 66.425498] nix-daemon[854]: reaped child process 1088, status = succeeded lb # [ 66.356931] niks3-server[975]: 2026/09/23 05:33:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.444515] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.446639] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.047Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 66.408240] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls worker1 # [ 66.494543] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.098Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.418563] niks3-server[975]: 2026/09/23 05:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 66.428550] niks3-server[975]: 2026/09/23 05:33:06 INFO Signed narinfos id=16 count=1 lb # [ 66.434648] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 66.442977] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 66.454483] niks3-server[975]: 2026/09/23 05:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete worker1 # [ 66.530831] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.134Z level=INFO msg="Upload complete. (278ms)" lb # [ 66.461249] niks3-server[975]: 2026/09/23 05:33:06 INFO Completed upload id=16 lb # [ 66.465204] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 66.545117] nix-grpc-daemon[769]: ts=2026-09-23T05:33:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 66.564332] nix-grpc-daemon[769]: ts=2026-09-23T05:33:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 66.572231] nix-daemon[854]: reaped child process 1051, status = succeeded client # [ 66.527921] systemd[1]: dedup2.service: Deactivated successfully. client # [ 66.531226] systemd[1]: dedup2.service: Consumed 204ms CPU time over 3.916s wall clock time, 33.2M memory peak, 6.1K incoming IP traffic, 5.8K outgoing IP traffic. client # [ 66.539427] systemd[1]: dedup1.service: Deactivated successfully. client # [ 66.543000] systemd[1]: dedup1.service: Consumed 178ms CPU time over 5.021s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.12 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 67.888922] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 67.905797] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 67.911552] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 67.885772] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 67.901715] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 67.916791] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 67.931644] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 68.092157] nix-daemon[839]: accepted connection from pid 990, user nix-grpc-daemon (trusted) worker2 # [ 68.111913] nix-daemon[839]: reaped child process 993, status = succeeded lb # [ 68.110347] niks3-server[975]: 2026/09/23 05:33:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 68.126649] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 68.128431] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.742Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 68.138936] niks3-server[975]: 2026/09/23 05:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 68.154099] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.770Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.153765] niks3-server[975]: 2026/09/23 05:33:07 INFO Signed narinfos id=17 count=1 lb # [ 68.163535] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 68.173981] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 68.194332] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.810Z level=INFO msg="Upload complete. (245ms)" lb # [ 68.188940] niks3-server[975]: 2026/09/23 05:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 68.196691] niks3-server[975]: 2026/09/23 05:33:07 INFO Completed upload id=17 worker2 # [ 68.204431] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 68.199351] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 68.220607] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 68.229885] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 68.233115] nix-daemon[839]: reaped child process 988, status = succeeded worker2 # [ 68.245916] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 69.616370] envoy[904]: conn peer=[2001:db8:1::1]:44052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9554 tx=4401 ms=6099 lb # [ 69.616703] envoy[904]: conn peer=[2001:db8:1::1]:59522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=129 lb # [ 69.616810] envoy[904]: conn peer=[2001:db8:1::1]:59534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=111 lb # [ 69.616925] envoy[904]: conn peer=[2001:db8:1::1]:59538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=92 lb # [ 69.617022] envoy[904]: conn peer=[2001:db8:1::1]:59506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2891 ms=218 lb # [ 69.617161] envoy[904]: conn peer=[2001:db8:1::1]:59558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=3642 lb # [ 69.617261] envoy[904]: conn peer=[2001:db8:1::1]:59544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=4788 worker2 # [ 74.065269] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 74.103562] nix-daemon[839]: reaped child process 995, status = succeeded worker2 # [ 74.106417] nix-daemon[839]: reaped child process 997, status = succeeded worker1 # [ 74.369051] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 74.383429] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 74.390703] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 74.399936] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 74.412966] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 74.427582] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 74.443475] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 74.596935] nix-daemon[854]: accepted connection from pid 1105, user nix-grpc-daemon (trusted) worker1 # [ 74.618604] nix-daemon[854]: reaped child process 1108, status = succeeded lb # [ 74.541402] niks3-server[975]: 2026/09/23 05:33:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 74.628528] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.231Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 74.630381] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.231Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 74.572634] niks3-server[975]: 2026/09/23 05:33:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 74.659978] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.263Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.584361] niks3-server[975]: 2026/09/23 05:33:14 INFO Signed narinfos id=18 count=1 lb # [ 74.597848] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 74.607200] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 74.618335] niks3-server[975]: 2026/09/23 05:33:14 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete worker1 # [ 74.692788] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.296Z level=INFO msg="Upload complete. (233ms)" lb # [ 74.625765] niks3-server[975]: 2026/09/23 05:33:14 INFO Completed upload id=18 lb # [ 74.628256] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 74.703182] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 74.720696] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 74.724278] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 74.729697] nix-daemon[854]: reaped child process 1103, status = succeeded worker1 # [ 74.741025] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 79.616962] envoy[904]: conn peer=[2001:db8:1::1]:59574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=953 ms=6265 worker1 # [ 81.157383] nix-grpc-daemon[769]: ts=2026-09-23T05:33:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 81.197220] nix-daemon[854]: reaped child process 1110, status = succeeded worker1 # [ 81.199988] nix-daemon[854]: reaped child process 1112, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.67 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 81.321211] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 9425ec8079084282bf7b2922f1ba8cb5 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 81.530884] nix[1089]: don't know how to build these paths: client # [ 81.531847] nix[1089]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 81.537452] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 81.620136] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 81.626515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 81.559089] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 81.575299] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 client # [ 81.579969] nix[1089]: copying 1 paths to 'lb:50051'... client # [ 81.582496] nix[1089]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 81.591168] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 81.606665] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 81.769329] nix-daemon[839]: accepted connection from pid 1020, user nix-grpc-daemon (trusted) worker2 # [ 81.787734] nix-daemon[839]: reaped child process 1023, status = succeeded lb # [ 81.789781] niks3-server[975]: 2026/09/23 05:33:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 81.806624] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 81.808409] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.422Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 81.818742] niks3-server[975]: 2026/09/23 05:33:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 81.834735] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.450Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.831453] niks3-server[975]: 2026/09/23 05:33:21 INFO Signed narinfos id=19 count=1 lb # [ 81.841225] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 81.850606] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 81.861861] niks3-server[975]: 2026/09/23 05:33:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker2 # [ 81.874301] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.490Z level=INFO msg="Upload complete. (250ms)" lb # [ 81.870408] niks3-server[975]: 2026/09/23 05:33:21 INFO Completed upload id=19 lb # [ 81.874295] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 81.884424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 81.903919] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 81.911266] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 81.917405] nix-daemon[839]: reaped child process 1018, status = succeeded worker2 # [ 81.930438] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # [ 81.921957] nix[1089]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 82.427177] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 551b97cc432a4961834d3078e21d7942 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) worker1 # [ 82.693668] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 client # [ 82.623864] nix[1105]: don't know how to build these paths: client # [ 82.625586] nix[1105]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 82.710593] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 82.645278] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 client # [ 82.644996] nix[1105]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 85.762221] systemd[1]: share1.service: Sent signal SIGINT to main process 1089 (nix) on client request. client # [ 85.772956] nix[1089]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 85.788595] nix[1089]: error: interrupted by the user client # [ 85.843522] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 85.845844] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 85.849177] systemd[1]: share1.service: Consumed 174ms CPU time over 4.527s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 87.397996] nix-daemon[839]: reaped child process 1027, status = succeeded worker2 # [ 87.535238] nix-daemon[839]: accepted connection from pid 1061, user nix-grpc-daemon (trusted) worker2 # [ 87.552777] nix-daemon[839]: reaped child process 1064, status = succeeded lb # [ 87.554412] niks3-server[975]: 2026/09/23 05:33:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 87.572454] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 87.574188] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.187Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 87.593523] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 87.608910] niks3-server[975]: 2026/09/23 05:33:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker2 # [ 87.621771] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.237Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.618144] niks3-server[975]: 2026/09/23 05:33:27 INFO Signed narinfos id=20 count=1 lb # [ 87.623181] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 87.633215] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls worker2 # [ 87.645242] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.261Z level=INFO msg="Upload complete. (294ms)" lb # [ 87.639440] niks3-server[975]: 2026/09/23 05:33:27 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 87.643540] niks3-server[975]: 2026/09/23 05:33:27 INFO Completed upload id=20 lb # [ 87.646474] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 87.657219] nix-grpc-daemon[769]: ts=2026-09-23T05:33:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 87.676424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 87.684251] nix-daemon[839]: reaped child process 1025, status = succeeded client # [ 87.686914] systemd[1]: share2.service: Deactivated successfully. client # [ 87.691907] systemd[1]: share2.service: Consumed 170ms CPU time over 5.261s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.06 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 88.451439] systemd[1]: Started Nix Daemon. client # [ 88.565587] nix-daemon[1157]: accepted connection from pid 1154, user root (trusted) worker2 # [ 88.717173] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 88.791982] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 88.806761] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 88.814018] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 88.823139] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 88.833954] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 88.847977] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 88.862791] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 89.013971] nix-daemon[854]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker1 # [ 89.031794] nix-daemon[854]: reaped child process 1160, status = succeeded lb # [ 88.962315] niks3-server[975]: 2026/09/23 05:33:28 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 89.048572] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.652Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 89.050801] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.654Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 88.993245] niks3-server[975]: 2026/09/23 05:33:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 89.079154] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.682Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.007826] niks3-server[975]: 2026/09/23 05:33:28 INFO Signed narinfos id=21 count=1 lb # [ 89.016510] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 89.025479] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 89.034780] niks3-server[975]: 2026/09/23 05:33:28 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 89.041895] niks3-server[975]: 2026/09/23 05:33:28 INFO Completed upload id=21 worker1 # [ 89.118750] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.722Z level=INFO msg="Upload complete. (240ms)" lb # [ 89.045987] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 89.128312] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 89.147008] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 89.156389] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 89.159977] nix-daemon[854]: reaped child process 1155, status = succeeded worker1 # [ 89.173375] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 89.621157] envoy[904]: conn peer=[2001:db8:1::1]:47364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=934 ms=6814 lb # [ 89.626106] envoy[904]: conn peer=[2001:db8:1::1]:43496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=797 ms=4263 lb # [ 89.630864] envoy[904]: conn peer=[2001:db8:1::1]:43510 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=5050 worker1 # [ 90.664784] nix-daemon[854]: reaped child process 1164, status = succeeded worker1 # [ 90.747466] nix-daemon[854]: accepted connection from pid 1180, user nix-grpc-daemon (trusted) worker1 # [ 90.765528] nix-daemon[854]: reaped child process 1183, status = succeeded lb # [ 90.694662] niks3-server[975]: 2026/09/23 05:33:30 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.783715] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 90.785477] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.385Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 90.739081] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 90.828687] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.432Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.754224] niks3-server[975]: 2026/09/23 05:33:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 90.763180] niks3-server[975]: 2026/09/23 05:33:30 INFO Signed narinfos id=22 count=1 lb # [ 90.770501] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls worker1 # [ 90.853721] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.457Z level=INFO msg="Upload complete. (269ms)" lb # [ 90.780268] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 90.788242] niks3-server[975]: 2026/09/23 05:33:30 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 90.793192] niks3-server[975]: 2026/09/23 05:33:30 INFO Completed upload id=22 worker1 # [ 90.866405] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 lb # [ 90.797654] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 90.885257] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 90.890693] nix-daemon[854]: reaped child process 1162, status = succeeded worker1 # [ 90.912158] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 90.919179] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 91.125091] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 91.198638] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 91.210922] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.814Z level=INFO msg="All 1 paths already cached" worker1 # [ 91.229178] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 91.235548] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 91.277327] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 91.292565] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 91.307562] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.911Z level=INFO msg="All 1 paths already cached" worker1 # [ 91.317307] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 91.334353] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 91.486473] nix-daemon[854]: accepted connection from pid 1189, user nix-grpc-daemon (trusted) worker1 # [ 91.505349] nix-daemon[854]: reaped child process 1192, status = succeeded lb # [ 91.433677] niks3-server[975]: 2026/09/23 05:33:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 91.520710] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.124Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 91.522272] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.124Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 91.464390] niks3-server[975]: 2026/09/23 05:33:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker1 # [ 91.548085] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.151Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.479717] niks3-server[975]: 2026/09/23 05:33:31 INFO Signed narinfos id=23 count=1 lb # [ 91.487361] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 91.494980] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 91.583662] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.187Z level=INFO msg="Upload complete. (235ms)" lb # [ 91.507784] niks3-server[975]: 2026/09/23 05:33:31 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 91.514918] niks3-server[975]: 2026/09/23 05:33:31 INFO Completed upload id=23 worker1 # [ 91.592285] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 91.518326] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 91.604560] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 91.616894] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 91.622286] nix-daemon[854]: reaped child process 1187, status = succeeded worker1 # [ 91.634575] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 92.327663] nix-daemon[854]: reaped child process 1196, status = succeeded worker1 # [ 92.416402] nix-daemon[854]: accepted connection from pid 1212, user nix-grpc-daemon (trusted) worker1 # [ 92.435779] nix-daemon[854]: reaped child process 1215, status = succeeded lb # [ 92.363594] niks3-server[975]: 2026/09/23 05:33:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 92.452566] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.055Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 92.454685] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.055Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" worker1 # [ 92.510727] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.114Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.421566] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 92.439821] niks3-server[975]: 2026/09/23 05:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 92.449777] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 92.458150] niks3-server[975]: 2026/09/23 05:33:32 INFO Signed narinfos id=24 count=1 worker1 # [ 92.543325] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.147Z level=INFO msg="Upload complete. (296ms)" lb # [ 92.468194] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 92.473403] niks3-server[975]: 2026/09/23 05:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 92.479369] niks3-server[975]: 2026/09/23 05:33:32 INFO Completed upload id=24 worker1 # [ 92.557340] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 lb # [ 92.485717] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker2 # [ 92.506439] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34358 worker1 # [ 92.580319] nix-daemon[854]: reaped child process 1194, status = succeeded worker2 # [ 92.528338] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 92.545765] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker1 # [ 92.657111] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 92.665680] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 92.905057] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 92.912894] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 92.920863] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.522Z level=INFO msg="All 2 paths already cached" worker1 # [ 92.933911] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 92.941241] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 92.950668] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 92.962697] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 92.977817] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.581Z level=INFO msg="All 2 paths already cached" worker1 # [ 92.988189] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 93.002914] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 93.157832] nix-daemon[854]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 93.176289] nix-daemon[854]: reaped child process 1224, status = succeeded lb # [ 93.105937] niks3-server[975]: 2026/09/23 05:33:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.192759] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.796Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 93.194350] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.796Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 93.140562] niks3-server[975]: 2026/09/23 05:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 93.227559] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.831Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.156191] niks3-server[975]: 2026/09/23 05:33:32 INFO Signed narinfos id=25 count=1 lb # [ 93.159102] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 93.172154] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 93.184944] niks3-server[975]: 2026/09/23 05:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 93.191796] niks3-server[975]: 2026/09/23 05:33:32 INFO Completed upload id=25 worker1 # [ 93.267694] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.871Z level=INFO msg="Upload complete. (249ms)" lb # [ 93.194694] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 93.279521] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 93.296551] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 93.306886] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 93.311163] nix-daemon[854]: reaped child process 1219, status = succeeded worker1 # [ 93.327001] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 94.230503] nix-daemon[854]: reaped child process 1228, status = succeeded worker1 # [ 94.299588] nix-daemon[854]: accepted connection from pid 1244, user nix-grpc-daemon (trusted) worker1 # [ 94.320922] nix-daemon[854]: reaped child process 1247, status = succeeded lb # [ 94.250521] niks3-server[975]: 2026/09/23 05:33:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 94.340402] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.941Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 94.342012] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.941Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 94.297909] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker1 # [ 94.385816] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.989Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.311614] niks3-server[975]: 2026/09/23 05:33:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 94.321207] niks3-server[975]: 2026/09/23 05:33:33 INFO Signed narinfos id=26 count=1 lb # [ 94.323601] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 94.331608] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst worker1 # [ 94.419826] nix-grpc-daemon[867]: time=2026-09-23T05:33:34.023Z level=INFO msg="Upload complete. (281ms)" lb # [ 94.344319] niks3-server[975]: 2026/09/23 05:33:33 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 94.349087] niks3-server[975]: 2026/09/23 05:33:33 INFO Completed upload id=26 lb # [ 94.352785] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 94.433377] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 94.457679] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 94.462869] nix-daemon[854]: reaped child process 1226, status = succeeded worker1 # [ 94.479535] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 94.486935] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=232 client # [ 94.545215] nix-daemon[1157]: reaped child process 1159, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.25 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.37 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.31 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 95.524417] nix-daemon[1157]: accepted connection from pid 1238, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 95.755490] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34776 worker2 # [ 95.690742] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker2 # [ 95.706334] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 95.721395] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 95.894042] nix-daemon[839]: accepted connection from pid 1073, user nix-grpc-daemon (trusted) worker2 # [ 95.914017] nix-daemon[839]: reaped child process 1076, status = succeeded lb # [ 95.915057] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.932778] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.548Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 95.934401] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.548Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 95.949114] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls worker2 # [ 95.964634] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.580Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.961451] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 95.970324] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=27 count=1 worker2 # [ 95.987221] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.603Z level=INFO msg="Upload complete. (249ms)" lb # [ 95.980553] niks3-server[975]: 2026/09/23 05:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 95.988502] niks3-server[975]: 2026/09/23 05:33:35 INFO Completed upload id=27 lb # [ 95.991489] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo lb # [ 95.998647] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker2 # [ 95.996360] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=120 worker2 # [ 96.011359] nix-daemon[839]: reaped child process 1071, status = succeeded worker1 # [ 96.083309] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 96.089841] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 96.058664] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 96.074275] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 96.092579] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 96.106064] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 96.255988] nix-daemon[839]: accepted connection from pid 1080, user nix-grpc-daemon (trusted) worker2 # [ 96.273344] nix-daemon[839]: reaped child process 1083, status = succeeded lb # [ 96.272991] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 96.282470] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 96.299434] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 96.302387] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 96.305066] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 96.329112] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 96.342366] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 96.354377] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls worker2 # [ 96.360852] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.976Z level=INFO msg="Uploading 3 narinfos" lb # [ 96.361553] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 96.368069] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=28 count=1 lb # [ 96.372512] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 96.379736] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=29 count=2 lb # [ 96.385618] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 96.393028] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 96.399824] niks3-server[975]: 2026/09/23 05:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 96.407769] niks3-server[975]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 96.412366] niks3-server[975]: 2026/09/23 05:33:36 INFO Completed upload id=28 lb # [ 96.414297] niks3-server[975]: 2026/09/23 05:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 96.417301] niks3-server[975]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 96.419520] niks3-server[975]: 2026/09/23 05:33:36 INFO Completed upload id=29 worker2 # [ 96.423955] nix-grpc-daemon[866]: time=2026-09-23T05:33:36.040Z level=INFO msg="Upload complete. (300ms)" worker2 # [ 96.436254] nix-grpc-daemon[769]: ts=2026-09-23T05:33:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 96.452624] nix-grpc-daemon[769]: ts=2026-09-23T05:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 96.463874] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 96.469250] nix-daemon[839]: reaped child process 1078, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker2 # [ 96.484262] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 97.749739] nix-daemon[839]: reaped child process 1087, status = succeeded worker2 # [ 97.889024] nix-daemon[839]: accepted connection from pid 1103, user nix-grpc-daemon (trusted) worker2 # [ 97.907249] nix-daemon[839]: reaped child process 1106, status = succeeded lb # [ 97.907029] niks3-server[975]: 2026/09/23 05:33:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 97.925637] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.540Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 97.939454] niks3-server[975]: 2026/09/23 05:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 97.955808] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.572Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.954493] niks3-server[975]: 2026/09/23 05:33:37 INFO Signed narinfos id=30 count=1 lb # [ 97.961542] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls worker2 # [ 97.980823] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.597Z level=INFO msg="Upload complete. (280ms)" lb # [ 97.969095] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 97.982901] niks3-server[975]: 2026/09/23 05:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 97.986677] niks3-server[975]: 2026/09/23 05:33:37 INFO Completed upload id=30 worker2 # [ 97.993027] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 lb # [ 97.989286] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 98.008147] nix-daemon[839]: reaped child process 1085, status = succeeded client # copying 1 paths... worker2 # [ 98.036537] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34358 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 98.069348] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 98.076164] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=120 client # [ 98.198742] nix-daemon[1157]: reaped child process 1241, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.86 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 98.374446] nix-daemon[1157]: accepted connection from pid 1259, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 98.597222] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34776 worker2 # [ 98.529644] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 98.539293] nix-grpc-daemon[866]: time=2026-09-23T05:33:38.155Z level=INFO msg="All 1 paths already cached" worker1 # [ 98.626506] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 98.636310] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 98.645334] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 98.654036] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34776 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 98.666936] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 98.681922] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 98.702015] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5) worker1 # [ 98.777340] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5) worker1 # [ 98.811028] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 3/5) worker1 # [ 98.852938] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 4/5) worker1 # [ 99.175501] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 99.182271] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 99.190841] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 99.364096] nix-daemon[854]: accepted connection from pid 1265, user nix-grpc-daemon (trusted) worker1 # [ 99.382518] nix-daemon[854]: reaped child process 1268, status = succeeded lb # [ 99.314381] niks3-server[975]: 2026/09/23 05:33:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 99.401419] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 99.402994] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.004Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 99.344579] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker1 # [ 99.433375] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.037Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.359968] niks3-server[975]: 2026/09/23 05:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 99.368968] niks3-server[975]: 2026/09/23 05:33:38 INFO Signed narinfos id=31 count=1 lb # [ 99.378035] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker1 # [ 99.460211] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.064Z level=INFO msg="Upload complete. (250ms)" lb # [ 99.383927] niks3-server[975]: 2026/09/23 05:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 99.391388] niks3-server[975]: 2026/09/23 05:33:38 INFO Completed upload id=31 worker1 # [ 99.468143] nix-grpc-daemon[769]: ts=2026-09-23T05:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 99.396311] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 99.479951] nix-grpc-daemon[769]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 99.492518] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 99.497614] nix-daemon[854]: reaped child process 1258, status = succeeded worker1 # [ 99.508861] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 99.619512] envoy[904]: conn peer=[2001:db8:1::1]:43522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3440 tx=1966 ms=2191 lb # [ 99.624605] envoy[904]: conn peer=[2001:db8:1::1]:37418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=68 lb # [ 99.629228] envoy[904]: conn peer=[2001:db8:1::1]:37426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=2155 ms=1487 lb # [ 99.633952] envoy[904]: conn peer=[2001:db8:1::1]:37438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=29 lb # [ 99.638614] envoy[904]: conn peer=[2001:db8:1::1]:37440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2389 ms=1641 lb # [ 99.643528] envoy[904]: conn peer=[2001:db8:1::1]:37454 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=76 lb # [ 99.648316] envoy[904]: conn peer=[2001:db8:1::1]:37460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4271 tx=2072 ms=2450 lb # [ 99.653009] envoy[904]: conn peer=[2001:db8:1::1]:37466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=70 client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 100.619099] nix-daemon[854]: reaped child process 1272, status = succeeded worker1 # [ 100.690200] nix-daemon[854]: accepted connection from pid 1288, user nix-grpc-daemon (trusted) worker1 # [ 100.706523] nix-daemon[854]: reaped child process 1291, status = succeeded lb # [ 100.637044] niks3-server[975]: 2026/09/23 05:33:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 100.724662] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.326Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 100.663453] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 100.678202] niks3-server[975]: 2026/09/23 05:33:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker1 # [ 100.761550] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.365Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.690952] niks3-server[975]: 2026/09/23 05:33:40 INFO Signed narinfos id=32 count=1 lb # [ 100.698979] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 100.788456] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.392Z level=INFO msg="Upload complete. (265ms)" lb # [ 100.711616] niks3-server[975]: 2026/09/23 05:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 100.717127] niks3-server[975]: 2026/09/23 05:33:40 INFO Completed upload id=32 lb # [ 100.723616] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 100.799640] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker1 # [ 100.820776] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 100.825927] nix-daemon[854]: reaped child process 1270, status = succeeded client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 100.852260] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 100.859021] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=120 client # [ 100.909529] nix-daemon[1157]: reaped child process 1262, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.71 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 101.080222] nix-daemon[1157]: accepted connection from pid 1280, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 101.234349] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.240691] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.251037] nix-grpc-daemon[866]: time=2026-09-23T05:33:40.866Z level=INFO msg="All 1 paths already cached" worker1 # [ 101.339041] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 101.345320] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 101.314942] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.330406] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.344671] nix-grpc-daemon[866]: time=2026-09-23T05:33:40.961Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 101.355902] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.371164] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 101.532846] nix-daemon[839]: accepted connection from pid 1110, user nix-grpc-daemon (trusted) worker2 # [ 101.551130] nix-daemon[839]: reaped child process 1113, status = succeeded lb # [ 101.548970] niks3-server[975]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.565309] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 101.566749] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.181Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 101.583089] niks3-server[975]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker2 # [ 101.596586] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.212Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.598214] niks3-server[975]: 2026/09/23 05:33:41 INFO Signed narinfos id=33 count=1 lb # [ 101.605091] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 101.623484] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.239Z level=INFO msg="Upload complete. (235ms)" lb # [ 101.616968] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 101.624753] niks3-server[975]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker2 # [ 101.632327] nix-grpc-daemon[769]: ts=2026-09-23T05:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 101.630866] niks3-server[975]: 2026/09/23 05:33:41 INFO Completed upload id=33 lb # [ 101.634642] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 101.646515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 worker2 # [ 101.655233] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 101.658236] nix-daemon[839]: reaped child process 1108, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 101.670660] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 102.760670] nix-daemon[839]: reaped child process 1117, status = succeeded worker2 # [ 102.848961] nix-daemon[839]: accepted connection from pid 1133, user nix-grpc-daemon (trusted) worker2 # [ 102.865538] nix-daemon[839]: reaped child process 1136, status = succeeded lb # [ 102.864923] niks3-server[975]: 2026/09/23 05:33:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 102.882678] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.497Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 102.898197] niks3-server[975]: 2026/09/23 05:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker2 # [ 102.916091] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.532Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.913495] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 102.924236] niks3-server[975]: 2026/09/23 05:33:42 INFO Signed narinfos id=34 count=1 lb # [ 102.931708] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 102.945503] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.561Z level=INFO msg="Upload complete. (264ms)" lb # [ 102.939842] niks3-server[975]: 2026/09/23 05:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 102.944198] niks3-server[975]: 2026/09/23 05:33:42 INFO Completed upload id=34 lb # [ 102.947611] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo client # copying 1 paths... worker2 # [ 102.957244] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker1 # [ 103.037138] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker2 # [ 102.977458] nix-daemon[839]: reaped child process 1115, status = succeeded worker1 # [ 103.050064] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 103.093573] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 103.100594] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=120 client # [ 103.149501] nix-daemon[1157]: reaped child process 1283, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.24 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 8.47 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.32 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds) worker2 # [ 104.143100] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 105.057626] nix-grpc-daemon[769]: ts=2026-09-23T05:33:44Z level=info event=signal_exit worker2 # [ 106.601274] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=scheduler_restarting worker1 # [ 106.682657] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53214 worker1 # [ 106.691597] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 106.639148] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 106.665476] nix-daemon[839]: reaped child process 841, status = succeeded worker2 # [ 107.207277] nix-grpc-daemon[769]: E0000 00:00:1790141626.823401 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 107.668552] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 107.676220] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 107.678846] systemd[1]: nix-grpc-daemon.service: Consumed 2.895s CPU time over 1min 27.887s wall clock time, 60.2M memory peak, 72.3K incoming IP traffic, 124.5K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 107.705525] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 107.707563] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 107.709584] systemd[1]: nix-grpc-daemon.socket: Consumed 520us CPU time over 1min 37.724s wall clock time, 256K memory peak, 89.9K incoming IP traffic, 79.3K outgoing IP traffic. lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 108.061603] nix-grpc-daemon[769]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34776 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.24 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 108.118540] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 108.068760] nix-grpc-daemon[769]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34776 worker1 # [ 108.256838] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 108.662657] systemd[1]: Starting scheduler-token.service... worker2 # [ 108.753518] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 108.760463] systemd[1]: Finished scheduler-token.service. worker2 # [ 108.762560] systemd[1]: scheduler-token.service: Consumed 56ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 108.785333] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 108.908666] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 108.910536] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 108.914336] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 108.920086] nix-grpc-daemon[1160]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 108.921496] nix-grpc-daemon[1160]: E0000 00:00:1790141628.535195 1160 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 108.933394] nix-grpc-daemon[1160]: E0000 00:00:1790141628.541412 1160 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 108.968130] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 108.970238] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 108.971208] nix-grpc-daemon[1160]: E0000 00:00:1790141628.550902 1201 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 108.973638] nix-grpc-daemon[1160]: E0000 00:00:1790141628.566933 1201 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 108.978546] nix-grpc-daemon[1160]: E0000 00:00:1790141628.567085 1201 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 108.983027] nix-grpc-daemon[1160]: E0000 00:00:1790141628.583752 1201 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 108.985614] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 109.055915] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53214 worker1 # [ 109.056502] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb # [ 109.601132] niks3-server[975]: 2026/09/23 05:33:49 INFO lead: released remote=192.168.1.3:53876 worker1 # [ 109.072531] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=signal_exit lb # [ 109.623302] envoy[904]: conn peer=[2001:db8:1::1]:37472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2124 ms=2321 lb # [ 109.628436] envoy[904]: conn peer=[2001:db8:1::1]:37474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=69 lb # [ 109.633108] envoy[904]: conn peer=[2001:db8:1::1]:44196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2099 ms=1853 lb # [ 109.638030] envoy[904]: conn peer=[2001:db8:1::1]:44198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=68 lb # [ 109.642746] envoy[904]: conn peer=[2001:db8:1::4]:46358 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2543 tx=851 ms=70949 lb # [ 109.647693] envoy[904]: conn peer=[2001:db8:1::1]:44202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=25 worker1 # [ 110.673183] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting worker2 # [ 110.605105] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 110.594482] envoy[904]: [2026-09-23 05:33:50.197][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 110.682868] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53214 worker1 # [ 110.694424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 110.698881] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 110.705253] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53214 worker1 # [ 110.710370] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 110.735064] nix-daemon[854]: reaped child process 856, status = succeeded worker2 # [ 110.805559] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.810942] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 110.916585] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 111.011856] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.017214] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 111.122062] nix-grpc-daemon[769]: E0000 00:00:1790141630.725870 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker1 # [ 111.243449] nix-grpc-daemon[769]: E0000 00:00:1790141630.847365 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 111.218579] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.223742] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.424941] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.429988] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 111.686942] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 111.694720] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 111.632933] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 111.703156] systemd[1]: nix-grpc-daemon.service: Consumed 3.789s CPU time over 1min 31.982s wall clock time, 73.1M memory peak, 98.8K incoming IP traffic, 201.3K outgoing IP traffic. worker2 # [ 111.637040] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 111.719547] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 111.725313] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 111.727056] systemd[1]: nix-grpc-daemon.socket: Consumed 518us CPU time over 1min 41.759s wall clock time, 256K memory peak, 359K incoming IP traffic, 110.6K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 111.833817] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.839007] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.956076] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 111.971197] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 111.995265] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5) worker2 # [ 112.022583] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5) worker2 # [ 112.040159] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.045154] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.163594] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5) worker2 # [ 112.246284] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.251401] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.324505] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5) worker2 # [ 112.452659] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.457692] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.660501] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.664607] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.861387] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.867093] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.922990] nix-grpc-daemon[1160]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 112.930957] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 112.953738] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 112.972101] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 113.001963] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=niks3_push_started pid=1221 worker2 # [ 113.068699] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.075791] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 113.166943] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.782Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 113.273061] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.277248] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 113.282299] nix-daemon[839]: accepted connection from pid 1233, user nix-grpc-daemon (trusted) worker2 # [ 113.300708] nix-daemon[839]: reaped child process 1236, status = succeeded lb # [ 113.301810] niks3-server[975]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 113.312139] niks3-server[975]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 113.330047] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 113.331519] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 113.335034] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 113.381500] niks3-server[975]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 113.396359] niks3-server[975]: 2026/09/23 05:33:52 INFO Signed narinfos id=35 count=1 worker2 # [ 113.404103] nix-grpc-daemon[1221]: time=2026-09-23T05:33:53.019Z level=INFO msg="Uploading 3 narinfos" lb # [ 113.403815] niks3-server[975]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 113.413925] niks3-server[975]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 113.422912] niks3-server[975]: 2026/09/23 05:33:52 INFO Signed narinfos id=36 count=2 lb # [ 113.426942] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 113.436117] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 113.442254] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 113.453685] niks3-server[975]: 2026/09/23 05:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 113.464261] niks3-server[975]: 2026/09/23 05:33:53 INFO Completed upload id=35 lb # [ 113.468623] niks3-server[975]: 2026/09/23 05:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete worker2 # [ 113.476596] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 lb # [ 113.472963] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 113.475269] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 113.482114] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 113.479881] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 113.482740] niks3-server[975]: 2026/09/23 05:33:53 INFO Completed upload id=36 worker2 # [ 113.488580] nix-grpc-daemon[1221]: time=2026-09-23T05:33:53.103Z level=INFO msg="Upload complete. (317ms)" worker2 # [ 113.500461] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 113.516906] nix-daemon[839]: reaped child process 1219, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.80 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 113.709135] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 113.717806] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 113.684577] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.689392] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 113.856707] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 113.858607] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 113.862559] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 113.868060] nix-grpc-daemon[1317]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 113.869522] nix-grpc-daemon[1317]: E0000 00:00:1790141633.471487 1317 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 113.803559] niks3-server[975]: 2026/09/23 05:33:53 INFO lead: acquired remote=192.168.1.3:37024 lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 113.914444] nix-grpc-daemon[1317]: E0000 00:00:1790141633.477632 1317 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 113.917864] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 113.920139] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_take_over worker1 # [ 113.921420] nix-grpc-daemon[1317]: E0000 00:00:1790141633.485599 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 113.923763] nix-grpc-daemon[1317]: E0000 00:00:1790141633.502714 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 113.929249] nix-grpc-daemon[1317]: E0000 00:00:1790141633.502881 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 113.931615] nix-grpc-daemon[1317]: E0000 00:00:1790141633.518419 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 113.936318] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 113.937166] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 113.938457] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 113.890584] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.895887] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 114.137634] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.142841] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.097000] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.102126] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.343610] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.349075] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.303256] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.308533] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.549811] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.554960] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.509697] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.514701] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.755595] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.764191] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:57352 worker1 # [ 114.769427] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 114.717186] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.795240] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 114.799758] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:57352 worker1 # [ 114.805314] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.14 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.80 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb # [ 115.059306] systemd[1]: Stopping niks3 server... worker2 # [ 115.067339] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_lock_lost status=200 worker1 # [ 115.137603] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_lock_lost status=200 lb # [ 115.066861] niks3-server[975]: 2026/09/23 05:33:54 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 115.075374] niks3-server[975]: 2026/09/23 05:33:54 INFO lead: released remote=192.168.1.3:37024 lb # [ 115.081728] systemd[1]: niks3.service: Deactivated successfully. lb # [ 115.083881] systemd[1]: Stopped niks3 server. lb # [ 115.085757] systemd[1]: niks3.service: Consumed 643ms CPU time over 1min 30.734s wall clock time, 27.4M memory peak, 90.1K incoming IP traffic, 132.9K outgoing IP traffic. lb # [ 115.094024] systemd[1]: Starting niks3 server... lb # [ 115.155137] niks3-server[1137]: 2026/09/23 05:33:54 goose: no migrations to run. current version: 20260920000000 lb # [ 115.160918] niks3-server[1137]: 2026/09/23 05:33:54 OK 1_commit_pending_closure.sql (7.24ms) lb # [ 115.164375] niks3-server[1137]: 2026/09/23 05:33:54 OK 2_object_stats_trigger.sql (3.09ms) lb # [ 115.166469] niks3-server[1137]: 2026/09/23 05:33:54 goose: up to current file version: 2 lb # [ 115.167971] niks3-server[1137]: 2026/09/23 05:33:54 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 115.177702] niks3-server[1137]: 2026/09/23 05:33:54 INFO Read proxy enabled — serving cache objects from S3 lb # [ 115.179033] niks3-server[1137]: 2026/09/23 05:33:54 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 115.183560] niks3-server[1137]: 2026/09/23 05:33:54 INFO systemd watchdog enabled interval=15s lb # [ 115.184906] niks3-server[1137]: 2026/09/23 05:33:54 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 115.187095] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 115.736133] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 115.752458] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 115.774742] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 115.791100] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5) worker1 # [ 115.826377] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5) worker1 # [ 115.925257] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 3/5) worker1 # [ 116.024069] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 576 ms (attempt 4/5) lb # [ 116.066314] niks3-server[1137]: 2026/09/23 05:33:55 INFO lead: acquired remote=192.168.1.3:37036 worker1 # [ 116.605774] nix-grpc-daemon[1317]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 116.612709] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 116.657552] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker1 # [ 116.664851] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 116.705511] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 116.723749] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57366 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker1 # [ 116.738151] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 116.753389] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 116.781933] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=info event=niks3_push_started pid=1390 worker1 # [ 116.931184] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.534Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 117.037219] nix-daemon[854]: accepted connection from pid 1403, user nix-grpc-daemon (trusted) worker1 # [ 117.054831] nix-daemon[854]: reaped child process 1406, status = succeeded lb # [ 116.986907] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 117.079419] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 117.083351] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.682Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 117.030857] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 117.066364] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 117.082507] niks3-server[1137]: 2026/09/23 05:33:56 INFO Signed narinfos id=37 count=1 worker1 # [ 117.163446] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.765Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.095467] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 117.108069] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 117.114769] niks3-server[1137]: 2026/09/23 05:33:56 INFO Completed upload id=37 worker1 # [ 117.191587] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.795Z level=INFO msg="Upload complete. (256ms)" lb # [ 117.120299] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker1 # [ 117.203575] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=544 worker1 # [ 117.216824] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 117.231561] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 117.235755] nix-daemon[854]: reaped child process 1388, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker1 # [ 117.246341] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker1 # [ 118.101928] nix-daemon[854]: reaped child process 1410, status = succeeded worker1 # [ 118.228618] nix-daemon[854]: accepted connection from pid 1426, user nix-grpc-daemon (trusted) worker1 # [ 118.246452] nix-daemon[854]: reaped child process 1429, status = succeeded lb # [ 118.177766] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 118.267747] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 118.272979] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.871Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 118.226693] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 118.240105] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker1 # [ 118.329588] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.932Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.246921] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 118.257218] niks3-server[1137]: 2026/09/23 05:33:57 INFO Signed narinfos id=38 count=1 lb # [ 118.267257] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete worker1 # [ 118.350860] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.954Z level=INFO msg="Upload complete. (292ms)" lb # [ 118.277734] niks3-server[1137]: 2026/09/23 05:33:57 INFO Completed upload id=38 lb # [ 118.279695] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 118.285456] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker1 # [ 118.374122] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker1 # [ 118.390614] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker1 # [ 118.394830] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 118.399112] nix-daemon[854]: reaped child process 1408, status = succeeded worker1 # [ 118.418230] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 118.433806] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 118.449408] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.053Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker1 # [ 118.459666] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 118.473728] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 118.626666] nix-daemon[854]: accepted connection from pid 1435, user nix-grpc-daemon (trusted) worker1 # [ 118.643537] nix-daemon[854]: reaped child process 1438, status = succeeded lb # [ 118.572452] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 118.658346] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.262Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 118.659817] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.262Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 118.600706] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker1 # [ 118.687161] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.290Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.616541] niks3-server[1137]: 2026/09/23 05:33:58 INFO Signed narinfos id=39 count=1 lb # [ 118.619911] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 118.633071] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 118.647942] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 118.654013] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker1 # [ 118.730921] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.334Z level=INFO msg="Upload complete. (241ms)" lb # [ 118.658101] niks3-server[1137]: 2026/09/23 05:33:58 INFO Completed upload id=39 worker1 # [ 118.741007] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=656 worker1 # [ 118.756514] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 118.768216] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 118.771355] nix-daemon[854]: reaped child process 1433, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker1 # [ 118.784254] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 119.572085] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:57368 lb # [ 119.624098] envoy[904]: conn peer=192.168.1.3:57412 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2491 tx=1124 ms=74980 client # farm-b> LOG-farm-b-after-niks3-restart worker1 # [ 119.829011] nix-daemon[854]: reaped child process 1442, status = succeeded worker1 # [ 119.936749] nix-daemon[854]: accepted connection from pid 1458, user nix-grpc-daemon (trusted) worker1 # [ 119.954176] nix-daemon[854]: reaped child process 1461, status = succeeded lb # [ 119.629170] envoy[904]: conn peer=[2001:db8:1::1]:41590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=282 ms=1559 lb # [ 119.882411] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.979697] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.582Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 119.981698] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.582Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 119.930973] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker1 # [ 120.015503] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.619Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.946681] niks3-server[1137]: 2026/09/23 05:33:59 INFO Signed narinfos id=40 count=1 lb # [ 119.957552] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 119.967074] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 119.976238] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst worker1 # [ 120.052980] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.656Z level=INFO msg="Upload complete. (256ms)" lb # [ 119.979298] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 119.985234] niks3-server[1137]: 2026/09/23 05:33:59 INFO Completed upload id=40 lb # [ 119.988211] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker1 # [ 120.067334] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker1 # [ 120.089817] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker1 # [ 120.094648] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 120.100200] nix-daemon[854]: reaped child process 1440, status = succeeded worker1 # [ 120.129253] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 120.144955] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 120.160031] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.763Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker1 # [ 120.171008] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 120.184440] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 120.338973] nix-daemon[854]: accepted connection from pid 1467, user nix-grpc-daemon (trusted) worker1 # [ 120.358819] nix-daemon[854]: reaped child process 1470, status = succeeded lb # [ 120.285399] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.371403] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.975Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 120.373516] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.975Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 120.317321] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls worker1 # [ 120.403683] nix-grpc-daemon[1390]: time=2026-09-23T05:34:00.007Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.327982] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 120.338382] niks3-server[1137]: 2026/09/23 05:33:59 INFO Signed narinfos id=41 count=1 lb # [ 120.343866] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker1 # [ 120.433714] nix-grpc-daemon[1390]: time=2026-09-23T05:34:00.037Z level=INFO msg="Upload complete. (233ms)" lb # [ 120.357779] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 120.363337] niks3-server[1137]: 2026/09/23 05:33:59 INFO Completed upload id=41 lb # [ 120.366875] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker1 # [ 120.442776] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=784 worker1 # [ 120.459648] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 worker1 # [ 120.466258] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker1 # [ 120.472322] nix-daemon[854]: reaped child process 1465, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker1 # [ 120.485230] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker1 # [ 121.915971] nix-daemon[854]: reaped child process 1474, status = succeeded worker1 # [ 122.010712] nix-daemon[854]: accepted connection from pid 1490, user nix-grpc-daemon (trusted) worker1 # [ 122.028310] nix-daemon[854]: reaped child process 1493, status = succeeded lb # [ 121.958719] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 122.047254] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.649Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 122.048902] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.649Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 121.999631] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign worker1 # [ 122.085771] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.689Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.013952] niks3-server[1137]: 2026/09/23 05:34:01 INFO Signed narinfos id=42 count=1 lb # [ 122.018525] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 122.030421] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 122.040583] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 122.043382] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 122.049994] niks3-server[1137]: 2026/09/23 05:34:01 INFO Completed upload id=42 worker1 # [ 122.126569] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.730Z level=INFO msg="Upload complete. (231ms)" lb # [ 122.052623] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker1 # [ 122.141013] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker1 # [ 122.155305] nix-daemon[854]: reaped child process 1472, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.66 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 7.31 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1 # [ 122.601142] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 122.883133] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:02Z level=info event=signal_exit lb # [ 123.075405] niks3-server[1137]: 2026/09/23 05:34:02 INFO lead: released remote=192.168.1.3:37036 worker1 # [ 124.152932] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting worker2 # [ 124.085308] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 124.075585] envoy[904]: [2026-09-23 05:34:03.677][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 124.158472] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 124.168868] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:57352 worker1 # [ 124.174282] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 124.178596] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:57352 worker1 # [ 124.183574] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 124.207950] nix-daemon[854]: reaped child process 1378, status = succeeded worker2 # [ 124.290965] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.296280] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 124.378981] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 124.497191] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.502346] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 124.724081] nix-grpc-daemon[1317]: E0000 00:00:1790141644.327706 1363 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true worker1 # [ 124.730701] nix-grpc-daemon[1317]: E0000 00:00:1790141644.328016 1363 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 124.703379] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.708597] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.909763] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.914943] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.117615] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.121684] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.318258] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.323319] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 125.422324] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 125.422591] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 125.422745] systemd[1]: nix-grpc-daemon.service: Consumed 1.203s CPU time over 11.701s wall clock time, 38.7M memory peak, 36.8K incoming IP traffic, 59.1K outgoing IP traffic. worker1 # [ 125.444349] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 125.450926] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 125.455709] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 11.753s wall clock time, 256K memory peak, 152K incoming IP traffic, 27.2K outgoing IP traffic. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2 # [ 125.524511] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.529583] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.730551] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.735810] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.937267] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.942401] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 126.068611] niks3-server[1137]: 2026/09/23 05:34:05 INFO lead: acquired remote=192.168.1.4:58172 worker2 # [ 126.079814] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_take_over worker2 # [ 126.145050] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.149156] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.345914] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.350908] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.14 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 126.552364] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.560639] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2 # [ 126.761788] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.767003] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.967950] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.996143] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 127.002063] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:49918 worker2 # [ 127.007532] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker2 # [ 127.879841] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45140 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 127.890336] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 127.918509] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker2 # [ 127.925975] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 127.963039] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 127.980302] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker2 # [ 127.995812] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 128.010823] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 128.169227] nix-daemon[839]: accepted connection from pid 1265, user nix-grpc-daemon (trusted) worker2 # [ 128.189746] nix-daemon[839]: reaped child process 1268, status = succeeded lb # [ 128.187705] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 128.204398] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 128.205854] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.820Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 128.218905] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls worker2 # [ 128.242778] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.858Z level=INFO msg="Uploading 1 narinfos" lb # [ 128.228915] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 128.241485] niks3-server[1137]: 2026/09/23 05:34:07 INFO Signed narinfos id=43 count=1 lb # [ 128.248682] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 128.263760] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 128.271943] niks3-server[1137]: 2026/09/23 05:34:07 INFO Completed upload id=43 worker2 # [ 128.278763] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.895Z level=INFO msg="Upload complete. (251ms)" lb # [ 128.275063] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker2 # [ 128.288307] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=520 worker2 # [ 128.306462] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 128.313879] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 128.319399] nix-daemon[839]: reaped child process 1263, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker2 # [ 128.333281] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker2 # [ 128.766629] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:49924 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 129.433067] nix-daemon[839]: reaped child process 1272, status = succeeded worker2 # [ 129.512748] nix-daemon[839]: accepted connection from pid 1288, user nix-grpc-daemon (trusted) worker2 # [ 129.535855] nix-daemon[839]: reaped child process 1291, status = succeeded lb # [ 129.527938] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 129.548352] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 129.549940] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.162Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 129.573998] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign worker2 # [ 129.593837] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.209Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.590620] niks3-server[1137]: 2026/09/23 05:34:09 INFO Signed narinfos id=44 count=1 lb # [ 129.603355] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 129.613144] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 129.630524] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 129.636880] envoy[904]: conn peer=[2001:db8:1::1]:41600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9753 tx=4458 ms=6418 lb # [ 129.639877] envoy[904]: conn peer=[2001:db8:1::3]:45224 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1605 tx=597 ms=10438 lb # [ 129.644457] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker2 # [ 129.650453] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.266Z level=INFO msg="Upload complete. (303ms)" lb # [ 129.648798] niks3-server[1137]: 2026/09/23 05:34:09 INFO Completed upload id=44 lb # [ 129.653096] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker2 # [ 129.672948] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker2 # [ 129.691940] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker2 # [ 129.695909] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 129.700756] nix-daemon[839]: reaped child process 1270, status = succeeded worker2 # [ 129.715658] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 129.731744] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 129.746885] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.363Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker2 # [ 129.758208] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 129.772241] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 129.923894] nix-daemon[839]: accepted connection from pid 1297, user nix-grpc-daemon (trusted) worker2 # [ 129.946555] nix-daemon[839]: reaped child process 1300, status = succeeded lb # [ 129.937980] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 129.955577] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.571Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 129.957379] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.571Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 129.984226] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker2 # [ 130.002724] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.618Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.999829] niks3-server[1137]: 2026/09/23 05:34:09 INFO Signed narinfos id=45 count=1 lb # [ 130.007798] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 130.014018] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 130.024088] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 130.029703] niks3-server[1137]: 2026/09/23 05:34:09 INFO Completed upload id=45 worker2 # [ 130.035539] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.651Z level=INFO msg="Upload complete. (247ms)" lb # [ 130.033002] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker2 # [ 130.045693] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 130.065809] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 130.078274] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 130.083227] nix-daemon[839]: reaped child process 1295, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker2 # [ 130.095973] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 131.380426] nix-daemon[839]: reaped child process 1304, status = succeeded worker2 # [ 131.479093] nix-daemon[839]: accepted connection from pid 1320, user nix-grpc-daemon (trusted) worker2 # [ 131.498843] nix-daemon[839]: reaped child process 1323, status = succeeded lb # [ 131.497435] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.515162] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.130Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 131.517083] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.130Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 131.541376] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker2 # [ 131.555701] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.171Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.554113] niks3-server[1137]: 2026/09/23 05:34:11 INFO Signed narinfos id=46 count=1 lb # [ 131.559886] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 131.567646] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 131.576926] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 131.584286] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 131.591725] niks3-server[1137]: 2026/09/23 05:34:11 INFO Completed upload id=46 worker2 # [ 131.597524] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.213Z level=INFO msg="Upload complete. (286ms)" lb # [ 131.593858] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker2 # [ 131.616865] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker2 # [ 131.632390] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker2 # [ 131.637883] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 131.642238] nix-daemon[839]: reaped child process 1302, status = succeeded worker2 # [ 131.659198] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 131.674695] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 131.689250] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.305Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker2 # [ 131.700100] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 131.713540] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 131.866012] nix-daemon[839]: accepted connection from pid 1329, user nix-grpc-daemon (trusted) worker2 # [ 131.882975] nix-daemon[839]: reaped child process 1332, status = succeeded lb # [ 131.884517] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.901943] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 131.903515] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.518Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 131.915789] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign worker2 # [ 131.928532] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.544Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.925901] niks3-server[1137]: 2026/09/23 05:34:11 INFO Signed narinfos id=47 count=1 lb # [ 131.935293] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker2 # [ 131.955421] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.571Z level=INFO msg="Upload complete. (225ms)" lb # [ 131.948786] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 131.954385] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 131.959886] niks3-server[1137]: 2026/09/23 05:34:11 INFO Completed upload id=47 worker2 # [ 131.964269] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 131.962049] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker2 # [ 131.977016] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 131.985604] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 131.990827] nix-daemon[839]: reaped child process 1327, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker2 # [ 132.000912] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 133.040274] nix-daemon[839]: reaped child process 1336, status = succeeded worker2 # [ 133.134259] nix-daemon[839]: accepted connection from pid 1352, user nix-grpc-daemon (trusted) worker2 # [ 133.153278] nix-daemon[839]: reaped child process 1355, status = succeeded lb # [ 133.152981] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.171671] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.787Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 133.173364] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.787Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 133.192602] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 133.206078] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 133.223392] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.839Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.217079] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 133.227252] niks3-server[1137]: 2026/09/23 05:34:12 INFO Signed narinfos id=48 count=1 lb # [ 133.232641] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker2 # [ 133.250613] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.866Z level=INFO msg="Upload complete. (239ms)" lb # [ 133.244793] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 133.252279] niks3-server[1137]: 2026/09/23 05:34:12 INFO Completed upload id=48 lb # [ 133.255072] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker2 # [ 133.263552] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker2 # [ 133.281402] nix-daemon[839]: reaped child process 1334, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.65 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker1 # [ 133.645062] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 133.653568] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 133.803408] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 133.808545] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 133.810445] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 133.836485] nix-grpc-daemon[1511]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 133.837958] nix-grpc-daemon[1511]: E0000 00:00:1790141653.418541 1511 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.861436] nix-grpc-daemon[1511]: E0000 00:00:1790141653.420524 1511 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.864268] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 133.866081] nix-grpc-daemon[1511]: E0000 00:00:1790141653.427180 1551 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.868472] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 133.872163] nix-grpc-daemon[1511]: E0000 00:00:1790141653.448761 1551 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.876585] nix-grpc-daemon[1511]: E0000 00:00:1790141653.448916 1551 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.879009] nix-grpc-daemon[1511]: E0000 00:00:1790141653.464367 1551 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 133.807325] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:49918 worker1 # [ 133.881384] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.814002] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.82 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 134.163378] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 451f01f512af46969b0e02fe37f40805 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker2 # [ 134.361438] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45140 client # [ 134.361885] nix[1406]: don't know how to build these paths: client # [ 134.363071] nix[1406]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 134.380788] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker2 # [ 134.387825] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 134.427242] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 134.443283] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45140 client # [ 134.451211] nix[1406]: copying 1 paths to 'lb:50051'... worker2 # [ 134.458288] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 client # [ 134.453900] nix[1406]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker2 # [ 134.473181] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 134.631407] nix-daemon[839]: accepted connection from pid 1368, user nix-grpc-daemon (trusted) worker2 # [ 134.653591] nix-daemon[839]: reaped child process 1371, status = succeeded lb # [ 134.645596] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.662473] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.278Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 134.663959] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.278Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 134.680736] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker2 # [ 134.699096] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.315Z level=INFO msg="Uploading 1 narinfos" lb # [ 134.693732] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 134.703780] niks3-server[1137]: 2026/09/23 05:34:14 INFO Signed narinfos id=49 count=1 worker2 # [ 134.718938] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.335Z level=INFO msg="Upload complete. (230ms)" lb # [ 134.713211] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 134.721899] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete worker2 # [ 134.727104] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 134.725563] niks3-server[1137]: 2026/09/23 05:34:14 INFO Completed upload id=49 lb # [ 134.728119] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker2 # [ 134.742583] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 worker2 # [ 134.748670] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 134.753606] nix-daemon[839]: reaped child process 1366, status = succeeded client # [ 134.755923] nix[1406]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker2 # [ 134.766082] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 137.075716] systemd-journald[270]: Received client request to rotate journal, rotating. worker1 # [ 137.096608] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/335b9ae880604d8189dd723b100c300d. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 137.073678] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 worker2 # [ 137.941912] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:17Z level=info event=draining lb # [ 139.618927] envoy[904]: conn peer=[2001:db8:1::1]:36738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9658 tx=4422 ms=5404 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 142.660609] nix-daemon[839]: reaped child process 1375, status = succeeded worker2 # [ 142.752957] nix-daemon[839]: accepted connection from pid 1416, user nix-grpc-daemon (trusted) worker2 # [ 142.772998] nix-daemon[839]: reaped child process 1419, status = succeeded lb # [ 142.770874] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 142.791901] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.405Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 142.794103] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.405Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" worker2 # [ 142.836655] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.452Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.822085] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 142.836070] niks3-server[1137]: 2026/09/23 05:34:22 INFO Signed narinfos id=50 count=1 lb # [ 142.841081] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 142.849757] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 142.858811] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 142.870981] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 142.876364] niks3-server[1137]: 2026/09/23 05:34:22 INFO Completed upload id=50 worker2 # [ 142.881826] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.498Z level=INFO msg="Upload complete. (301ms)" lb # [ 142.878918] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker2 # [ 142.897231] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 142.915124] nix-daemon[839]: reaped child process 1373, status = succeeded client # [ 142.920331] systemd[1]: rs.service: Deactivated successfully. client # [ 142.925335] systemd[1]: rs.service: Consumed 188ms CPU time over 8.758s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 142.942506] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:22Z level=info event=signal_exit lb # [ 143.072317] niks3-server[1137]: 2026/09/23 05:34:22 INFO lead: released remote=192.168.1.4:58172 lb # [ 143.747469] niks3-server[1137]: 2026/09/23 05:34:23 INFO lead: acquired remote=192.168.1.3:48088 worker1 # [ 143.825633] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_take_over worker1 # [ 144.152309] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 144.082728] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting lb # [ 144.074955] envoy[904]: [2026-09-23 05:34:23.675][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 144.087733] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 144.096805] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49918 worker2 # [ 144.101466] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 144.106019] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49918 worker2 # [ 144.111237] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 144.134806] nix-daemon[839]: reaped child process 1213, status = succeeded worker1 # [ 144.353291] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.358477] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 144.308645] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 144.559102] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.568158] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:51860 worker1 # [ 144.573299] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.19 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2 # [ 144.999733] nix-grpc-daemon[1160]: E0000 00:00:1790141664.615883 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true worker2 # [ 145.076873] nix-grpc-daemon[1160]: E0000 00:00:1790141664.693162 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2 # [ 145.976284] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 145.982872] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 145.987325] systemd[1]: nix-grpc-daemon.service: Consumed 1.822s CPU time over 37.204s wall clock time, 40.2M memory peak, 62.3K incoming IP traffic, 146.5K outgoing IP traffic. worker2 # [ 146.007350] systemd[1]: Starting scheduler-token.service... worker2 # [ 146.084576] systemd[1]: scheduler-token.service: Deactivated successfully. worker1 # [ 146.153672] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:51866 worker2 # [ 146.088987] systemd[1]: Finished scheduler-token.service. worker2 # [ 146.090880] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 79ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 146.116316] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 146.224358] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 146.226329] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 146.230133] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 146.236062] nix-grpc-daemon[1429]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 146.237544] nix-grpc-daemon[1429]: E0000 00:00:1790141665.850968 1429 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.246680] nix-grpc-daemon[1429]: E0000 00:00:1790141665.857361 1429 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.280126] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 146.281853] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 146.282658] nix-grpc-daemon[1429]: E0000 00:00:1790141665.866320 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.285478] nix-grpc-daemon[1429]: E0000 00:00:1790141665.881512 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.288348] nix-grpc-daemon[1429]: E0000 00:00:1790141665.881670 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.290859] nix-grpc-daemon[1429]: E0000 00:00:1790141665.896004 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.298963] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.378814] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 146.383962] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:51860 worker1 # [ 146.393267] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.00 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 147.164936] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 483d48691df540d494ba2e008b566930 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 147.370917] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50566 worker2 # [ 147.385358] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) client # [ 147.394052] nix[1442]: don't know how to build these paths: client # [ 147.395299] nix[1442]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker1 # [ 147.485210] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 147.491401] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 147.551950] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51870 worker1 # [ 147.566465] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted) worker1 # [ 147.589377] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5) worker1 # [ 147.675213] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5) worker1 # [ 147.836483] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 357 ms (attempt 3/5) worker1 # [ 148.203006] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 614 ms (attempt 4/5) worker1 # [ 148.820603] nix-grpc-daemon[1511]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 148.828570] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 148.846360] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51870 client # [ 148.783791] nix[1442]: copying 1 paths to 'lb:50051'... worker1 # [ 148.864531] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51870 client # [ 148.790484] nix[1442]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 148.879469] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted) worker1 # [ 148.905231] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=info event=niks3_push_started pid=1641 worker1 # [ 149.049776] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.653Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 149.158562] nix-daemon[854]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 149.175399] nix-daemon[854]: reaped child process 1656, status = succeeded lb # [ 149.106239] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.194006] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 149.195444] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.796Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 149.148936] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker1 # [ 149.236508] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.839Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.163096] niks3-server[1137]: 2026/09/23 05:34:28 INFO Signed narinfos id=51 count=1 lb # [ 149.171519] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 149.187891] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 149.199254] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo lb # [ 149.206053] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 149.210764] niks3-server[1137]: 2026/09/23 05:34:28 INFO Completed upload id=51 worker1 # [ 149.287572] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.891Z level=INFO msg="Upload complete. (234ms)" worker1 # [ 149.298860] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51870 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 149.316265] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51870 worker1 # [ 149.327263] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted) worker1 # [ 149.330853] nix-daemon[854]: reaped child process 1639, status = succeeded client # [ 149.265850] nix[1442]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 149.344869] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted) lb # [ 149.623938] envoy[904]: conn peer=[2001:db8:1::1]:47100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1518 ms=8546 lb # [ 149.629879] envoy[904]: conn peer=[2001:db8:1::4]:59060 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=27953 tx=1774 ms=35318 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker1 # [ 151.101547] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1511 (nix-grpc-daemon) on client request. worker1 # [ 151.103131] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 151.041496] niks3-server[1137]: 2026/09/23 05:34:30 INFO lead: released remote=192.168.1.3:48088 client # [ 151.048813] nix[1442]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 151.055550] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:30Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker1 # [ 151.132458] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 151.133967] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. lb # [ 151.050314] envoy[904]: [2026-09-23 05:34:30.651][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} client # [ 151.055143] nix[1442]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker1 # [ 151.135234] systemd[1]: nix-grpc-daemon.service: Consumed 529ms CPU time over 17.473s wall clock time, 34.7M memory peak, 15.6K incoming IP traffic, 26.7K outgoing IP traffic. worker1 # [ 151.157497] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker1 # [ 151.159014] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 151.196195] nix-daemon[854]: reaped child process 1629, status = succeeded worker1 # [ 151.199615] nix-daemon[854]: reaped child process 1658, status = succeeded worker1 # [ 151.208661] nix-daemon[1660]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 151.213973] nix-daemon[854]: reaped child process 1660, status = failed with exit code 1 worker1 # [ 151.274237] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 151.276757] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 151.280441] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 151.285591] nix-grpc-daemon[1696]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 151.287030] nix-grpc-daemon[1696]: E0000 00:00:1790141670.888823 1696 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 151.225372] niks3-server[1137]: 2026/09/23 05:34:30 INFO lead: acquired remote=192.168.1.3:50960 worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.24 seconds) client: must succeed: sleep 5.0 worker1 # [ 151.320429] nix-grpc-daemon[1696]: E0000 00:00:1790141670.894666 1696 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 151.323962] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 151.255990] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.347333] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=scheduler_take_over worker1 # [ 151.348946] nix-grpc-daemon[1696]: E0000 00:00:1790141670.906513 1737 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 151.351437] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:47772 worker1 # [ 151.353023] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 151.353840] nix-grpc-daemon[1696]: E0000 00:00:1790141670.930627 1737 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 151.358782] nix-grpc-daemon[1696]: E0000 00:00:1790141670.930756 1737 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 151.363385] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 151.365180] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:47788 worker1 # [ 151.370804] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 151.372736] nix-grpc-daemon[1696]: E0000 00:00:1790141670.950614 1737 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 151.375075] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.377244] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:47788 worker1 # [ 151.379386] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 151.638177] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:31Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 151.644062] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:31Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 151.629592] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 worker2 # [ 151.644959] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5) worker2 # [ 151.743214] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5) worker2 # [ 151.753822] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 275 ms (attempt 3/5) worker2 # [ 152.037867] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 4/5) worker2 # [ 152.122254] nix-grpc-daemon[1429]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 152.128450] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 152.143765] nix-grpc-daemon[1429]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'... worker2 # [ 152.167800] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker2 # [ 152.190293] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) client # [ 152.182719] nix[1442]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 156.616966] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:36Z level=info event=niks3_push_started pid=1536 worker2 # [ 156.634045] nix-daemon[839]: reaped child process 1511, status = succeeded worker2 # [ 156.770252] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.386Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 156.877480] nix-daemon[839]: accepted connection from pid 1549, user nix-grpc-daemon (trusted) worker2 # [ 156.898971] nix-daemon[839]: reaped child process 1552, status = succeeded lb # [ 156.894105] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 156.916058] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 156.917556] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.531Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 156.953017] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign worker2 # [ 156.973889] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.588Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.968353] niks3-server[1137]: 2026/09/23 05:34:36 INFO Signed narinfos id=52 count=1 lb # [ 156.980239] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 156.983178] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 157.002747] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 157.005557] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo lb # [ 157.009678] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 157.014128] niks3-server[1137]: 2026/09/23 05:34:36 INFO Completed upload id=52 worker2 # [ 157.020963] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.637Z level=INFO msg="Upload complete. (247ms)" worker2 # [ 157.032205] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker2 # [ 157.052328] nix-daemon[839]: reaped child process 1509, status = succeeded client # [ 157.062042] systemd[1]: mid.service: Deactivated successfully. client # [ 157.067154] systemd[1]: mid.service: Consumed 197ms CPU time over 9.898s wall clock time, 41.3M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.60 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 157.736545] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 97153613141c4c558e48483ed1a088a6 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 157.915591] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57404 client # [ 157.915518] nix[1478]: don't know how to build these paths: client # [ 157.916531] nix[1478]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 158.005369] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 158.011758] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 158.072125] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 158.086407] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 158.109545] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5) worker1 # [ 158.165478] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 2/5) worker1 # [ 158.235075] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 3/5) worker1 # [ 158.253855] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 625 ms (attempt 4/5) worker1 # [ 158.918750] nix-grpc-daemon[1696]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 158.919068] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 158.943492] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 client # [ 158.880336] nix[1478]: copying 1 paths to 'lb:50051'... worker1 # [ 158.958858] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 client # [ 158.884173] nix[1478]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 158.975498] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 159.018429] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=info event=niks3_push_started pid=1777 worker1 # [ 159.171434] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.774Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 159.276895] nix-daemon[854]: accepted connection from pid 1791, user nix-grpc-daemon (trusted) worker1 # [ 159.296136] nix-daemon[854]: reaped child process 1794, status = succeeded lb # [ 159.222858] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 159.311713] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.914Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 159.315016] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.914Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 159.261974] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker1 # [ 159.351398] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.954Z level=INFO msg="Uploading 1 narinfos" lb # [ 159.278684] niks3-server[1137]: 2026/09/23 05:34:38 INFO Signed narinfos id=53 count=1 lb # [ 159.286170] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 159.294234] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker1 # [ 159.385383] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.989Z level=INFO msg="Upload complete. (210ms)" lb # [ 159.308307] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 159.314769] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker1 # [ 159.392247] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 159.319563] niks3-server[1137]: 2026/09/23 05:34:38 INFO Completed upload id=53 worker1 # [ 159.407181] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 159.413212] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 159.418037] nix-daemon[854]: reaped child process 1772, status = succeeded worker1 # [ 159.429703] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # [ 159.350882] nix[1478]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... lb # [ 159.624243] envoy[904]: conn peer=[2001:db8:1::3]:33648 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=831 tx=393 ms=17258 lb # [ 159.629208] envoy[904]: conn peer=[2001:db8:1::1]:40846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1988 ms=9684 client: must succeed: systemctl kill -s INT intr worker1 # [ 160.394259] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:39Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 160.317577] systemd[1]: intr.service: Sent signal SIGINT to main process 1478 (nix) on client request. client # [ 160.325771] nix[1478]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 160.335398] nix[1478]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 160.357905] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 160.362334] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 160.364436] systemd[1]: intr.service: Consumed 164ms CPU time over 2.624s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8K outgoing IP traffic. worker1 # [ 160.474857] nix-daemon[1798]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 160.479966] nix-daemon[854]: reaped child process 1796, status = succeeded worker1 # [ 160.484297] nix-daemon[854]: reaped child process 1798, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 2.77 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 160.535141] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 89c55c05c4ea43f68fd916032b558900 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) client # [ 160.727390] nix[1499]: don't know how to build these paths: client # [ 160.728613] nix[1499]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 160.801855] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 160.817981] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 160.823721] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 160.796610] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 worker2 # [ 160.813164] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57404 client # [ 160.820141] nix[1499]: copying 1 paths to 'lb:50051'... client # [ 160.822424] nix[1499]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 160.828623] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57404 worker2 # [ 160.844998] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker2 # [ 161.008938] nix-daemon[839]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker2 # [ 161.028499] nix-daemon[839]: reaped child process 1579, status = succeeded lb # [ 161.027747] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 161.046582] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 161.048455] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.661Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 161.056437] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker2 # [ 161.077307] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.692Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.070141] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 161.075931] niks3-server[1137]: 2026/09/23 05:34:40 INFO Signed narinfos id=54 count=1 lb # [ 161.082566] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 161.092506] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 161.101808] niks3-server[1137]: 2026/09/23 05:34:40 INFO Completed upload id=54 worker2 # [ 161.108636] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.725Z level=INFO msg="Upload complete. (247ms)" lb # [ 161.106190] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 161.120938] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 161.131750] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 worker2 # [ 161.141569] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker2 # [ 161.144353] nix-daemon[839]: reaped child process 1574, status = succeeded client # [ 161.147871] nix[1499]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2 # [ 161.158639] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # [ 163.546396] nixos[1615]: switching to system configuration /nix/store/n3w0z58pg0if7vb7i025y1idh9rrsqkh-nixos-system-worker2-test worker2 # activating the configuration... worker2 # [ 163.551714] systemd[1]: Stopped target Remote File Systems. worker2 # [ 163.557211] systemd[1]: Stopped target Local File Systems. worker2 # setting up /etc... worker2 # [ 164.124801] systemd[1]: Reload requested from client PID 1615 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 164.126924] systemd[1]: Reloading... worker2 # [ 169.019904] systemd[1]: Reloading finished in 4886 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 169.136756] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # [ 169.258180] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # [ 169.261123] systemd[1]: Reached target Reactivate sysinit units. worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service worker2 # [ 169.265372] dbus-broker-launch[493]: Caught SIGHUP, trigger reload. worker2 # [ 169.266337] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 169.273957] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 169.291682] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 169.301331] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' worker2 # [ 169.302331] dbus-broker-launch[493]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 169.316226] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 169.330850] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 169.338944] systemd[1]: Reached target Remote File Systems. worker2 # [ 169.342116] systemd[1]: Starting linger-users.service... worker2 # [ 169.348411] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 169.349782] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 169.360237] systemd[1]: Reached target Local File Systems. worker2 # [ 169.372321] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 169.373752] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 169.374530] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 169.375795] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 169.389996] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 169.391634] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 169.403807] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 169.421298] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 169.424757] systemd[1]: Finished linger-users.service. lb # [ 169.628937] envoy[904]: conn peer=[2001:db8:1::1]:52020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=809 ms=2435 worker2 # [ 169.718435] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 169.719652] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 169.975215] nixos[1615]: finished switching to system configuration /nix/store/n3w0z58pg0if7vb7i025y1idh9rrsqkh-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.88 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker2 # [ 170.259512] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:49Z level=info event=draining worker1 # [ 170.333479] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47816 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 170.346788] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 170.373414] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 170.380064] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 170.426241] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 170.442204] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 client # copying 1 paths to 'lb:50051'... worker1 # [ 170.458103] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 170.481421] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 170.639295] nix-daemon[854]: accepted connection from pid 1845, user nix-grpc-daemon (trusted) worker1 # [ 170.661636] nix-daemon[854]: reaped child process 1848, status = succeeded lb # [ 170.583530] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.671769] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.274Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 170.673928] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.274Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 170.624961] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 170.637938] niks3-server[1137]: 2026/09/23 05:34:50 INFO Signed narinfos id=55 count=1 worker1 # [ 170.718835] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.322Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.652624] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 170.663507] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 170.671078] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 170.677004] niks3-server[1137]: 2026/09/23 05:34:50 INFO Completed upload id=55 worker1 # [ 170.753210] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.357Z level=INFO msg="Upload complete. (254ms)" lb # [ 170.679781] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 170.764382] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=528 worker1 # [ 170.783726] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 170.791640] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 170.798503] nix-daemon[854]: reaped child process 1843, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker1 # [ 170.813130] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 172.324629] nix-daemon[854]: reaped child process 1852, status = succeeded worker1 # [ 172.397586] nix-daemon[854]: accepted connection from pid 1868, user nix-grpc-daemon (trusted) worker1 # [ 172.418508] nix-daemon[854]: reaped child process 1871, status = succeeded lb # [ 172.346919] niks3-server[1137]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.434924] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.036Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 172.437924] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.036Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 172.382486] niks3-server[1137]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 172.399633] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst worker1 # [ 172.487376] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.091Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.412282] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 172.422374] niks3-server[1137]: 2026/09/23 05:34:52 INFO Signed narinfos id=56 count=1 lb # [ 172.431816] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker1 # [ 172.514625] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.118Z level=INFO msg="Upload complete. (286ms)" lb # [ 172.439049] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 172.443821] niks3-server[1137]: 2026/09/23 05:34:52 INFO Completed upload id=56 lb # [ 172.450057] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 172.529785] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 172.553228] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 172.557706] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.562272] nix-daemon[854]: reaped child process 1850, status = succeeded worker1 # [ 172.589627] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 172.611406] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 172.625681] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.229Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 172.636377] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 172.658954] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 172.818043] nix-daemon[854]: accepted connection from pid 1877, user nix-grpc-daemon (trusted) worker1 # [ 172.833741] nix-daemon[854]: reaped child process 1880, status = succeeded lb # [ 172.763630] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.849146] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.452Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 172.850731] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.452Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 172.793294] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker1 # [ 172.880691] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.484Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.808804] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 172.819488] niks3-server[1137]: 2026/09/23 05:34:52 INFO Signed narinfos id=57 count=1 lb # [ 172.825961] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker1 # [ 172.912323] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.516Z level=INFO msg="Upload complete. (236ms)" lb # [ 172.836804] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 172.840459] niks3-server[1137]: 2026/09/23 05:34:52 INFO Completed upload id=57 lb # [ 172.844451] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker1 # [ 172.924315] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 172.938152] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 172.947525] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 172.950965] nix-daemon[854]: reaped child process 1875, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker1 # [ 172.964159] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 173.822895] nix-daemon[854]: reaped child process 1884, status = succeeded worker1 # [ 173.949694] nix-daemon[854]: accepted connection from pid 1900, user nix-grpc-daemon (trusted) worker1 # [ 173.969711] nix-daemon[854]: reaped child process 1903, status = succeeded lb # [ 173.898604] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 173.989840] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.592Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 173.991828] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.592Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 173.945406] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker1 # [ 174.031347] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.634Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.960582] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 173.972999] niks3-server[1137]: 2026/09/23 05:34:53 INFO Signed narinfos id=58 count=1 lb # [ 173.975709] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 173.989769] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete worker1 # [ 174.066770] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.670Z level=INFO msg="Upload complete. (293ms)" lb # [ 173.994672] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 173.999154] niks3-server[1137]: 2026/09/23 05:34:53 INFO Completed upload id=58 lb # [ 174.002619] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 174.082243] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 174.104341] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 174.110066] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 174.115368] nix-daemon[854]: reaped child process 1882, status = succeeded worker1 # [ 174.145589] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 174.160814] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 174.174505] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.778Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 174.186069] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 174.203071] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 174.358153] nix-daemon[854]: accepted connection from pid 1909, user nix-grpc-daemon (trusted) worker1 # [ 174.377683] nix-daemon[854]: reaped child process 1912, status = succeeded lb # [ 174.305767] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.391432] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.995Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 174.393091] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.995Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 174.335425] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign worker1 # [ 174.420260] nix-grpc-daemon[1777]: time=2026-09-23T05:34:54.024Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.350368] niks3-server[1137]: 2026/09/23 05:34:53 INFO Signed narinfos id=59 count=1 lb # [ 174.357870] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 174.365771] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 174.376115] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete worker1 # [ 174.457413] nix-grpc-daemon[1777]: time=2026-09-23T05:34:54.061Z level=INFO msg="Upload complete. (241ms)" lb # [ 174.383169] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo lb # [ 174.388128] niks3-server[1137]: 2026/09/23 05:34:53 INFO Completed upload id=59 worker1 # [ 174.469044] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 174.483799] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 174.490317] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 174.495362] nix-daemon[854]: reaped child process 1907, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker1 # [ 174.507902] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 175.947391] nix-daemon[854]: reaped child process 1916, status = succeeded worker1 # [ 176.052347] nix-daemon[854]: accepted connection from pid 1932, user nix-grpc-daemon (trusted) worker1 # [ 176.069632] nix-daemon[854]: reaped child process 1935, status = succeeded lb # [ 176.000963] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.088374] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.691Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.089847] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.691Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 176.038551] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker1 # [ 176.124457] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.728Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.053424] niks3-server[1137]: 2026/09/23 05:34:55 INFO Signed narinfos id=60 count=1 lb # [ 176.061525] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 176.068976] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 176.078960] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 176.163364] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.767Z level=INFO msg="Upload complete. (242ms)" lb # [ 176.086646] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 176.091426] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo lb # [ 176.097967] niks3-server[1137]: 2026/09/23 05:34:55 INFO Completed upload id=60 worker1 # [ 176.174197] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker1 # [ 176.191570] nix-daemon[854]: reaped child process 1914, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.13 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 176.635202] nix-daemon[839]: reaped child process 1583, status = succeeded worker2 # [ 176.770048] nix-daemon[839]: accepted connection from pid 1755, user nix-grpc-daemon (trusted) worker2 # [ 176.790428] nix-daemon[839]: reaped child process 1758, status = succeeded lb # [ 176.788647] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 176.806449] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 176.808167] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.421Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" worker2 # [ 176.849184] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.465Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.831219] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 176.845819] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 176.856982] niks3-server[1137]: 2026/09/23 05:34:56 INFO Signed narinfos id=61 count=1 lb # [ 176.859526] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 176.881073] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.497Z level=INFO msg="Upload complete. (294ms)" lb # [ 176.874109] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 176.879111] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 176.885217] niks3-server[1137]: 2026/09/23 05:34:56 INFO Completed upload id=61 worker2 # [ 176.893230] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 lb # [ 176.889629] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 176.914517] nix-daemon[839]: reaped child process 1581, status = succeeded client # [ 176.923950] systemd[1]: sw.service: Deactivated successfully. client # [ 176.928447] systemd[1]: sw.service: Consumed 184ms CPU time over 16.392s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.6K outgoing IP traffic. worker2 # [ 177.261619] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:56Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 177.621955] systemd[1]: Starting scheduler-token.service... worker2 # [ 177.684516] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 177.688576] systemd[1]: Finished scheduler-token.service. worker2 # [ 177.690140] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 66ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 179.259041] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:58Z level=info event=scheduler_restarting worker1 # [ 179.331269] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:47788 worker1 # [ 179.336511] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:58Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 179.286382] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 179.313565] nix-daemon[839]: reaped child process 1486, status = succeeded lb # [ 179.623115] envoy[904]: conn peer=[2001:db8:1::1]:52436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=4414 ms=5869 lb # [ 179.628257] envoy[904]: conn peer=[2001:db8:1::1]:52032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3047 tx=1559 ms=16180 lb # [ 179.633140] envoy[904]: conn peer=[2001:db8:1::4]:48672 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2289 tx=476 ms=32978 worker2 # [ 180.032582] nix-grpc-daemon[1429]: E0000 00:00:1790141699.648813 1475 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true worker2 # [ 180.286241] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 180.292406] systemd[1]: nix-grpc-daemon.service: Consumed 875ms CPU time over 34.182s wall clock time, 36.3M memory peak, 21.9K incoming IP traffic, 29.9K outgoing IP traffic. worker2 # [ 180.313669] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 180.464448] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 180.466913] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 180.470852] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 180.476054] nix-grpc-daemon[1775]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 180.477705] nix-grpc-daemon[1775]: E0000 00:00:1790141700.092115 1775 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.98 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 180.524511] nix-grpc-daemon[1775]: E0000 00:00:1790141700.098644 1775 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 180.527280] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 180.529274] nix-grpc-daemon[1775]: E0000 00:00:1790141700.106312 1816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 180.531589] nix-grpc-daemon[1775]: E0000 00:00:1790141700.122551 1816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 180.534060] nix-grpc-daemon[1775]: E0000 00:00:1790141700.122769 1816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 180.539548] nix-grpc-daemon[1775]: E0000 00:00:1790141700.139397 1816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 180.544097] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 180.545065] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 180.617233] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:47788 worker1 # [ 180.623469] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 20.24 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 180.947802] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 17ca131077564394ac222f4ef3b6812c client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2 # [ 181.158523] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 181.173414] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # [ 181.184354] nix[1555]: don't know how to build these paths: client # [ 181.186572] nix[1555]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv client # [ 181.189444] nix[1555]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv client # [ 181.191379] nix[1555]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv client # [ 181.193200] nix[1555]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker1 # [ 181.275613] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=0 worker1 # [ 181.282738] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.289828] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=0 worker1 # [ 181.295283] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 181.302531] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=0 worker1 # [ 181.308239] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=0 worker1 # [ 181.322556] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker2 # [ 181.253054] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker1 # [ 181.338926] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker2 # [ 181.269763] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) client # [ 181.276290] nix[1555]: copying 1 paths to 'lb:50051'... worker1 # [ 181.353526] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 client # [ 181.278660] nix[1555]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'... worker1 # [ 181.370173] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker2 # [ 181.342951] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5) worker1 # [ 181.526657] nix-daemon[854]: accepted connection from pid 1950, user nix-grpc-daemon (trusted) worker1 # [ 181.546756] nix-daemon[854]: reaped child process 1953, status = succeeded lb # [ 181.475135] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.561612] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 181.563173] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.165Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)" lb # [ 181.506501] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 181.592863] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.196Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.521985] niks3-server[1137]: 2026/09/23 05:35:01 INFO Signed narinfos id=62 count=1 lb # [ 181.529332] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst worker2 # [ 181.539280] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5) lb # [ 181.539123] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls worker1 # [ 181.624541] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.228Z level=INFO msg="Upload complete. (238ms)" lb # [ 181.548298] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 181.551731] niks3-server[1137]: 2026/09/23 05:35:01 INFO Completed upload id=62 lb # [ 181.555307] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo worker1 # [ 181.637854] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 181.652001] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 181.661438] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 181.664367] nix-daemon[854]: reaped child process 1948, status = succeeded client # [ 181.598182] nix[1555]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051... worker1 # [ 181.678118] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker2 # [ 181.886107] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 4/5) worker2 # [ 181.899577] nix-grpc-daemon[1775]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 181.906617] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 181.924208] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 client # [ 181.931844] nix[1555]: copying 1 paths to 'lb:50051'... worker2 # [ 181.938959] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 client # [ 181.934350] nix[1555]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'... worker2 # [ 181.953317] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 181.980848] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=info event=niks3_push_started pid=1836 worker2 # [ 182.128422] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.744Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 182.238367] nix-daemon[839]: accepted connection from pid 1849, user nix-grpc-daemon (trusted) worker2 # [ 182.254316] nix-daemon[839]: reaped child process 1852, status = succeeded lb # [ 182.258608] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received uploads request method=POST path=/api/pending_closures lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 182.270520] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 182.271936] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.885Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)" lb # [ 182.284414] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls worker2 # [ 182.300557] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.916Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.293288] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 182.300838] niks3-server[1137]: 2026/09/23 05:35:01 INFO Signed narinfos id=63 count=1 lb # [ 182.307350] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 182.315560] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst worker2 # [ 182.326320] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.942Z level=INFO msg="Upload complete. (194ms)" lb # [ 182.320269] niks3-server[1137]: 2026/09/23 05:35:01 INFO Completed upload id=63 worker2 # [ 182.332282] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 182.325136] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo worker2 # [ 182.342004] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 182.347170] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 182.350622] nix-daemon[839]: reaped child process 1834, status = succeeded worker2 # [ 182.360189] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # [ 182.352674] nix[1555]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051... worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds) worker1 # [ 183.708838] nix-daemon[854]: reaped child process 1957, status = succeeded worker1 # [ 183.795302] nix-daemon[854]: accepted connection from pid 1985, user nix-grpc-daemon (trusted) worker1 # [ 183.814522] nix-daemon[854]: reaped child process 1988, status = succeeded lb # [ 183.744089] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.832214] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 183.833657] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.433Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)" lb # [ 183.786732] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign worker1 # [ 183.871977] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.475Z level=INFO msg="Uploading 1 narinfos" worker2 # [ 183.815080] nix-daemon[839]: reaped child process 1856, status = succeeded lb # [ 183.801782] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=64 count=1 lb # [ 183.810881] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls lb # [ 183.821141] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst worker1 # [ 183.904636] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.508Z level=INFO msg="Upload complete. (216ms)" lb # [ 183.829956] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv lb # [ 183.836940] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 183.841242] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=64 worker1 # [ 183.916824] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=1 lb # [ 183.843719] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo worker1 # [ 183.930499] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 183.934528] nix-daemon[854]: reaped child process 1955, status = succeeded worker1 # [ 183.958460] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 183.967606] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 client # [ 183.905268] nix[1555]: copying 1 paths to 'lb:50051'... worker1 # [ 183.981672] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 client # [ 183.905537] nix[1555]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'... worker2 # [ 183.920130] nix-daemon[839]: accepted connection from pid 1881, user nix-grpc-daemon (trusted) worker1 # [ 183.997899] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker2 # [ 183.938333] nix-daemon[839]: reaped child process 1884, status = succeeded lb # [ 183.938882] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 183.956342] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 183.957896] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.570Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)" lb # [ 183.982639] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst worker2 # [ 184.001369] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.617Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.994708] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv lb # [ 184.003358] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 184.012580] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=65 count=1 worker2 # [ 184.019426] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.635Z level=INFO msg="Upload complete. (250ms)" lb # [ 184.015459] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls worker1 # [ 184.096368] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=1 lb # [ 184.020804] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 184.024287] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=65 worker2 # [ 184.029038] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=1 lb # [ 184.027038] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo worker2 # [ 184.048081] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 184.052935] nix-daemon[839]: reaped child process 1854, status = succeeded worker2 # [ 184.059245] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 client # [ 184.060618] nix[1555]: copying 1 paths to 'lb:50051'... client # [ 184.065439] nix[1555]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'... worker2 # [ 184.073609] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 184.083513] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker1 # [ 184.170803] nix-daemon[854]: accepted connection from pid 1992, user nix-grpc-daemon (trusted) worker1 # [ 184.191007] nix-daemon[854]: reaped child process 1995, status = succeeded lb # [ 184.112835] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.198391] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.802Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 184.200199] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.802Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)" lb # [ 184.142988] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls worker1 # [ 184.229530] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.833Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.155217] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 184.163810] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=66 count=1 lb # [ 184.167494] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst worker1 # [ 184.253134] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.857Z level=INFO msg="Upload complete. (240ms)" lb # [ 184.178031] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 184.182209] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=66 lb # [ 184.184779] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo worker1 # [ 184.260102] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 184.274153] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 184.279712] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 184.284853] nix-daemon[854]: reaped child process 1990, status = succeeded worker2 # [ 184.219969] nix-daemon[839]: accepted connection from pid 1888, user nix-grpc-daemon (trusted) worker1 # [ 184.295471] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # [ 184.217228] nix[1555]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051... worker2 # [ 184.239872] nix-daemon[839]: reaped child process 1891, status = succeeded lb # [ 184.238649] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 184.253902] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 184.255376] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.870Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)" lb # [ 184.267217] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker2 # [ 184.279813] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.895Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.280882] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=67 count=1 lb # [ 184.283261] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls worker2 # [ 184.303019] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.919Z level=INFO msg="Upload complete. (207ms)" lb # [ 184.296315] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst lb # [ 184.301323] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 184.305344] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=67 worker2 # [ 184.312712] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 184.309035] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo worker2 # [ 184.321873] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 184.327708] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 184.331781] nix-daemon[839]: reaped child process 1886, status = succeeded worker2 # [ 184.341793] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # [ 184.333729] nix[1555]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051... worker1 # [ 186.124988] nix-daemon[854]: reaped child process 1999, status = succeeded worker1 # [ 186.218908] nix-daemon[854]: accepted connection from pid 2024, user nix-grpc-daemon (trusted) worker1 # [ 186.238658] nix-daemon[854]: reaped child process 2027, status = succeeded lb # [ 186.166615] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 186.180982] nix-daemon[839]: reaped child process 1895, status = succeeded worker1 # [ 186.256386] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.858Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 186.257871] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.858Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)" lb # [ 186.211976] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 186.298272] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.902Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.225845] niks3-server[1137]: 2026/09/23 05:35:05 INFO Signed narinfos id=68 count=1 lb # [ 186.233860] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv lb # [ 186.242150] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 186.252401] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls worker1 # [ 186.327977] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.931Z level=INFO msg="Upload complete. (219ms)" lb # [ 186.254972] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 186.259702] niks3-server[1137]: 2026/09/23 05:35:05 INFO Completed upload id=68 lb # [ 186.261703] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo worker1 # [ 186.340782] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=1 worker1 # [ 186.352170] nix-daemon[854]: reaped child process 1997, status = succeeded worker2 # [ 186.287007] nix-daemon[839]: accepted connection from pid 1920, user nix-grpc-daemon (trusted) worker2 # [ 186.304352] nix-daemon[839]: reaped child process 1923, status = succeeded lb # [ 186.303330] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 186.319282] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.934Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 186.320918] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.934Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)" lb # [ 186.336745] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker2 # [ 186.349890] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.966Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.350162] niks3-server[1137]: 2026/09/23 05:35:05 INFO Signed narinfos id=69 count=1 lb # [ 186.355834] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst worker2 # [ 186.376476] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.992Z level=INFO msg="Upload complete. (223ms)" lb # [ 186.369586] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls lb # [ 186.374070] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 186.378502] niks3-server[1137]: 2026/09/23 05:35:05 INFO Completed upload id=69 lb # [ 186.383851] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv lb # [ 186.388997] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo worker2 # [ 186.385158] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=1 worker2 # [ 186.399860] nix-daemon[839]: reaped child process 1893, status = succeeded client # [ 186.418029] systemd[1]: burst.service: Deactivated successfully. client # [ 186.423058] systemd[1]: burst.service: Consumed 224ms CPU time over 5.474s wall clock time, 51.7M memory peak, 14.2K incoming IP traffic, 18.9K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.87 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 187.888856] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 187.895439] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 187.937124] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 187.944137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 187.950028] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 187.955287] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 187.985386] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 188.000526] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 188.014972] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 188.029622] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 188.182802] nix-daemon[854]: accepted connection from pid 2040, user nix-grpc-daemon (trusted) worker1 # [ 188.201585] nix-daemon[854]: reaped child process 2043, status = succeeded lb # [ 188.129774] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.218577] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 188.220353] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.821Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 188.162622] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 188.251418] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.855Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.177742] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 188.186736] niks3-server[1137]: 2026/09/23 05:35:07 INFO Signed narinfos id=70 count=1 worker1 # [ 188.273557] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.877Z level=INFO msg="Upload complete. (228ms)" lb # [ 188.196603] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 188.282738] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 188.205252] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 188.211894] niks3-server[1137]: 2026/09/23 05:35:07 INFO Completed upload id=70 lb # [ 188.215445] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 188.296882] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 188.305998] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 188.311469] nix-daemon[854]: reaped child process 2038, status = succeeded worker1 # [ 188.322133] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 189.631966] nix-daemon[854]: reaped child process 2047, status = succeeded worker1 # [ 189.705626] nix-daemon[854]: accepted connection from pid 2063, user nix-grpc-daemon (trusted) lb # [ 189.627431] envoy[904]: conn peer=[2001:db8:1::1]:47760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9688 tx=5198 ms=5252 worker1 # [ 189.723413] nix-daemon[854]: reaped child process 2066, status = succeeded lb # [ 189.652226] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.739629] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.342Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 189.741443] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.342Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" worker1 # [ 189.780915] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.384Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.693362] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 189.709788] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 189.719734] niks3-server[1137]: 2026/09/23 05:35:09 INFO Signed narinfos id=71 count=1 lb # [ 189.725287] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 189.813400] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.417Z level=INFO msg="Upload complete. (278ms)" lb # [ 189.731871] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 189.740842] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 189.745962] niks3-server[1137]: 2026/09/23 05:35:09 INFO Completed upload id=71 lb # [ 189.749045] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 189.833057] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=1 worker1 # [ 189.848782] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 189.855554] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 189.861137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 189.866249] nix-daemon[854]: reaped child process 2045, status = succeeded worker1 # [ 189.870290] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 189.879527] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 189.893105] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 190.022697] nix-daemon[854]: accepted connection from pid 2072, user nix-grpc-daemon (trusted) worker1 # [ 190.039597] nix-daemon[854]: reaped child process 2075, status = succeeded lb # [ 189.968781] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.054202] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 190.055689] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.657Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 189.998192] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 190.088255] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.692Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.013514] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 190.024676] niks3-server[1137]: 2026/09/23 05:35:09 INFO Signed narinfos id=72 count=1 lb # [ 190.034846] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 190.117030] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.721Z level=INFO msg="Upload complete. (215ms)" lb # [ 190.045998] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 190.124137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 190.051618] niks3-server[1137]: 2026/09/23 05:35:09 INFO Completed upload id=72 lb # [ 190.054622] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 190.139505] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 190.145949] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 190.150452] nix-daemon[854]: reaped child process 2070, status = succeeded worker1 # [ 190.161173] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 191.636677] nix-daemon[854]: reaped child process 2079, status = succeeded worker1 # [ 191.734970] nix-daemon[854]: accepted connection from pid 2095, user nix-grpc-daemon (trusted) worker1 # [ 191.752999] nix-daemon[854]: reaped child process 2098, status = succeeded lb # [ 191.681034] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.768402] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.371Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.772727] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.371Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 191.720816] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 191.810191] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.414Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.736319] niks3-server[1137]: 2026/09/23 05:35:11 INFO Signed narinfos id=73 count=1 lb # [ 191.739108] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 191.751854] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 191.763382] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 191.840287] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.444Z level=INFO msg="Upload complete. (267ms)" lb # [ 191.768264] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 191.772892] niks3-server[1137]: 2026/09/23 05:35:11 INFO Completed upload id=73 lb # [ 191.775893] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 191.851223] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=1 worker1 # [ 191.876297] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 191.881485] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 191.886913] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 191.892112] nix-daemon[854]: reaped child process 2077, status = succeeded worker1 # [ 191.894009] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 191.902456] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 191.913197] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 192.050670] nix-daemon[854]: accepted connection from pid 2104, user nix-grpc-daemon (trusted) worker1 # [ 192.067311] nix-daemon[854]: reaped child process 2107, status = succeeded lb # [ 191.996999] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.082322] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.686Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 192.083777] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.686Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 192.025255] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign worker1 # [ 192.108542] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.712Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.039369] niks3-server[1137]: 2026/09/23 05:35:11 INFO Signed narinfos id=74 count=1 lb # [ 192.047112] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 192.060870] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst worker1 # [ 192.143803] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.747Z level=INFO msg="Upload complete. (219ms)" lb # [ 192.067778] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 192.078169] niks3-server[1137]: 2026/09/23 05:35:11 INFO Completed upload id=74 lb # [ 192.081000] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 192.152273] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 192.166799] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 192.175463] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 192.178148] nix-daemon[854]: reaped child process 2102, status = succeeded worker1 # [ 192.190465] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 193.418935] nix-daemon[854]: reaped child process 2111, status = succeeded worker1 # [ 193.522794] nix-daemon[854]: accepted connection from pid 2127, user nix-grpc-daemon (trusted) worker1 # [ 193.540460] nix-daemon[854]: reaped child process 2130, status = succeeded lb # [ 193.471037] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.559257] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 193.560980] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.162Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 193.516325] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker1 # [ 193.600108] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.203Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.528787] niks3-server[1137]: 2026/09/23 05:35:13 INFO Signed narinfos id=75 count=1 lb # [ 193.531344] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 193.542881] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 193.547520] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 193.557081] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete worker1 # [ 193.633281] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.237Z level=INFO msg="Upload complete. (232ms)" lb # [ 193.560042] niks3-server[1137]: 2026/09/23 05:35:13 INFO Completed upload id=75 lb # [ 193.563558] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 193.649625] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=1 worker1 # [ 193.668339] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 193.673374] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 193.678437] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 193.683131] nix-daemon[854]: reaped child process 2109, status = succeeded worker1 # [ 193.686145] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 193.698013] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.302Z level=INFO msg="All 1 paths already cached" worker1 # [ 193.705664] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 193.718767] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 193.850815] nix-daemon[854]: accepted connection from pid 2136, user nix-grpc-daemon (trusted) worker1 # [ 193.868457] nix-daemon[854]: reaped child process 2139, status = succeeded lb # [ 193.798804] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.883931] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 193.886161] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.489Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 193.833974] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 193.917189] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.521Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.848428] niks3-server[1137]: 2026/09/23 05:35:13 INFO Signed narinfos id=76 count=1 lb # [ 193.851452] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls worker1 # [ 193.942152] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.546Z level=INFO msg="Upload complete. (214ms)" lb # [ 193.866796] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst lb # [ 193.873304] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 193.877935] niks3-server[1137]: 2026/09/23 05:35:13 INFO Completed upload id=76 worker1 # [ 193.952298] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 193.880958] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 193.965612] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 193.971693] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 193.976668] nix-daemon[854]: reaped child process 2134, status = succeeded worker1 # [ 193.988462] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 195.092419] nix-daemon[854]: reaped child process 2143, status = succeeded worker1 # [ 195.166315] nix-daemon[854]: accepted connection from pid 2159, user nix-grpc-daemon (trusted) worker1 # [ 195.184962] nix-daemon[854]: reaped child process 2162, status = succeeded lb # [ 195.114805] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.203922] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.805Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 195.206010] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.805Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 195.148983] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 195.161875] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst worker1 # [ 195.253714] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.857Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.176684] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 195.189889] niks3-server[1137]: 2026/09/23 05:35:14 INFO Signed narinfos id=77 count=1 lb # [ 195.197152] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 195.285561] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.889Z level=INFO msg="Upload complete. (287ms)" lb # [ 195.210298] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 195.218125] niks3-server[1137]: 2026/09/23 05:35:14 INFO Completed upload id=77 lb # [ 195.221614] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 195.303393] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=1 worker1 # [ 195.324918] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 195.335884] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 195.342601] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 195.348915] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 195.367011] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.951Z level=INFO msg="All 1 paths already cached" worker1 # [ 195.370756] nix-daemon[854]: reaped child process 2141, status = succeeded worker1 # [ 195.373385] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 195.375539] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 195.505568] nix-daemon[854]: accepted connection from pid 2168, user nix-grpc-daemon (trusted) worker1 # [ 195.521410] nix-daemon[854]: reaped child process 2171, status = succeeded lb # [ 195.449799] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.535230] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.138Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 195.536803] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.138Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 195.476786] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 195.561335] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.165Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.492682] niks3-server[1137]: 2026/09/23 05:35:15 INFO Signed narinfos id=78 count=1 lb # [ 195.495892] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 195.586825] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.190Z level=INFO msg="Upload complete. (206ms)" lb # [ 195.509907] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 195.517046] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete worker1 # [ 195.594715] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 195.521547] niks3-server[1137]: 2026/09/23 05:35:15 INFO Completed upload id=78 lb # [ 195.523594] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 195.604734] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 195.613978] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 195.618230] nix-daemon[854]: reaped child process 2166, status = succeeded worker1 # [ 195.627916] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 196.496775] nix-daemon[854]: reaped child process 2175, status = succeeded worker1 # [ 196.597819] nix-daemon[854]: accepted connection from pid 2191, user nix-grpc-daemon (trusted) worker1 # [ 196.616361] nix-daemon[854]: reaped child process 2194, status = succeeded lb # [ 196.543889] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.630880] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.233Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.632616] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.233Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 196.584984] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 196.673134] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.276Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.599517] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 196.610623] niks3-server[1137]: 2026/09/23 05:35:16 INFO Signed narinfos id=79 count=1 lb # [ 196.615335] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 196.622928] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 196.705687] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.309Z level=INFO msg="Upload complete. (267ms)" lb # [ 196.629308] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 196.633004] niks3-server[1137]: 2026/09/23 05:35:16 INFO Completed upload id=79 lb # [ 196.637551] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 196.722934] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=1 worker1 # [ 196.739696] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 196.750699] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 196.756179] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 196.762202] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 196.767861] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.370Z level=INFO msg="All 1 paths already cached" worker1 # [ 196.783774] nix-daemon[854]: reaped child process 2173, status = succeeded worker1 # [ 196.785893] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 196.790670] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 196.916941] nix-daemon[854]: accepted connection from pid 2200, user nix-grpc-daemon (trusted) worker1 # [ 196.934520] nix-daemon[854]: reaped child process 2203, status = succeeded lb # [ 196.860338] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.945707] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.549Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.947150] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.549Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 196.892237] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 196.978392] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.582Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.905394] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 196.915332] niks3-server[1137]: 2026/09/23 05:35:16 INFO Signed narinfos id=80 count=1 lb # [ 196.919242] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 197.005834] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.609Z level=INFO msg="Upload complete. (211ms)" lb # [ 196.929845] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 196.934414] niks3-server[1137]: 2026/09/23 05:35:16 INFO Completed upload id=80 worker1 # [ 197.012325] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 196.937675] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 197.024751] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 197.030232] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 197.035365] nix-daemon[854]: reaped child process 2198, status = succeeded worker1 # [ 197.045075] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 197.942908] nix-daemon[854]: reaped child process 2207, status = succeeded worker1 # [ 198.012888] nix-daemon[854]: accepted connection from pid 2223, user nix-grpc-daemon (trusted) worker1 # [ 198.032970] nix-daemon[854]: reaped child process 2226, status = succeeded lb # [ 197.955158] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.042808] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.644Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 198.044370] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.644Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 197.995992] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 198.005571] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst worker1 # [ 198.093047] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.696Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.020444] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 198.029096] niks3-server[1137]: 2026/09/23 05:35:17 INFO Signed narinfos id=81 count=1 lb # [ 198.031838] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker1 # [ 198.122112] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.726Z level=INFO msg="Upload complete. (266ms)" lb # [ 198.046976] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 198.050783] niks3-server[1137]: 2026/09/23 05:35:17 INFO Completed upload id=81 lb # [ 198.054152] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 198.136803] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=1 worker1 # [ 198.152285] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 198.158317] nix-daemon[854]: reaped child process 2205, status = succeeded worker1 # [ 198.179810] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 198.188121] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 198.199393] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 198.213216] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.817Z level=INFO msg="All 2 paths already cached" worker1 # [ 198.223192] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 198.239423] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 198.377933] nix-daemon[854]: accepted connection from pid 2232, user nix-grpc-daemon (trusted) worker1 # [ 198.396944] nix-daemon[854]: reaped child process 2235, status = succeeded lb # [ 198.325751] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.411216] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.014Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 198.412849] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.014Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" worker1 # [ 198.440767] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.044Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.357929] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 198.373701] niks3-server[1137]: 2026/09/23 05:35:17 INFO Signed narinfos id=82 count=1 lb # [ 198.380771] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 198.387252] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 198.474528] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.078Z level=INFO msg="Upload complete. (224ms)" lb # [ 198.396732] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 198.406335] niks3-server[1137]: 2026/09/23 05:35:18 INFO Completed upload id=82 worker1 # [ 198.485095] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 198.410193] niks3-server[1137]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 198.499970] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 198.508163] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 198.511074] nix-daemon[854]: reaped child process 2230, status = succeeded worker1 # [ 198.521058] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 199.622620] nix-daemon[854]: reaped child process 2239, status = succeeded worker1 # [ 199.700993] nix-daemon[854]: accepted connection from pid 2255, user nix-grpc-daemon (trusted) worker1 # [ 199.722944] nix-daemon[854]: reaped child process 2258, status = succeeded lb # [ 199.645253] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 199.734592] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.337Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 199.736081] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.337Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 199.691050] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 199.702759] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign worker1 # [ 199.784170] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.387Z level=INFO msg="Uploading 1 narinfos" lb # [ 199.713848] niks3-server[1137]: 2026/09/23 05:35:19 INFO Signed narinfos id=83 count=1 lb # [ 199.718242] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 199.727523] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 199.732884] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete worker1 # [ 199.809476] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.413Z level=INFO msg="Upload complete. (277ms)" lb # [ 199.735987] niks3-server[1137]: 2026/09/23 05:35:19 INFO Completed upload id=83 lb # [ 199.739291] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 199.821880] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=1 worker1 # [ 199.838984] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 199.843959] nix-daemon[854]: reaped child process 2237, status = succeeded worker1 # [ 199.865227] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 199.880615] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 199.894329] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.498Z level=INFO msg="All 2 paths already cached" worker1 # [ 199.904147] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 199.919957] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 200.067134] nix-daemon[854]: accepted connection from pid 2262, user nix-grpc-daemon (trusted) worker1 # [ 200.085431] nix-daemon[854]: reaped child process 2265, status = succeeded lb # [ 200.011325] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.096776] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.700Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 200.098305] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.700Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 200.040531] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign worker1 # [ 200.125422] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.729Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.055837] niks3-server[1137]: 2026/09/23 05:35:19 INFO Signed narinfos id=84 count=1 lb # [ 200.062627] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 200.073593] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 200.155551] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.759Z level=INFO msg="Upload complete. (220ms)" lb # [ 200.079245] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 200.088697] niks3-server[1137]: 2026/09/23 05:35:19 INFO Completed upload id=84 worker1 # [ 200.163799] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 200.090923] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 200.178190] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 200.187717] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 200.190897] nix-daemon[854]: reaped child process 2260, status = succeeded worker1 # [ 200.202147] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 201.028664] nix-daemon[854]: reaped child process 2269, status = succeeded worker1 # [ 201.130512] nix-daemon[854]: accepted connection from pid 2285, user nix-grpc-daemon (trusted) worker1 # [ 201.145914] nix-daemon[854]: reaped child process 2288, status = succeeded lb # [ 201.076941] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 201.164379] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 201.165856] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.766Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 201.121046] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker1 # [ 201.206925] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.810Z level=INFO msg="Uploading 1 narinfos" lb # [ 201.134642] niks3-server[1137]: 2026/09/23 05:35:20 INFO Signed narinfos id=85 count=1 lb # [ 201.142291] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 201.151829] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker1 # [ 201.238677] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.842Z level=INFO msg="Upload complete. (225ms)" lb # [ 201.162355] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 201.167165] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 201.173447] niks3-server[1137]: 2026/09/23 05:35:20 INFO Completed upload id=85 lb # [ 201.178555] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 201.252243] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=1 worker1 # [ 201.267270] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 201.272305] nix-daemon[854]: reaped child process 2267, status = succeeded worker1 # [ 201.297008] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 201.312424] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 201.326869] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.930Z level=INFO msg="All 2 paths already cached" worker1 # [ 201.336693] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 worker1 # [ 201.352943] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 201.494469] nix-daemon[854]: accepted connection from pid 2292, user nix-grpc-daemon (trusted) worker1 # [ 201.513489] nix-daemon[854]: reaped child process 2295, status = succeeded lb # [ 201.434794] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 201.523810] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.127Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 201.525660] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.127Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 201.466998] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign worker1 # [ 201.554433] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.158Z level=INFO msg="Uploading 1 narinfos" lb # [ 201.483076] niks3-server[1137]: 2026/09/23 05:35:21 INFO Signed narinfos id=86 count=1 lb # [ 201.489841] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls worker1 # [ 201.577414] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.181Z level=INFO msg="Upload complete. (215ms)" lb # [ 201.501101] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 201.584106] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 201.510779] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete lb # [ 201.515107] niks3-server[1137]: 2026/09/23 05:35:21 INFO Completed upload id=86 worker1 # [ 201.593129] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 lb # [ 201.518283] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 201.603627] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 201.608785] nix-daemon[854]: reaped child process 2290, status = succeeded worker1 # [ 201.621711] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 202.649585] nix-daemon[854]: reaped child process 2299, status = succeeded worker1 # [ 202.750965] nix-daemon[854]: accepted connection from pid 2315, user nix-grpc-daemon (trusted) worker1 # [ 202.767776] nix-daemon[854]: reaped child process 2318, status = succeeded lb # [ 202.698101] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.785889] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.787330] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.388Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 202.738413] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign worker1 # [ 202.824945] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.428Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.753777] niks3-server[1137]: 2026/09/23 05:35:22 INFO Signed narinfos id=87 count=1 lb # [ 202.760763] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 202.767808] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 202.777778] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker1 # [ 202.859347] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.463Z level=INFO msg="Upload complete. (229ms)" lb # [ 202.783591] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 202.787347] niks3-server[1137]: 2026/09/23 05:35:22 INFO Completed upload id=87 lb # [ 202.790911] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 202.873417] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=1 worker1 # [ 202.887835] nix-daemon[854]: reaped child process 2297, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.28 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 202.991106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 93639ee65da94e75b2455bd0d7a37e51 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 203.193174] nix[1624]: don't know how to build these paths: client # [ 203.194201] nix[1624]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker1 # [ 203.268966] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47816 worker2 # [ 203.206930] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 203.301864] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 203.231683] nix[1624]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 209.633359] envoy[904]: conn peer=[2001:db8:1::1]:47776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27123 tx=12319 ms=15003 client # [ 233.236872] nix[1624]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 233.250371] nix[1624]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 233.290222] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 233.298478] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 233.302353] systemd[1]: gpu.service: Consumed 202ms CPU time over 30.304s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.06 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.07 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 235.335692] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:54Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 236.174584] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 236.183202] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 worker1 # [ 236.275212] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:55Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 236.275619] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:55Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 236.250659] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 236.265977] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 236.281830] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 236.296977] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 236.460163] nix-daemon[839]: accepted connection from pid 1931, user nix-grpc-daemon (trusted) worker2 # [ 236.482500] nix-daemon[839]: reaped child process 1934, status = succeeded lb # [ 236.477745] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 236.496414] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 236.498306] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.112Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 236.514972] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 236.546930] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.162Z level=INFO msg="Uploading 1 narinfos" lb # [ 236.533978] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign lb # [ 236.544278] niks3-server[1137]: 2026/09/23 05:35:56 INFO Signed narinfos id=88 count=1 lb # [ 236.556122] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 236.569891] niks3-server[1137]: 2026/09/23 05:35:56 INFO Completed upload id=88 worker2 # [ 236.575751] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.192Z level=INFO msg="Upload complete. (261ms)" lb # [ 236.576070] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls worker2 # [ 236.588655] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 236.589588] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 236.600560] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 236.615673] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 236.624187] nix-daemon[839]: reaped child process 1929, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... worker2 # [ 236.632073] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 237.500806] nix-daemon[839]: reaped child process 1938, status = succeeded worker2 # [ 237.605603] nix-daemon[839]: accepted connection from pid 1954, user nix-grpc-daemon (trusted) worker2 # [ 237.623436] nix-daemon[839]: reaped child process 1957, status = succeeded lb # [ 237.624815] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 237.643757] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 237.645639] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.258Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 237.662644] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 237.676489] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls worker2 # [ 237.691123] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.307Z level=INFO msg="Uploading 1 narinfos" lb # [ 237.685011] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign lb # [ 237.693477] niks3-server[1137]: 2026/09/23 05:35:57 INFO Signed narinfos id=89 count=1 lb # [ 237.697942] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 237.708314] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete lb # [ 237.715066] niks3-server[1137]: 2026/09/23 05:35:57 INFO Completed upload id=89 lb # [ 237.717651] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 237.720859] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.337Z level=INFO msg="Upload complete. (279ms)" worker2 # [ 237.733290] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=1 worker1 # [ 237.808504] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:57Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 237.808789] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:57Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 237.755006] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 237.761694] nix-daemon[839]: reaped child process 1936, status = succeeded worker2 # [ 237.769000] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 237.781178] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.397Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 237.791601] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 237.807368] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 237.947482] nix-daemon[839]: accepted connection from pid 1961, user nix-grpc-daemon (trusted) worker2 # [ 237.963805] nix-daemon[839]: reaped child process 1964, status = succeeded lb # [ 237.964242] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 237.980087] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.596Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 237.981578] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.596Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 237.992268] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign worker2 # [ 238.006146] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.622Z level=INFO msg="Uploading 1 narinfos" lb # [ 238.006660] niks3-server[1137]: 2026/09/23 05:35:57 INFO Signed narinfos id=90 count=1 lb # [ 238.013579] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst lb # [ 238.024905] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls worker2 # [ 238.038047] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.654Z level=INFO msg="Upload complete. (219ms)" lb # [ 238.031765] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 238.038738] niks3-server[1137]: 2026/09/23 05:35:57 INFO Completed upload id=90 lb # [ 238.042950] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 238.044497] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 238.057866] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 238.066695] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 238.069110] nix-daemon[839]: reaped child process 1959, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 238.081545] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 239.306789] nix-daemon[839]: reaped child process 1968, status = succeeded worker2 # [ 239.431353] nix-daemon[839]: accepted connection from pid 1984, user nix-grpc-daemon (trusted) worker2 # [ 239.450458] nix-daemon[839]: reaped child process 1987, status = succeeded lb # [ 239.449327] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.470291] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 239.471989] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.084Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 239.493540] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign worker2 # [ 239.508768] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.124Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.509035] niks3-server[1137]: 2026/09/23 05:35:59 INFO Signed narinfos id=91 count=1 lb # [ 239.515055] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 239.525250] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker2 # [ 239.537445] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.153Z level=INFO msg="Upload complete. (246ms)" lb # [ 239.535857] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 239.542149] niks3-server[1137]: 2026/09/23 05:35:59 INFO Completed upload id=91 worker2 # [ 239.549384] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=1 lb # [ 239.545713] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo lb # [ 239.549841] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst worker2 # [ 239.566285] nix-daemon[839]: reaped child process 1966, status = succeeded worker1 # [ 239.635358] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:59Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker1 # [ 239.642199] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:59Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 239.611184] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 239.626223] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33100 lb # [ 239.634699] envoy[904]: conn peer=[2001:db8:1::1]:38154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30070 worker2 # [ 239.647384] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.263Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 239.658825] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 239.672118] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 239.834976] nix-daemon[839]: accepted connection from pid 1991, user nix-grpc-daemon (trusted) worker2 # [ 239.853599] nix-daemon[839]: reaped child process 1994, status = succeeded lb # [ 239.850182] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.865581] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 239.867074] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.481Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 239.879590] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 239.898335] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.514Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.894320] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign lb # [ 239.904241] niks3-server[1137]: 2026/09/23 05:35:59 INFO Signed narinfos id=92 count=1 worker2 # [ 239.918488] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.534Z level=INFO msg="Upload complete. (228ms)" lb # [ 239.913906] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 239.927093] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 239.922466] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 239.926201] niks3-server[1137]: 2026/09/23 05:35:59 INFO Completed upload id=92 lb # [ 239.928955] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 239.938956] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 worker2 # [ 239.947892] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 239.951019] nix-daemon[839]: reaped child process 1989, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 239.961508] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 241.193214] nix-daemon[839]: reaped child process 1998, status = succeeded worker2 # [ 241.295190] nix-daemon[839]: accepted connection from pid 2014, user nix-grpc-daemon (trusted) worker2 # [ 241.315889] nix-daemon[839]: reaped child process 2017, status = succeeded lb # [ 241.313952] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.332385] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.946Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 241.333823] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.946Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 241.356442] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 241.371445] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.987Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.369662] niks3-server[1137]: 2026/09/23 05:36:00 INFO Signed narinfos id=93 count=1 lb # [ 241.378578] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 241.387353] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 241.396756] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls worker2 # [ 241.408603] nix-grpc-daemon[1836]: time=2026-09-23T05:36:01.024Z level=INFO msg="Upload complete. (235ms)" lb # [ 241.402490] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 241.407024] niks3-server[1137]: 2026/09/23 05:36:01 INFO Completed upload id=93 lb # [ 241.413947] niks3-server[1137]: 2026/09/23 05:36:01 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 241.421060] nix-grpc-daemon[1775]: ts=2026-09-23T05:36:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=1 worker2 # [ 241.436761] nix-daemon[839]: reaped child process 1996, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.51 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 242.339560] nix-grpc-daemon[1696]: ts=2026-09-23T05:36:01Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.31 seconds) (finished: run the VM test script, in 243.87 seconds) test script finished in 244.11s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.03 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh