tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker2 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... worker1 # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 49) worker2: QEMU running (pid 52) client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 50) worker1: QEMU running (pid 51) worker2 # Formatting '/build/vm-state-worker2/tmp.0uZclR9QTt', fmt=raw size=1073741824 worker1 # Formatting '/build/vm-state-worker1/tmp.pG58sVlxY7', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.BLazXudYBz', fmt=raw size=1073741824 worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # Discarding device blocks: 0/262144 done lb # Formatting '/build/vm-state-lb/tmp.uANvhdVuxc', fmt=raw size=1073741824 worker2 # Discarding device blocks: 0/262144 done worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Filesystem UUID: 6791fd92-16a0-4211-9718-3e507c5a93dc lb # Discarding device blocks: 0/262144 done worker2 # Filesystem UUID: 92c65040-f127-4aa2-8d3f-abf260cc7c64 worker1 # Superblock backups stored on blocks: lb # Creating filesystem with 262144 4k blocks and 65536 inodes client # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.48 seconds) worker2 # Superblock backups stored on blocks: worker1 # 32768, 98304, 163840, 229376 lb # Filesystem UUID: 403c5b88-34c7-4837-963b-b9ae6e158bd5 worker2 # 32768, 98304, 163840, 229376 worker1 # lb # Superblock backups stored on blocks: worker2 # worker1 # Allocating group tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done lb # worker2 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done lb # Allocating group tables: 0/8 done worker2 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done lb # Writing inode tables: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # lb # Creating journal (8192 blocks): done worker2 # worker1 # Virtualisation disk image created. lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Virtualisation disk image created. worker1 # Starting virtiofs daemons... lb # worker2 # Starting virtiofs daemons... worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Virtualisation disk image created. worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Starting virtiofs daemons... worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests client # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service client # Creating filesystem with 262144 4k blocks and 65536 inodes client: waiting for the VM to finish booting client # Filesystem UUID: 8e1e7877-f7b8-4126-bb0a-62d6c498188f lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests client # Superblock backups stored on blocks: lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests client # 32768, 98304, 163840, 229376 client # client # Allocating group tables: 0/8 done client # Writing inode tables: 0/8 done client # Creating journal (8192 blocks): done client # Writing superblocks and filesystem accounting information: 0/8 done client # client # Virtualisation disk image created. client # Starting virtiofs daemons... client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] KASLR enabled lb # [ 0.000000] random: crng init done lb # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Zone ranges: lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] DMA32 empty lb # [ 0.000000] Normal empty lb # [ 0.000000] Device empty lb # [ 0.000000] Movable zone start for each node lb # [ 0.000000] Early memory node ranges lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vyrpk0yvg81ch7n98c7n14mzchdldfk4-nixos-system-lb-test/init regInfo=/nix/store/56sfh65ns0h9zv3wlb18ccjn63ky2v3j-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/56sfh65ns0h9zv3wlb18ccjn63ky2v3j-closure-info/registration", will be passed to user space. lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Policy zone: DMA lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] KASLR enabled worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] random: crng init done lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker2 # [ 0.000000] KASLR enabled client # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] random: crng init done client # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] Normal empty worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty client # [ 0.000000] Device empty worker2 # [ 0.000000] Normal empty client # [ 0.000000] Movable zone start for each node lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] Device empty client # [ 0.000000] Early memory node ranges lb # [ 0.000034] arm-pv: using stolen time PV worker2 # [ 0.000000] Movable zone start for each node client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Early memory node ranges lb # [ 0.000653] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000817] Console: colour dummy device 80x25 client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000824] printk: legacy console [tty0] enabled client # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.001011] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.001018] pid_max: default: 32768 minimum: 301 client # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] KASLR enabled client # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.001088] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.001258] landlock: Up and running. worker1 # [ 0.000000] random: crng init done worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.001261] Yama: becoming mindful. worker1 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] efi: UEFI not found. lb # [ 0.001921] LSM support for eBPF active worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.002086] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.002118] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.003947] rcu: Hierarchical SRCU implementation. worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] Zone ranges: worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.003952] rcu: Max phase no-delay instances is 1000. client # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005186] fsl-mc MSI: its@8080000 domain created client # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.005283] EFI services will not be available. worker1 # [ 0.000000] Normal empty client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] Device empty lb # [ 0.005372] smp: Bringing up secondary CPUs ... lb # [ 0.005381] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] Early memory node ranges lb # [ 0.005383] SMP: Total of 1 processors activated. client # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.005389] CPU: All CPU(s) started at EL1 worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.005399] CPU features: detected: Branch Target Identification worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005404] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.005407] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.005410] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test/init regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.005414] CPU features: detected: Common not Private translations worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.005417] CPU features: detected: CRC32 instructions client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space. lb # [ 0.005420] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration", will be passed to user space. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.005424] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.005427] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.005430] CPU features: detected: E0PD worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.005432] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.005435] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.005439] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.005442] CPU features: detected: Fine Grained Traps client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.005446] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.005451] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.005453] CPU features: detected: LSE atomic instructions worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.005456] CPU features: detected: Privileged Access Never worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.005459] CPU features: detected: PMUv3 client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.005462] CPU features: detected: RAS Extension Support worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test/init regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.005465] CPU features: detected: RASv1p1 Extension Support client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.005467] CPU features: detected: Random Number Generator client # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.005470] CPU features: detected: Speculation barrier (SB) client # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.005473] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.005476] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.005480] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.005521] alternatives: applying system-wide alternatives worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] software IO TLB: area num 1. lb # [ 0.008518] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.008682] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.009075] devtmpfs: initialized worker1 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.010952] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.010980] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.011155] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.011160] 0 pages in range for non-PLT usage client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.011161] 508272 pages in range for PLT usage client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.011267] pinctrl core: initialized pinctrl subsystem client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.012025] DMI not present or invalid. client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.015725] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.017943] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.018116] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.018321] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.018342] audit: initializing netlink subsys (disabled) worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.018922] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 lb # [ 0.018923] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.018927] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.018930] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.018935] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.018960] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.018969] cpuidle: using governor ladder client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.018975] cpuidle: using governor menu worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.019164] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000029] arm-pv: using stolen time PV lb # [ 0.019181] ASID allocator initialised with 65536 entries worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.020439] Serial: AMBA PL011 UART driver worker2 # [ 0.000030] arm-pv: using stolen time PV worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000443] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.025972] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000587] Console: colour dummy device 80x25 worker2 # [ 0.000510] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.026104] printk: console [ttyAMA0] enabled client # [ 0.000594] printk: legacy console [tty0] enabled worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000676] Console: colour dummy device 80x25 worker2 # [ 0.000684] printk: legacy console [tty0] enabled worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000803] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.000810] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000876] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000889] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000883] pid_max: default: 32768 minimum: 301 client # [ 0.001041] landlock: Up and running. worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.001043] Yama: becoming mindful. worker2 # [ 0.000952] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.001537] LSM support for eBPF active worker2 # [ 0.001090] landlock: Up and running. worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.001093] Yama: becoming mindful. client # [ 0.001697] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.001535] LSM support for eBPF active worker2 # [ 0.001690] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.001731] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.003565] rcu: Hierarchical SRCU implementation. worker2 # [ 0.001723] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.003570] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.003508] rcu: Hierarchical SRCU implementation. client # [ 0.004799] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.003512] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.004889] EFI services will not be available. lb # [ 0.145270] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.004746] fsl-mc MSI: its@8080000 domain created client # [ 0.004966] smp: Bringing up secondary CPUs ... lb # [ 0.145288] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.004836] EFI services will not be available. worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.004974] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000028] arm-pv: using stolen time PV worker2 # [ 0.004910] smp: Bringing up secondary CPUs ... lb # [ 0.145293] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.004977] SMP: Total of 1 processors activated. worker2 # [ 0.004918] smp: Brought up 1 node, 1 CPU client # [ 0.004982] CPU: All CPU(s) started at EL1 lb # [ 0.145297] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.004921] SMP: Total of 1 processors activated. worker1 # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.004992] CPU features: detected: Branch Target Identification worker2 # [ 0.004926] CPU: All CPU(s) started at EL1 lb # [ 0.145302] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.000556] Console: colour dummy device 80x25 client # [ 0.004997] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004936] CPU features: detected: Branch Target Identification lb # [ 0.145306] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.000563] printk: legacy console [tty0] enabled worker2 # [ 0.004940] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.145311] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.005000] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.000755] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.145315] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.000761] pid_max: default: 32768 minimum: 301 worker2 # [ 0.004943] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.005003] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.000839] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.005007] CPU features: detected: Common not Private translations worker2 # [ 0.004947] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.001016] landlock: Up and running. client # [ 0.005010] CPU features: detected: CRC32 instructions worker1 # [ 0.001019] Yama: becoming mindful. worker2 # [ 0.004951] CPU features: detected: Common not Private translations lb # [ 0.152410] fbcon: Taking over console worker1 # [ 0.001445] LSM support for eBPF active client # [ 0.005013] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004955] CPU features: detected: CRC32 instructions lb # [ 0.152423] ACPI: Interpreter disabled. worker1 # [ 0.001584] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005017] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.004958] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.001615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005019] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004961] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.005023] CPU features: detected: E0PD worker1 # [ 0.003436] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004964] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.003440] rcu: Max phase no-delay instances is 1000. client # [ 0.005025] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004967] CPU features: detected: E0PD worker1 # [ 0.004677] fsl-mc MSI: its@8080000 domain created client # [ 0.005028] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004970] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.004767] EFI services will not be available. client # [ 0.005032] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.004829] smp: Bringing up secondary CPUs ... worker2 # [ 0.004973] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005035] CPU features: detected: Fine Grained Traps worker1 # [ 0.004836] smp: Brought up 1 node, 1 CPU worker2 # [ 0.004977] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.004839] SMP: Total of 1 processors activated. worker2 # [ 0.004980] CPU features: detected: Fine Grained Traps worker1 # [ 0.004844] CPU: All CPU(s) started at EL1 client # [ 0.005038] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005043] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.004854] CPU features: detected: Branch Target Identification worker2 # [ 0.004983] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005046] CPU features: detected: LSE atomic instructions worker1 # [ 0.004858] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004988] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005049] CPU features: detected: Privileged Access Never worker2 # [ 0.004991] CPU features: detected: LSE atomic instructions lb # [ 0.161297] iommu: Default domain type: Translated client # [ 0.005052] CPU features: detected: PMUv3 worker1 # [ 0.004862] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004993] CPU features: detected: Privileged Access Never client # [ 0.005055] CPU features: detected: RAS Extension Support lb # [ 0.161307] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.004996] CPU features: detected: PMUv3 worker1 # [ 0.004865] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.161676] SCSI subsystem initialized client # [ 0.005057] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.004999] CPU features: detected: RAS Extension Support worker1 # [ 0.004869] CPU features: detected: Common not Private translations client # [ 0.005060] CPU features: detected: Random Number Generator worker2 # [ 0.005001] CPU features: detected: RASv1p1 Extension Support lb # [ 0.163575] usbcore: registered new interface driver usbfs worker1 # [ 0.004872] CPU features: detected: CRC32 instructions client # [ 0.005063] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.005004] CPU features: detected: Random Number Generator lb # [ 0.163606] usbcore: registered new interface driver hub client # [ 0.005065] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.004875] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.005007] CPU features: detected: Speculation barrier (SB) lb # [ 0.163620] usbcore: registered new device driver usb client # [ 0.005068] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.004878] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.163895] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.005009] CPU features: detected: Stage-2 Force Write-Back client # [ 0.005073] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.004882] CPU features: detected: Data independent timing control (DIT) lb # [ 0.163902] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.004885] CPU features: detected: E0PD lb # [ 0.163918] PTP clock support registered lb # [ 0.163967] EDAC MC: Ver: 3.0.0 worker1 # [ 0.004887] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.168428] scmi_core: SCMI protocol bus registered worker1 # [ 0.004891] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.004904] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.169408] FPGA manager framework worker1 # [ 0.004907] CPU features: detected: Fine Grained Traps lb # [ 0.170342] vgaarb: loaded worker1 # [ 0.004911] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.170956] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.004915] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.004918] CPU features: detected: LSE atomic instructions worker1 # [ 0.004921] CPU features: detected: Privileged Access Never worker1 # [ 0.004924] CPU features: detected: PMUv3 lb # [ 0.173881] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.004927] CPU features: detected: RAS Extension Support lb # [ 0.173915] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.005012] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.004929] CPU features: detected: RASv1p1 Extension Support client # [ 0.005110] alternatives: applying system-wide alternatives worker2 # [ 0.005017] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.004932] CPU features: detected: Random Number Generator client # [ 0.007995] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005055] alternatives: applying system-wide alternatives worker1 # [ 0.004935] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.004937] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.007944] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.177530] netfs: FS-Cache loaded client # [ 0.008153] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker1 # [ 0.004940] CPU features: detected: TLB range maintenance instructions lb # [ 0.177648] pnp: PnP ACPI: disabled client # [ 0.008524] devtmpfs: initialized worker1 # [ 0.004944] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.008178] Memory: 1405712K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) client # [ 0.010310] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.008539] devtmpfs: initialized worker1 # [ 0.004982] alternatives: applying system-wide alternatives client # [ 0.010331] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.010252] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.007819] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.010506] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.010273] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.010510] 0 pages in range for non-PLT usage worker2 # [ 0.010449] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.010511] 508272 pages in range for PLT usage worker1 # [ 0.007973] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker1 # [ 0.008368] devtmpfs: initialized worker2 # [ 0.010453] 0 pages in range for non-PLT usage client # [ 0.010621] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.010455] 508272 pages in range for PLT usage client # [ 0.011355] DMI not present or invalid. worker1 # [ 0.010220] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.181317] NET: Registered PF_INET protocol family worker2 # [ 0.010560] pinctrl core: initialized pinctrl subsystem client # [ 0.014540] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.011355] DMI not present or invalid. lb # [ 0.181573] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.010242] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.016722] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.014508] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.010409] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.016897] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.010413] 0 pages in range for non-PLT usage worker2 # [ 0.016676] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.010414] 508272 pages in range for PLT usage client # [ 0.017092] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.016868] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.010524] pinctrl core: initialized pinctrl subsystem client # [ 0.017113] audit: initializing netlink subsys (disabled) worker1 # [ 0.011311] DMI not present or invalid. worker2 # [ 0.017064] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.017673] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.014415] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.017085] audit: initializing netlink subsys (disabled) client # [ 0.017675] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.016550] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.017651] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.017679] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.017652] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.016718] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.017682] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.017656] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.016917] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.017687] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.017659] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.016936] audit: initializing netlink subsys (disabled) client # [ 0.017711] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.017663] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.017451] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.017719] cpuidle: using governor ladder client # [ 0.017725] cpuidle: using governor menu worker2 # [ 0.017687] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.017453] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.017695] cpuidle: using governor ladder client # [ 0.017910] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.017457] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.017701] cpuidle: using governor menu client # [ 0.017925] ASID allocator initialised with 65536 entries worker1 # [ 0.017460] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.017882] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.019117] Serial: AMBA PL011 UART driver worker1 # [ 0.017465] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.017895] ASID allocator initialised with 65536 entries client # [ 0.024307] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.019120] Serial: AMBA PL011 UART driver worker1 # [ 0.017487] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.024420] printk: console [ttyAMA0] enabled worker1 # [ 0.017496] cpuidle: using governor ladder worker2 # [ 0.024303] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.017502] cpuidle: using governor menu worker2 # [ 0.024454] printk: console [ttyAMA0] enabled client # [ 0.143867] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.017682] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.143884] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.017695] ASID allocator initialised with 65536 entries worker2 # [ 0.144376] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.018867] Serial: AMBA PL011 UART driver client # [ 0.143890] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.144392] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.143894] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.023999] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.144397] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.024136] printk: console [ttyAMA0] enabled client # [ 0.143898] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.144402] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.143903] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.144407] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.143907] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.144411] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.143911] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.144415] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.144420] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.151300] fbcon: Taking over console client # [ 0.151313] ACPI: Interpreter disabled. worker2 # [ 0.151744] fbcon: Taking over console worker2 # [ 0.151756] ACPI: Interpreter disabled. worker1 # [ 0.142293] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.142308] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.142314] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.142318] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.142323] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.158371] iommu: Default domain type: Translated worker1 # [ 0.142327] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.158380] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.142332] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.160057] SCSI subsystem initialized worker1 # [ 0.142336] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.159711] iommu: Default domain type: Translated client # [ 0.160736] usbcore: registered new interface driver usbfs worker2 # [ 0.159719] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.160768] usbcore: registered new interface driver hub worker2 # [ 0.160075] SCSI subsystem initialized worker1 # [ 0.149759] fbcon: Taking over console client # [ 0.160784] usbcore: registered new device driver usb worker1 # [ 0.149773] ACPI: Interpreter disabled. client # [ 0.161054] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.162110] usbcore: registered new interface driver usbfs worker2 # [ 0.162141] usbcore: registered new interface driver hub client # [ 0.161060] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.162156] usbcore: registered new device driver usb client # [ 0.161071] PTP clock support registered client # [ 0.161128] EDAC MC: Ver: 3.0.0 worker2 # [ 0.162440] pps_core: LinuxPPS API ver. 1 registered client # [ 0.165785] scmi_core: SCMI protocol bus registered worker2 # [ 0.162446] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.209945] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.162455] PTP clock support registered client # [ 0.166758] FPGA manager framework worker2 # [ 0.162513] EDAC MC: Ver: 3.0.0 client # [ 0.167644] vgaarb: loaded lb # [ 0.209983] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.167195] scmi_core: SCMI protocol bus registered client # [ 0.168255] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.210008] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.168150] FPGA manager framework lb # [ 0.210084] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.169066] vgaarb: loaded lb # [ 0.210230] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.169689] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.158726] iommu: Default domain type: Translated lb # [ 0.210302] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.158736] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.210335] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.159072] SCSI subsystem initialized worker2 # [ 0.171745] VFS: Disk quotas dquot_6.6.0 lb # [ 0.210371] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.173138] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.171779] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.161085] usbcore: registered new interface driver usbfs lb # [ 0.210469] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.173179] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.161117] usbcore: registered new interface driver hub lb # [ 0.210487] NET: Registered PF_XDP protocol family lb # [ 0.210503] PCI: CLS 0 bytes, default 64 worker1 # [ 0.161133] usbcore: registered new device driver usb client # [ 0.174768] netfs: FS-Cache loaded client # [ 0.174882] pnp: PnP ACPI: disabled lb # [ 0.210720] Trying to unpack rootfs image as initramfs... worker1 # [ 0.161401] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.175358] netfs: FS-Cache loaded worker1 # [ 0.161407] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.175469] pnp: PnP ACPI: disabled worker1 # [ 0.161417] PTP clock support registered worker1 # [ 0.161467] EDAC MC: Ver: 3.0.0 worker1 # [ 0.166051] scmi_core: SCMI protocol bus registered worker1 # [ 0.167038] FPGA manager framework worker1 # [ 0.167926] vgaarb: loaded client # [ 0.178761] NET: Registered PF_INET protocol family worker1 # [ 0.168538] clocksource: Switched to clocksource arch_sys_counter client # [ 0.179017] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.228515] kvm [1]: HYP mode not available worker2 # [ 0.179258] NET: Registered PF_INET protocol family worker2 # [ 0.179508] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.174545] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.174585] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.176213] netfs: FS-Cache loaded worker1 # [ 0.176330] pnp: PnP ACPI: disabled worker1 # [ 0.180199] NET: Registered PF_INET protocol family worker1 # [ 0.180448] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.208042] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.208082] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.208107] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.208176] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.212161] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.212244] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.208265] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.208306] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.213940] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208334] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.213981] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208409] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.214079] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.208552] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.214099] NET: Registered PF_XDP protocol family client # [ 0.214118] PCI: CLS 0 bytes, default 64 worker2 # [ 0.208634] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.214343] Trying to unpack rootfs image as initramfs... worker2 # [ 0.208667] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208702] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208791] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.208857] NET: Registered PF_XDP protocol family worker2 # [ 0.208874] PCI: CLS 0 bytes, default 64 worker2 # [ 0.209089] Trying to unpack rootfs image as initramfs... worker1 # [ 0.209653] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.209700] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.209727] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.209800] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.224439] kvm [1]: HYP mode not available worker1 # [ 0.209943] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.210023] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.210058] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.227521] kvm [1]: HYP mode not available worker1 # [ 0.210093] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.210182] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.210224] NET: Registered PF_XDP protocol family worker1 # [ 0.210241] PCI: CLS 0 bytes, default 64 worker1 # [ 0.210491] Trying to unpack rootfs image as initramfs... worker1 # [ 0.226109] kvm [1]: HYP mode not available lb # [ 0.319452] Initialise system trusted keyrings lb # [ 0.320197] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.321440] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.322215] 9p: Installing v9fs 9p2000 file system support lb # [ 0.342826] Key type asymmetric registered lb # [ 0.342849] Asymmetric key parser 'x509' registered lb # [ 0.342911] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.351024] io scheduler mq-deadline registered lb # [ 0.351044] io scheduler kyber registered client # [ 0.316826] Initialise system trusted keyrings client # [ 0.317569] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.363104] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.318828] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.363719] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.364105] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.319598] 9p: Installing v9fs 9p2000 file system support lb # [ 0.364123] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.364135] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.322238] Initialise system trusted keyrings lb # [ 0.364144] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.322993] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.364172] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.324272] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.364197] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.364270] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.325064] 9p: Installing v9fs 9p2000 file system support lb # [ 0.364280] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.364287] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.364292] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.317053] Initialise system trusted keyrings lb # [ 0.364297] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.317800] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.364377] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.319051] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.364815] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.364998] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.319833] 9p: Installing v9fs 9p2000 file system support lb # [ 0.365015] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.365044] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.365061] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.365520] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.365700] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.365716] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.365745] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.366185] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.366364] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.366379] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.366409] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.366850] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.367053] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.367069] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.367099] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.367118] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.367599] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.367782] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.367812] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.368263] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.368447] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.348355] Key type asymmetric registered client # [ 0.348379] Asymmetric key parser 'x509' registered lb # [ 0.368476] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.348437] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.368859] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.350601] io scheduler mq-deadline registered lb # [ 0.369037] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.350611] io scheduler kyber registered lb # [ 0.369279] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.340508] Key type asymmetric registered lb # [ 0.369469] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.340528] Asymmetric key parser 'x509' registered lb # [ 0.369499] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.369949] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.353767] Key type asymmetric registered worker2 # [ 0.353789] Asymmetric key parser 'x509' registered lb # [ 0.370138] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.353859] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.370167] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.355985] io scheduler mq-deadline registered lb # [ 0.370620] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.355996] io scheduler kyber registered lb # [ 0.370809] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.370840] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.348653] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.360393] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.349695] io scheduler mq-deadline registered worker1 # [ 0.349707] io scheduler kyber registered client # [ 0.361795] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.362179] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.362196] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.362209] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.362218] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.362238] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.362274] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.419407] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.362349] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.419696] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.362358] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.419715] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.362364] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.419746] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.362369] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.420220] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.365816] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.362374] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.420405] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.366458] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.420422] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.362457] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.366847] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.420452] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.362902] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.366863] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.421025] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.363083] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.363099] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421038] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.366875] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.363129] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421043] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.366883] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.363144] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.421089] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.366904] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.363589] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.360672] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.421137] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.363766] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.361266] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.366928] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.363782] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421186] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.361622] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.367013] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.363810] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421235] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.367022] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.361639] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.367028] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.421283] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.361651] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.367033] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.383168] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.421331] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.361659] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.367039] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.383354] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.421389] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.383370] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.361679] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.367114] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.421437] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.383399] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.367566] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.361702] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.383860] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.421484] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.367753] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.361789] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.384043] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.367770] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421553] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.361799] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.384058] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421600] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.367801] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.361805] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.384087] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.367818] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.421622] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.361810] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.384105] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.421644] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.361815] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.368304] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.368489] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.421665] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.361880] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.368506] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421688] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.362318] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.368536] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421709] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.362507] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.421732] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.368990] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.362525] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.369174] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.421757] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.362557] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.369191] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421779] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.362574] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.369222] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421802] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.363034] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.421824] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.369676] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.363219] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.369885] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.421847] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.363236] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.369902] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421868] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.363267] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.369933] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421890] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.363716] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.369952] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.421912] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.363900] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.421933] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.370436] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.363917] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.421954] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.370626] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.363948] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.421976] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.370657] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.422003] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.364410] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.364617] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.371116] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.422013] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.371305] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.364634] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.422017] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.396654] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.422831] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.364665] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.371336] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.396860] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.364685] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.371741] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.396890] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.365146] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.371924] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.397356] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.365335] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.458425] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 34767 usecs worker2 # [ 0.372207] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.397543] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.365367] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.372395] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.397573] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.372426] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.365840] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.397961] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.366028] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.372891] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.398143] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.366060] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.373079] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.398434] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.373109] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.366451] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.398621] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.366636] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.373565] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.398651] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.366914] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.399102] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.367102] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.399288] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.367133] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.399318] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.367582] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.399780] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.367770] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.399965] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.367802] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.399995] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.368262] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.400467] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.368449] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.400727] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.368480] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.400745] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.400774] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.401232] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.417808] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.401423] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.417851] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.401439] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.418350] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.401469] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.418634] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.402031] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.418653] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402042] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.418684] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.402048] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.419150] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.419335] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.402093] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.419352] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402139] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.419383] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.402185] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.419974] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.402231] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.419986] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.402278] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.419991] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.402323] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.417070] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.420045] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.417378] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.402369] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.420095] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.417397] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.485173] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.402416] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.420143] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.417428] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.402463] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.420193] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.417895] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.418080] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.402531] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.420242] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.418097] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402577] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.420290] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.418128] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.402598] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.420338] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.418702] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.402619] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.420386] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.418714] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.402640] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.420434] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.402661] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.418720] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.402682] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.420508] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.418766] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.402703] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.420554] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.418815] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.402728] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.420577] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.488224] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.418865] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.402750] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.420600] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.490166] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.418915] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.420622] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.402771] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.492158] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.420644] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.418965] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.494208] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.402793] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.420666] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.402816] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.419015] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.402837] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.420688] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.419065] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.402858] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.420714] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.419114] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.402879] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.420736] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.402900] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.419163] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.420758] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.402921] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.420781] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.419232] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.402942] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.420804] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.419279] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.402967] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.420825] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.402976] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.419302] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.420847] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.402981] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.419325] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.420869] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.403795] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.419348] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.420890] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.419371] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.420912] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.419394] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.420934] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.420961] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.419417] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.420971] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.419444] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.420976] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.419467] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.419491] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.419514] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.419538] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.419561] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.419584] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.460400] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55283 usecs lb # [ 0.512587] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.419606] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.514353] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.419629] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.419651] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.419674] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.419701] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.419711] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.517687] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.419716] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.420517] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.465846] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.456926] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 35562 usecs lb # [ 0.524293] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.526694] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.481151] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.483369] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.536901] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.486943] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.488954] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.482581] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.494421] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.549679] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.499980] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.551284] msm_serial: driver initialized lb # [ 0.551431] SuperH (H)SCI(F) driver initialized lb # [ 0.551486] STM32 USART driver initialized client # [ 0.500492] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.488765] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.502489] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.490788] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.505870] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.507945] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.512416] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.514161] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.515845] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.500716] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.502731] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.505157] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.517880] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.519600] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.506826] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.521379] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.526350] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.516614] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.584205] loop: module loaded lb # [ 0.584375] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.518436] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.531612] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.520209] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.585203] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.532361] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.534728] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.534532] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.536772] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.591543] megasas: 07.734.00.00-rc1 lb # [ 0.592239] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.530761] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.547487] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.549883] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.550045] msm_serial: driver initialized client # [ 0.550192] SuperH (H)SCI(F) driver initialized client # [ 0.550245] STM32 USART driver initialized lb # [ 0.602992] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.603017] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.604587] Using buffer write method lb # [ 0.604649] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.607552] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.547877] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.607575] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.608886] Using buffer write method lb # [ 0.608911] Concatenating MTD devices: lb # [ 0.608915] (0): "0.flash" worker2 # [ 0.558966] msm_serial: driver initialized lb # [ 0.608927] (1): "0.flash" lb # [ 0.608930] into device "0.flash" worker2 # [ 0.559117] SuperH (H)SCI(F) driver initialized worker2 # [ 0.559172] STM32 USART driver initialized worker1 # [ 0.550547] msm_serial: driver initialized worker1 # [ 0.550692] SuperH (H)SCI(F) driver initialized worker1 # [ 0.550751] STM32 USART driver initialized client # [ 0.585243] loop: module loaded client # [ 0.585410] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.586142] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.588458] loop: module loaded worker2 # [ 0.588621] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.589345] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.583636] loop: module loaded worker1 # [ 0.583809] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.594256] megasas: 07.734.00.00-rc1 worker1 # [ 0.584657] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.594947] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.596905] megasas: 07.734.00.00-rc1 worker1 # [ 0.589155] megasas: 07.734.00.00-rc1 client # [ 0.597627] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.589891] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.599749] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.599774] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.601277] Using buffer write method client # [ 0.601352] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.602976] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.603028] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.605730] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.605754] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.607341] Using buffer write method worker2 # [ 0.607418] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.609302] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.609326] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.610859] Using buffer write method worker2 # [ 0.610878] Concatenating MTD devices: worker2 # [ 0.610882] (0): "0.flash" worker2 # [ 0.610886] (1): "0.flash" worker1 # [ 0.601591] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.601624] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.610897] into device "0.flash" worker1 # [ 0.603144] Using buffer write method worker1 # [ 0.603208] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.621897] Using buffer write method client # [ 0.621932] Concatenating MTD devices: client # [ 0.621936] (0): "0.flash" client # [ 0.621946] (1): "0.flash" client # [ 0.621949] into device "0.flash" worker1 # [ 0.618589] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.618620] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.620495] Using buffer write method worker1 # [ 0.620523] Concatenating MTD devices: worker1 # [ 0.620527] (0): "0.flash" worker1 # [ 0.623581] (1): "0.flash" worker1 # [ 0.623592] into device "0.flash" lb # [ 0.856301] Freeing initrd memory: 26948K lb # [ 0.862300] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.865945] thunder_xcv, ver 1.0 lb # [ 0.865988] thunder_bgx, ver 1.0 lb # [ 0.866009] nicpf, ver 1.0 lb # [ 0.866579] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.866586] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.866609] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.866617] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.866646] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.866652] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.866675] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.866681] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.866815] sky2: driver version 1.30 lb # [ 0.875648] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.875680] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.876005] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.878724] usbcore: registered new interface driver usb-storage lb # [ 0.878829] usbcore: registered new interface driver usbserial_generic lb # [ 0.878843] usbserial: USB Serial support registered for generic lb # [ 0.881710] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.883237] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.883260] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573) lb # [ 0.883558] i2c_dev: i2c /dev entries driver lb # [ 0.888825] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.888835] sdhci: Copyright(c) Pierre Ossman lb # [ 0.889100] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.889474] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.893626] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.893992] hub 1-0:1.0: USB hub found lb # [ 0.894011] hub 1-0:1.0: 6 ports detected lb # [ 0.896087] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.896331] usbcore: registered new interface driver usbhid lb # [ 0.896337] usbhid: USB HID core driver lb # [ 0.899281] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.900820] drop_monitor: Initializing network drop monitor service lb # [ 0.900949] NET: Registered PF_INET6 protocol family lb # [ 0.903937] Segment Routing with IPv6 lb # [ 0.903968] In-situ OAM (IOAM) with IPv6 lb # [ 0.904014] NET: Registered PF_PACKET protocol family lb # [ 0.905687] 9pnet: Installing 9P2000 support lb # [ 0.905737] Key type dns_resolver registered client # [ 0.855319] Freeing initrd memory: 26972K worker2 # [ 0.858292] Freeing initrd memory: 26944K worker1 # [ 0.847211] Freeing initrd memory: 26976K client # [ 0.861546] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.912610] registered taskstats version 1 lb # [ 0.912770] Loading compiled-in X.509 certificates worker1 # [ 0.853150] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.864417] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.865262] thunder_xcv, ver 1.0 lb # [ 0.921593] Demotion targets for Node 0: null client # [ 0.865306] thunder_bgx, ver 1.0 lb # [ 0.921692] Key type .fscrypt registered client # [ 0.865329] nicpf, ver 1.0 worker2 # [ 0.868012] thunder_xcv, ver 1.0 lb # [ 0.921699] Key type fscrypt-provisioning registered client # [ 0.865897] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.868066] thunder_bgx, ver 1.0 worker2 # [ 0.868088] nicpf, ver 1.0 lb # [ 0.921801] ima: No TPM chip found, activating TPM-bypass! client # [ 0.865904] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.868625] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.921820] ima: Allocated hash algorithm: sha1 client # [ 0.865928] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.921840] ima: No architecture policies found worker2 # [ 0.868632] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.865935] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.868655] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.856832] thunder_xcv, ver 1.0 client # [ 0.865964] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.868662] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.856871] thunder_bgx, ver 1.0 lb # [ 0.925780] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.865969] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.856903] nicpf, ver 1.0 worker2 # [ 0.868692] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.857498] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.865996] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.868697] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.857505] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.866002] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.868719] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.866135] sky2: driver version 1.30 worker1 # [ 0.857531] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.868725] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.867705] usbcore: registered new interface driver usb-storage worker1 # [ 0.857539] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.868854] sky2: driver version 1.30 worker1 # [ 0.857570] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.867754] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.857576] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.867768] usbserial: USB Serial support registered for generic client # [ 0.869016] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.857605] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.857611] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.869043] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.877394] usbcore: registered new interface driver usb-storage worker1 # [ 0.857753] sky2: driver version 1.30 client # [ 0.869316] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.877513] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.859434] usbcore: registered new interface driver usb-storage worker2 # [ 0.877529] usbserial: USB Serial support registered for generic worker1 # [ 0.859488] usbcore: registered new interface driver usbserial_generic client # [ 0.880808] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.878391] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.881096] hub 1-0:1.0: USB hub found worker1 # [ 0.859503] usbserial: USB Serial support registered for generic client # [ 0.881111] hub 1-0:1.0: 6 ports detected worker2 # [ 0.878416] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.860135] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.878635] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.861213] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.883218] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.861239] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.883706] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.861425] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.884815] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.885247] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.873224] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.884843] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573) worker1 # [ 0.873543] hub 1-0:1.0: USB hub found client # [ 0.885150] i2c_dev: i2c /dev entries driver worker1 # [ 0.873560] hub 1-0:1.0: 6 ports detected worker2 # [ 0.885274] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573) worker2 # [ 0.885569] i2c_dev: i2c /dev entries driver worker1 # [ 0.876739] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.876765] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573) worker1 # [ 0.877145] i2c_dev: i2c /dev entries driver client # [ 0.890363] sdhci: Secure Digital Host Controller Interface driver client # [ 0.890373] sdhci: Copyright(c) Pierre Ossman lb # [ 0.944140] clk: Disabling unused clocks client # [ 0.890638] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.944163] PM: genpd: Disabling unused power domains client # [ 0.891020] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.890731] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.890741] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.891007] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.948286] Freeing unused kernel memory: 4736K client # [ 0.895225] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.882327] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.891369] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.948490] Run /init as init process client # [ 0.895497] usbcore: registered new interface driver usbhid worker1 # [ 0.882338] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.892858] hid: raw HID events driver (C) Jiri Kosina client # [ 0.895503] usbhid: USB HID core driver worker1 # [ 0.882595] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.893104] usbcore: registered new interface driver usbhid worker2 # [ 0.893110] usbhid: USB HID core driver worker1 # [ 0.882968] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.884475] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.898257] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.898569] hub 1-0:1.0: USB hub found worker1 # [ 0.887980] usbcore: registered new interface driver usbhid worker2 # [ 0.898589] hub 1-0:1.0: 6 ports detected client # [ 0.898348] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.887991] usbhid: USB HID core driver client # [ 0.899852] drop_monitor: Initializing network drop monitor service client # [ 0.900000] NET: Registered PF_INET6 protocol family worker2 # [ 0.901080] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.903058] Segment Routing with IPv6 client # [ 0.903088] In-situ OAM (IOAM) with IPv6 client # [ 0.903138] NET: Registered PF_PACKET protocol family worker1 # [ 0.890253] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.903663] drop_monitor: Initializing network drop monitor service worker1 # [ 0.891735] drop_monitor: Initializing network drop monitor service worker2 # [ 0.903829] NET: Registered PF_INET6 protocol family worker1 # [ 0.891893] NET: Registered PF_INET6 protocol family worker2 # [ 0.905755] Segment Routing with IPv6 worker1 # [ 0.895004] Segment Routing with IPv6 worker2 # [ 0.905771] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.895021] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.905799] NET: Registered PF_PACKET protocol family worker1 # [ 0.895048] NET: Registered PF_PACKET protocol family client # [ 0.905000] 9pnet: Installing 9P2000 support client # [ 0.905048] Key type dns_resolver registered worker2 # [ 0.907451] 9pnet: Installing 9P2000 support worker2 # [ 0.907501] Key type dns_resolver registered client # [ 0.911891] registered taskstats version 1 worker1 # [ 0.896881] 9pnet: Installing 9P2000 support lb # [ 0.965142] systemd[1]: Successfully made /usr/ read-only. client # [ 0.912034] Loading compiled-in X.509 certificates worker1 # [ 0.896933] Key type dns_resolver registered worker1 # [ 0.903667] registered taskstats version 1 worker1 # [ 0.903805] Loading compiled-in X.509 certificates worker2 # [ 0.914036] registered taskstats version 1 worker2 # [ 0.914172] Loading compiled-in X.509 certificates client # [ 0.920636] Demotion targets for Node 0: null client # [ 0.920744] Key type .fscrypt registered client # [ 0.920750] Key type fscrypt-provisioning registered client # [ 0.920841] ima: No TPM chip found, activating TPM-bypass! client # [ 0.920860] ima: Allocated hash algorithm: sha1 client # [ 0.920880] ima: No architecture policies found worker2 # [ 0.922815] Demotion targets for Node 0: null worker2 # [ 0.922916] Key type .fscrypt registered worker1 # [ 0.912274] Demotion targets for Node 0: null worker1 # [ 0.912366] Key type .fscrypt registered worker2 # [ 0.922922] Key type fscrypt-provisioning registered client # [ 0.925000] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.912372] Key type fscrypt-provisioning registered worker2 # [ 0.923013] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.912462] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.923036] ima: Allocated hash algorithm: sha1 worker1 # [ 0.912481] ima: Allocated hash algorithm: sha1 worker2 # [ 0.923056] ima: No architecture policies found worker1 # [ 0.912501] ima: No architecture policies found worker2 # [ 0.927067] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.916591] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.942870] clk: Disabling unused clocks client # [ 0.942893] PM: genpd: Disabling unused power domains worker2 # [ 0.945442] clk: Disabling unused clocks worker2 # [ 0.945464] PM: genpd: Disabling unused power domains client # [ 0.946979] Freeing unused kernel memory: 4736K worker1 # [ 0.934040] clk: Disabling unused clocks client # [ 0.947163] Run /init as init process worker1 # [ 0.934060] PM: genpd: Disabling unused power domains worker2 # [ 0.949557] Freeing unused kernel memory: 4736K worker1 # [ 0.938159] Freeing unused kernel memory: 4736K worker1 # [ 0.938372] Run /init as init process worker2 # [ 0.950327] Run /init as init process client # [ 0.962498] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.963834] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.954692] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.139035] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.128338] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.120611] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.145772] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.291383] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.299894] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.312118] systemd[1]: Detected virtualization qemu. lb # [ 1.312206] systemd[1]: Detected architecture arm64. lb # [ 1.312232] systemd[1]: Running in initrd. lb # [ 1.313144] systemd[1]: Initializing machine ID from random generator. lb # [ 1.313445] systemd[1]: Hostname set to . client # [ 1.280866] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.273133] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.298129] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.297063] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.309163] systemd[1]: Detected virtualization qemu. client # [ 1.311150] systemd[1]: Detected architecture arm64. worker1 # [ 1.289489] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.313115] systemd[1]: Running in initrd. worker1 # [ 1.301758] systemd[1]: Detected virtualization qemu. worker1 # [ 1.303738] systemd[1]: Detected architecture arm64. worker2 # [ 1.304059] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.315709] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.305718] systemd[1]: Running in initrd. worker2 # [ 1.304117] systemd[1]: Detected virtualization qemu. client # [ 1.318699] systemd[1]: Hostname set to . worker2 # [ 1.304186] systemd[1]: Detected architecture arm64. worker1 # [ 1.308306] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.304211] systemd[1]: Running in initrd. worker1 # [ 1.311336] systemd[1]: Hostname set to . worker2 # [ 1.305097] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.305394] systemd[1]: Hostname set to . lb # [ 1.379203] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.368547] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.360783] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.381953] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.499035] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.488314] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.484586] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.501747] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.624303] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.655782] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.661404] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.617277] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.613055] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.630699] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.652319] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.641137] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.657859] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.646886] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.666615] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.673981] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.743823] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.753408] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.754623] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.756106] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.757201] systemd[1]: Reached target Path Units. lb # [ 1.758038] systemd[1]: Reached target Slice Units. lb # [ 1.758876] systemd[1]: Reached target Swaps. lb # [ 1.759701] systemd[1]: Reached target Timer Units. lb # [ 1.760731] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.761987] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.763268] systemd[1]: Listening on Journal Sockets. lb # [ 1.763416] systemd[1]: Listening on udev Control Socket. lb # [ 1.763552] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.763579] systemd[1]: Reached target Socket Units. lb # [ 1.768745] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.769921] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.779149] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.787239] systemd[1]: Starting Journal Service... client # [ 1.736248] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.730188] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.746838] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.748067] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.747684] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.749578] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.750678] systemd[1]: Reached target Path Units. worker1 # [ 1.739003] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.751530] systemd[1]: Reached target Slice Units. client # [ 1.752433] systemd[1]: Reached target Swaps. worker1 # [ 1.740242] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.753242] systemd[1]: Reached target Timer Units. worker1 # [ 1.741873] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.754301] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.742991] systemd[1]: Reached target Path Units. worker1 # [ 1.743846] systemd[1]: Reached target Slice Units. client # [ 1.755594] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.744757] systemd[1]: Reached target Swaps. client # [ 1.756927] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.756256] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.745547] systemd[1]: Reached target Timer Units. client # [ 1.757969] systemd[1]: Listening on udev Control Socket. lb # [ 1.810952] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.746600] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.757532] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.759038] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.747858] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.811992] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.759973] systemd[1]: Reached target Socket Units. worker1 # [ 1.749097] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.750132] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.759087] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.751207] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.759126] systemd[1]: Reached target Path Units. worker2 # [ 1.759148] systemd[1]: Reached target Slice Units. worker1 # [ 1.752165] systemd[1]: Reached target Socket Units. client # [ 1.762627] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.759169] systemd[1]: Reached target Swaps. worker2 # [ 1.759191] systemd[1]: Reached target Timer Units. client # [ 1.762711] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.759393] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.759568] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.754867] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.759720] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.759867] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.754950] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.759958] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.759982] systemd[1]: Reached target Socket Units. worker2 # [ 1.772049] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.772431] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.773328] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.831221] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.764737] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.780553] systemd[1]: Starting Journal Service... lb # [ 1.836014] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.772817] systemd[1]: Starting Journal Service... worker2 # [ 1.784957] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.847104] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.848631] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.859356] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.814074] systemd[1]: Starting Journal Service... worker2 # [ 1.816617] systemd[1]: Starting Load Kernel Modules... client # [ 1.816539] systemd[1]: Starting Load Kernel Modules... client # [ 1.816659] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.817569] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.820043] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.808711] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.808830] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.812153] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.828912] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.839656] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.831438] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.899609] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.845879] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.846620] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.902080] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.840674] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.852654] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.853733] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.858200] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.862272] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.850971] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.864381] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.918845] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.856902] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.857213] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.880420] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.872608] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.884369] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.931103] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.937188] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.937433] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.937443] [drm] features: -context_init lb # [ 1.938229] [drm] number of scanouts: 1 lb # [ 1.938248] [drm] number of cap sets: 0 client # [ 1.889536] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.889767] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.889777] [drm] features: -context_init client # [ 1.890483] [drm] number of scanouts: 1 client # [ 1.890501] [drm] number of cap sets: 0 worker1 # [ 1.886216] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.886453] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.886464] [drm] features: -context_init worker1 # [ 1.887168] [drm] number of scanouts: 1 worker1 # [ 1.887187] [drm] number of cap sets: 0 worker2 # [ 1.902761] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.955579] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.955922] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.955950] systemd[1]: Reached target Local File Systems. lb # [ 1.960019] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.967468] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.905449] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.967488] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.917949] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.916630] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.908916] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.920411] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.916649] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.921788] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.910047] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.910065] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 1.924529] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.927205] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.938883] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.939119] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.939129] [drm] features: -context_init worker2 # [ 1.939827] [drm] number of scanouts: 1 worker2 # [ 1.939844] [drm] number of cap sets: 0 lb # [ 1.995283] Console: switching to colour frame buffer device 160x50 client # [ 1.947464] Console: switching to colour frame buffer device 160x50 worker1 # [ 1.944848] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.958396] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.958413] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 2.027565] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.978114] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.032401] systemd[1]: Finished Load Kernel Modules. client # [ 1.983278] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.021914] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 1.985310] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.973960] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.987071] systemd[1]: Reached target Local File Systems. worker1 # [ 1.978202] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.043952] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.978415] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.978449] systemd[1]: Reached target Local File Systems. worker2 # [ 1.986022] Console: switching to colour frame buffer device 160x50 worker1 # [ 1.982539] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.992665] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.047225] systemd[1]: Started Journal Service. lb # [ 2.036226] systemd-modules-load[74]: Module 'virtio_balloon' is built in client # [ 1.996845] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.044345] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.006381] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.006777] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 2.006811] systemd[1]: Reached target Local File Systems. lb # [ 2.048346] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.010830] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.012550] systemd[1]: Finished Load Kernel Modules. lb # [ 2.052382] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.000719] systemd[1]: Finished Load Kernel Modules. client # [ 2.020831] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.009280] systemd[1]: Starting Apply Kernel Variables... lb # [ 2.064083] systemd[1]: Starting Create System Files and Directories... lb # [ 2.066655] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.033989] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.047049] systemd[1]: Starting Apply Kernel Variables... client # [ 2.044719] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 2.037457] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.048576] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.052835] systemd[1]: Started Journal Service. worker2 # [ 2.064275] systemd[1]: Started Journal Service. client # [ 2.049788] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.050940] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.105400] systemd[1]: Finished Create System Files and Directories. client # [ 2.051980] systemd-modules-load[73]: Module 'virtio_rng' is built in client # [ 2.062163] systemd[1]: Started Journal Service. lb # [ 2.112219] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.052312] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.053527] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.064297] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 2.054648] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.055700] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.066615] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker2 # [ 2.067766] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.072502] systemd[1]: Starting Create System Files and Directories... lb # [ 2.128876] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.080489] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.081576] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.082838] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.072968] systemd[1]: Starting Create System Files and Directories... client # [ 2.085287] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.080089] systemd[1]: Finished Apply Kernel Variables. client # [ 2.103450] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.097699] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.108596] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker2 # [ 2.113611] systemd[1]: Finished Apply Kernel Variables. client # [ 2.120618] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.121087] systemd[1]: Finished Create System Files and Directories. lb # [ 2.187860] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.140365] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.139094] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.158641] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.157819] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.240541] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.244081] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.214113] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.211389] systemd[1]: Starting Virtual Console Setup... client # [ 2.220884] systemd[1]: Starting Virtual Console Setup... client # [ 2.272545] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.275991] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.272489] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.284553] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.275887] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.288005] systemd[1]: Finished Virtual Console Setup. lb # [ 2.939814] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.940882] systemd[1]: Reached target System Initialization. lb # [ 2.941671] systemd[1]: Reached target Basic System. worker2 # [ 2.963637] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.964664] systemd[1]: Reached target System Initialization. worker2 # [ 2.965471] systemd[1]: Reached target Basic System. worker1 # [ 2.970240] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.971212] systemd[1]: Reached target System Initialization. worker1 # [ 2.976082] systemd[1]: Reached target Basic System. client # [ 3.009511] systemd[1]: Finished Coldplug All udev Devices. client # [ 3.011149] systemd[1]: Reached target System Initialization. client # [ 3.012006] systemd[1]: Reached target Basic System. lb # [ 3.107692] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.137345] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.143643] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.111837] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.151474] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.165727] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.170914] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.227810] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.233685] systemd[1]: Reached target Initrd Root Device. lb # [ 3.240148] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.197272] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.202283] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.218104] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.224335] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.228110] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.228144] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.230630] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.234909] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.291719] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.300093] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.308367] systemd[1]: Mounting /sysroot... worker1 # [ 3.260474] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.263916] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.268798] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.293014] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.358909] EXT4-fs (vda): mounted filesystem 403c5b88-34c7-4837-963b-b9ae6e158bd5 r/w with ordered data mode. Quota mode: none. lb # [ 3.349395] systemd[1]: Mounted /sysroot. worker2 # [ 3.300089] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.355286] systemd[1]: Reached target Initrd Root File System. lb # [ 3.358163] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.305509] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.311429] systemd[1]: Mounting /sysroot... client # [ 3.315093] systemd[1]: Reached target Initrd Root Device. client # [ 3.319429] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.326974] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.392435] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.357475] EXT4-fs (vda): mounted filesystem 92c65040-f127-4aa2-8d3f-abf260cc7c64 r/w with ordered data mode. Quota mode: none. lb # [ 3.397167] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.336131] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.401264] systemd[1]: Reloading... worker2 # [ 3.353195] systemd[1]: Mounted /sysroot. worker2 # [ 3.355230] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.350300] systemd[1]: Mounting /sysroot... worker2 # [ 3.361024] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.371037] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.379138] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.381455] systemd[1]: Mounting /sysroot... worker2 # [ 3.383926] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.390149] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.396341] EXT4-fs (vda): mounted filesystem 6791fd92-16a0-4211-9718-3e507c5a93dc r/w with ordered data mode. Quota mode: none. worker2 # [ 3.396722] systemd[1]: Reloading... worker1 # [ 3.390402] systemd[1]: Mounted /sysroot. worker1 # [ 3.393025] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.398889] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.439520] EXT4-fs (vda): mounted filesystem 8e1e7877-f7b8-4126-bb0a-62d6c498188f r/w with ordered data mode. Quota mode: none. client # [ 3.429061] systemd[1]: Mounted /sysroot. client # [ 3.432689] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.427832] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.437143] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.434134] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.439616] systemd[1]: Reloading... client # [ 3.465849] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.470471] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.474825] systemd[1]: Reloading... lb # [ 3.650224] systemd[1]: Reloading finished in 250 ms. worker2 # [ 3.626228] systemd[1]: Reloading finished in 233 ms. lb # [ 3.686892] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.691999] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.698758] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.701487] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.703610] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.661574] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.666066] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker2 # [ 3.671232] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.664699] systemd[1]: Reloading finished in 227 ms. worker2 # [ 3.675082] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.677281] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.687807] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.692309] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker1 # [ 3.699339] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.709878] systemd[1]: Reloading finished in 236 ms. worker1 # [ 3.701890] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.704071] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.734637] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.740461] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.746369] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.749506] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.752246] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.838851] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.849356] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.812145] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.871500] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.809694] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.827404] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.819395] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.835225] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.838437] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.833784] systemd[1]: Mounting /sysroot/run... lb # [ 3.896474] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.842251] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.855205] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.861084] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.866066] systemd[1]: Mounting /sysroot/run... client # [ 3.866284] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.934532] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.882888] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.875656] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.958814] fuse: init (API version 7.45) worker2 # [ 3.895377] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.951570] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.958322] systemd[1]: Mounted /sysroot/run. lb # [ 3.980334] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.981232] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.927730] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.996703] virtiofs virtio7: discovered new tag: shared lb # [ 3.997480] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.936882] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 4.005478] virtiofs virtio8: discovered new tag: xchg lb # [ 4.006231] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 3.931258] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.959000] fuse: init (API version 7.45) lb # [ 4.005182] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.948847] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.976313] fuse: init (API version 7.45) worker1 # [ 3.971226] fuse: init (API version 7.45) client # [ 3.982433] virtiofs virtio6: discovered new tag: nix-store lb # [ 4.023884] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.983325] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.962527] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.985750] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.986643] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.974549] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.979028] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.979938] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 4.034525] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 3.998851] virtiofs virtio7: discovered new tag: shared worker2 # [ 3.986767] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.999638] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.042217] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.003291] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.004089] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 3.994075] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.994847] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.007144] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.995031] systemd[1]: Mounted /sysroot/run. client # [ 4.007897] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.052100] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.013035] virtiofs virtio8: discovered new tag: xchg lb # [ 4.053179] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.002505] virtiofs virtio8: discovered new tag: xchg worker1 # [ 4.003288] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.022461] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 4.001689] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.014237] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.012000] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 4.020199] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.022131] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 4.014121] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.023660] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.027339] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.017895] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.027452] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.030792] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 4.021171] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.046283] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.057531] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.059337] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.062889] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.710464] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.714792] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.672925] (udev-worker)[92]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.677511] (udev-worker)[92]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.752201] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.757052] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.760191] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.761101] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.718727] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.721341] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.723236] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.729039] systemd[1]: Starting Virtual Console Setup... lb # [ 4.785928] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.787463] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.725096] (udev-worker)[95]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.728267] (udev-worker)[95]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.743142] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.796309] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.744795] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.751678] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.763930] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.826167] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.766507] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.829189] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.767545] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.777559] systemd-vconsole-setup[155]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.780725] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.772307] systemd[1]: Starting Virtual Console Setup... lb # [ 4.835671] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.801129] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.795613] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.804200] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.799514] systemd[1]: Finished Virtual Console Setup. lb # [ 4.868751] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.816116] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.803958] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.871329] systemd[1]: Reached target Initrd File Systems. lb # [ 4.876310] systemd[1]: Starting Find NixOS closure... lb # [ 4.878465] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 4.836861] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.842790] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.843613] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.846089] systemd[1]: Starting Find NixOS closure... lb # [ 4.907866] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.909256] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.852242] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.919326] systemd[1]: Finished Find NixOS closure. client # [ 4.866470] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.920867] systemd[1]: Reached target Initrd Default Target. client # [ 4.867620] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.923426] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.870288] systemd[1]: Stopping Virtual Console Setup... client # [ 4.875877] systemd[1]: Starting Virtual Console Setup... client # [ 4.878990] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.881150] systemd[1]: Stopped Virtual Console Setup. client # [ 4.884451] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.888443] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.891185] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.892863] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.885618] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.894828] systemd[1]: Reached target Initrd File Systems. client # [ 4.897228] systemd[1]: Starting Find NixOS closure... lb # [ 4.952773] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.954220] systemd[1]: Stopped target Basic System. lb # [ 4.955287] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.887869] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.903602] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.904228] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.960568] systemd[1]: Stopped target Path Units. worker1 # [ 4.899187] systemd[1]: Finished Find NixOS closure. lb # [ 4.961316] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.962418] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.900809] systemd[1]: Reached target Initrd Default Target. lb # [ 4.963523] systemd[1]: Stopped target Slice Units. lb # [ 4.964830] systemd[1]: Stopped target Socket Units. worker1 # [ 4.903555] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.968199] systemd[1]: Stopped target System Initialization. lb # [ 4.969095] systemd[1]: Stopped target Swaps. lb # [ 4.969745] systemd[1]: Stopped target Timer Units. lb # [ 4.971563] systemd[1]: dbus.socket: Deactivated successfully. client # [ 4.922286] systemd[1]: Starting Virtual Console Setup... lb # [ 4.976813] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.980868] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.983056] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.935393] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.988165] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.989358] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.928450] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.929651] systemd[1]: Stopped target Basic System. lb # [ 4.992498] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 4.940295] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.932498] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.994636] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 4.933331] systemd[1]: Stopped target Path Units. client # [ 4.943447] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.997933] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 4.935664] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.998807] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 4.937322] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.999905] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 4.946057] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 5.001231] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.002235] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.003103] systemd[1]: Stopped target Local File Systems. lb # [ 5.003870] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 4.952421] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.943331] systemd[1]: Stopped target Slice Units. worker2 # [ 4.954732] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.956386] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 5.010285] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 4.949013] systemd[1]: Stopped target Socket Units. lb # [ 5.011350] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 4.949750] systemd[1]: Stopped target System Initialization. worker1 # [ 4.950556] systemd[1]: Stopped target Swaps. worker1 # [ 4.951188] systemd[1]: Stopped target Timer Units. worker1 # [ 4.951891] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.016668] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 4.963740] systemd[1]: Finished Find NixOS closure. lb # [ 5.017700] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.964644] systemd[1]: Reached target Initrd Default Target. lb # [ 5.018766] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.019490] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 4.958584] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.959441] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.964401] systemd[1]: Stopped Find NixOS closure. client # [ 4.971959] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.965246] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 5.027426] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.028655] systemd[1]: systemd-udevd.service: Consumed 1.626s CPU time over 3.052s wall clock time, 22.2M memory peak. worker1 # [ 4.967897] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.030816] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.031767] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 4.970226] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.032838] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 4.971019] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.033824] systemd[1]: Closed udev Control Socket. lb # [ 5.034538] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.982894] systemd[1]: Stopped target Initrd Default Target. lb # [ 5.035299] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.984684] systemd[1]: Stopped target Basic System. worker2 # [ 4.988886] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.989816] systemd[1]: Stopped target Path Units. worker1 # [ 4.980256] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.990735] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.981115] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.044217] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 4.990411] systemd-vconsole-setup[160]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 5.045307] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.993328] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 5.046561] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 4.984250] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 4.996445] systemd[1]: Stopped target Slice Units. worker1 # [ 4.987381] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 4.997971] systemd[1]: Finished Virtual Console Setup. worker2 # [ 5.000154] systemd[1]: Stopped target Socket Units. lb # [ 5.052251] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.000985] systemd[1]: Stopped target System Initialization. lb # [ 5.053262] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.001843] systemd[1]: Stopped target Swaps. lb # [ 5.054155] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.003032] systemd[1]: Stopped target Timer Units. lb # [ 5.055142] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.993582] systemd[1]: Stopped Create System Files and Directories. client # [ 5.002387] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.995415] systemd[1]: Stopped target Local File Systems. client # [ 5.004258] systemd[1]: Stopped target Basic System. worker1 # [ 4.996443] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.005574] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.008489] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.998426] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 5.000180] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.008466] systemd[1]: Stopped target Path Units. worker2 # [ 5.010193] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.001015] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.009217] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.012097] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.002380] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.003413] systemd[1]: Stopped Virtual Console Setup. client # [ 5.011234] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.004432] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.005362] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.015841] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.006371] systemd[1]: systemd-udevd.service: Consumed 1.597s CPU time over 2.997s wall clock time, 22.2M memory peak. client # [ 5.015990] systemd[1]: Stopped target Slice Units. worker1 # [ 5.007777] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.016817] systemd[1]: Stopped target Socket Units. worker2 # [ 5.019044] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.017525] systemd[1]: Stopped target System Initialization. client # [ 5.018333] systemd[1]: Stopped target Swaps. client # [ 5.018953] systemd[1]: Stopped target Timer Units. client # [ 5.019635] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.012271] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.013263] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.016081] systemd[1]: Closed udev Control Socket. worker1 # [ 5.016855] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.028285] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.029411] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.030144] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.020254] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.031135] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.082933] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.021396] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.031822] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.084865] systemd[1]: Finished Cleanup udev Database. lb # [ 5.086406] systemd[1]: Reached target Switch Root. worker1 # [ 5.024224] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.032884] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.025418] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.033811] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.034971] systemd[1]: Stopped Find NixOS closure. lb # [ 5.089317] systemd[1]: Starting NixOS Activation... client # [ 5.035672] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.028133] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.032143] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.042600] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.033068] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.034093] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.043816] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.047715] systemd[1]: Stopped Create System Files and Directories. client # [ 5.046558] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.049162] systemd[1]: Stopped target Local File Systems. client # [ 5.047556] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.050697] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.049905] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.051942] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.056392] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.057456] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.058213] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.059228] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.057447] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.061085] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.061964] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.060631] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.062953] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.063886] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.065029] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.065832] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.064231] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.066783] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.065370] systemd[1]: Stopped Create System Files and Directories. client # [ 5.066836] systemd[1]: Stopped target Local File Systems. client # [ 5.067713] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.059909] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.068824] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.069807] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.067990] systemd[1]: systemd-udevd.service: Consumed 1.561s CPU time over 3.048s wall clock time, 22.3M memory peak. client # [ 5.070591] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.063331] systemd[1]: Finished Cleanup udev Database. client # [ 5.071637] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.064649] systemd[1]: Reached target Switch Root. worker2 # [ 5.074987] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.076301] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.068645] systemd[1]: Starting NixOS Activation... worker2 # [ 5.080264] systemd[1]: Closed udev Control Socket. worker2 # [ 5.081225] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.082088] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.080296] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.081450] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.083421] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.082515] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.088251] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.089366] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.088175] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.090347] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.089065] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.091326] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.090010] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.090994] systemd[1]: systemd-udevd.service: Consumed 1.658s CPU time over 3.074s wall clock time, 22.4M memory peak. client # [ 5.096231] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.097762] systemd[1]: Closed udev Control Socket. client # [ 5.098476] systemd[1]: Starting Cleanup udev Database... client # [ 5.099226] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.104144] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.105039] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.106137] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.107114] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.112192] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.172649] initrd-nixos-activation-start[176]: booting system configuration /nix/store/vyrpk0yvg81ch7n98c7n14mzchdldfk4-nixos-system-lb-test worker2 # [ 5.122696] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.123809] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.128434] systemd[1]: Reached target Switch Root. worker2 # [ 5.129192] systemd[1]: Starting NixOS Activation... client # [ 5.128688] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.131852] systemd[1]: Finished Cleanup udev Database. client # [ 5.132804] systemd[1]: Reached target Switch Root. client # [ 5.134065] systemd[1]: Starting NixOS Activation... lb # [ 5.205356] initrd-nixos-activation-start[176]: running activation script... worker1 # [ 5.160899] initrd-nixos-activation-start[179]: booting system configuration /nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test worker1 # [ 5.195094] initrd-nixos-activation-start[179]: running activation script... worker2 # [ 5.208096] initrd-nixos-activation-start[179]: booting system configuration /nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test client # [ 5.221387] initrd-nixos-activation-start[178]: booting system configuration /nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test worker2 # [ 5.240949] initrd-nixos-activation-start[179]: running activation script... client # [ 5.254891] initrd-nixos-activation-start[178]: running activation script... lb # [ 5.467287] initrd-nixos-activation-start[199]: setting up /etc... worker1 # [ 5.460413] initrd-nixos-activation-start[202]: setting up /etc... worker2 # [ 5.501049] initrd-nixos-activation-start[202]: setting up /etc... client # [ 5.528304] initrd-nixos-activation-start[201]: setting up /etc... lb # [ 5.618466] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.620328] systemd[1]: Finished NixOS Activation. lb # [ 5.623009] systemd[1]: Starting Switch Root... worker1 # [ 5.582567] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.585996] systemd[1]: Finished NixOS Activation. worker1 # [ 5.587168] systemd[1]: Starting Switch Root... lb # [ 5.649490] systemd[1]: Switching root. worker1 # [ 5.613709] systemd[1]: Switching root. worker2 # [ 5.646349] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.649788] systemd[1]: Finished NixOS Activation. worker2 # [ 5.650970] systemd[1]: Starting Switch Root... client # [ 5.650624] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.654050] systemd[1]: Finished NixOS Activation. client # [ 5.655235] systemd[1]: Starting Switch Root... worker2 # [ 5.675942] systemd[1]: Switching root. client # [ 5.681659] systemd[1]: Switching root. lb # [ 5.832992] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.795607] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.862930] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.866613] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 6.390733] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.402860] systemd[1]: Detected virtualization qemu. worker1 # [ 6.330373] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.406499] systemd[1]: Detected architecture arm64. worker1 # [ 6.342441] systemd[1]: Detected virtualization qemu. lb # [ 6.409670] systemd[1]: Detected first boot. worker1 # [ 6.345444] systemd[1]: Detected architecture arm64. worker1 # [ 6.349194] systemd[1]: Detected first boot. lb # [ 6.415733] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.354646] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.406541] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.407983] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.418626] systemd[1]: Detected virtualization qemu. client # [ 6.420392] systemd[1]: Detected virtualization qemu. worker2 # [ 6.421512] systemd[1]: Detected architecture arm64. client # [ 6.423322] systemd[1]: Detected architecture arm64. worker2 # [ 6.425206] systemd[1]: Detected first boot. client # [ 6.427039] systemd[1]: Detected first boot. worker2 # [ 6.430119] systemd[1]: Initializing machine ID from random generator. client # [ 6.433062] systemd[1]: Initializing machine ID from random generator. lb # [ 6.742750] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.677152] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.753364] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.760475] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.857848] systemd[1]: Applying preset policy. lb # [ 6.923447] systemd[1]: Applying preset policy. worker2 # [ 6.951731] systemd[1]: Applying preset policy. client # [ 6.954323] systemd[1]: Applying preset policy. worker1 # [ 7.082198] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.162105] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.188098] systemd[1]: Populated /etc with preset unit settings. client # [ 7.191286] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.304251] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.305958] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.309963] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.314103] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.316040] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.317454] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.317847] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.318129] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.318367] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.318599] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.318834] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.319064] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.319286] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.319520] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.319761] systemd[1]: Reached target Path Units. worker1 # [ 7.319995] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.320221] systemd[1]: Reached target Slice Units. worker1 # [ 7.320443] systemd[1]: Reached target Swaps. worker1 # [ 7.331116] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.397015] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.334078] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.398303] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.336216] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.338620] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.339784] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.403650] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 7.407425] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.343806] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.409324] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.345922] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.411421] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.349351] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.413647] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.350889] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.415849] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.352380] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.417818] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.354829] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.419694] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.421566] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.423422] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.360275] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.424505] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.361634] systemd[1]: Listening on udev Control Socket. lb # [ 7.426069] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.363146] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.427805] systemd[1]: Reached target Path Units. lb # [ 7.429873] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.366647] systemd[1]: Mounting Huge Pages File System... lb # [ 7.430926] systemd[1]: Reached target Slice Units. lb # [ 7.432326] systemd[1]: Reached target Swaps. lb # [ 7.435200] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.438141] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.372805] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.440378] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.442736] systemd[1]: Listening on Factory Reset Management. lb # [ 7.444777] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.380690] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.448373] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.450538] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.453329] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.454859] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.456376] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.392338] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.458757] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.399231] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 7.464376] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.465763] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.400472] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.467283] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.416638] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.417944] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.471294] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.418379] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.419604] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.422491] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.423799] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 7.479255] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.426377] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.416250] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.428175] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.417519] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.428350] systemd[1]: Created slice User and Session Slice. client # [ 7.430311] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.419459] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.431520] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.430688] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.421827] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.432993] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.433488] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.487931] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.435232] systemd[1]: Expecting device /dev/hvc0... client # [ 7.435927] systemd[1]: Expecting device /dev/hvc0... client # [ 7.437744] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.437081] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.439064] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.439578] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.440166] systemd[1]: Stopped target initrd-fs.target. client # [ 7.441555] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.442389] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.443373] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.443429] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.445360] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.446484] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.445686] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.447642] systemd[1]: Reached target Path Units. lb # [ 7.500871] systemd[1]: Mounting Kernel Trace File System... client # [ 7.448141] systemd[1]: Reached target Path Units. worker2 # [ 7.449333] systemd[1]: Reached target Remote File Systems. client # [ 7.450329] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.436958] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.451173] systemd[1]: Reached target Slice Units. worker2 # [ 7.452167] systemd[1]: Reached target Swaps. worker1 # [ 7.437325] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.452131] systemd[1]: Reached target Slice Units. client # [ 7.454543] systemd[1]: Reached target Swaps. worker2 # [ 7.454938] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.457733] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.457974] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.511421] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.460696] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.460204] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.513622] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.462872] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.462552] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.462958] systemd[1]: Listening on Hostname Service Socket. client # [ 7.465203] systemd[1]: Listening on Factory Reset Management. client # [ 7.466389] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.467348] systemd[1]: Starting Journal Log Access Socket... client # [ 7.470224] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.469545] systemd[1]: Listening on Journal Audit Socket. client # [ 7.472232] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.472902] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.475680] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.475238] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.477288] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.477307] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.464769] systemd[1]: Starting Journal Service... client # [ 7.479634] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.478971] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.481410] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.483929] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.537268] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.485238] systemd[1]: Listening on udev Control Socket. lb # [ 7.537657] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.487583] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.486771] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.537936] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.489017] systemd[1]: Listening on udev Control Socket. lb # [ 7.538206] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.490480] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.490856] systemd[1]: Mounting Huge Pages File System... client # [ 7.493847] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.498508] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.501385] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.494766] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.507343] systemd[1]: Mounting Kernel Debug File System... client # [ 7.513453] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.568876] systemd[1]: Mounting FUSE Control File System... client # [ 7.518818] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.519539] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.512782] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.575056] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.529884] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.520259] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.522456] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.533897] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.536493] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.538700] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.529097] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.596376] systemd[1]: Starting Journal Service... worker2 # [ 7.545252] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.535216] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.538936] systemd[1]: Mounted Huge Pages File System. client # [ 7.551251] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.549887] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.550239] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.542326] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.550520] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.555172] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.548091] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.550283] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.550752] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.551132] systemd[1]: Mounted FUSE Control File System. client # [ 7.566858] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.620270] systemd[1]: Starting Load Kernel Modules... client # [ 7.572158] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.576429] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.576835] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.639908] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.589093] systemd[1]: Mounting FUSE Control File System... client # [ 7.591309] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.649169] systemd-journald[269]: Collecting audit messages is enabled. worker2 # [ 7.600852] systemd[1]: Starting Journal Service... worker1 # [ 7.593486] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.659638] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.661424] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.614070] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.603071] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.655468] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.619089] systemd[1]: Starting Journal Service... lb # [ 7.660483] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.629580] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.635566] systemd[1]: Starting Load Kernel Modules... lb # [ 7.689489] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.645160] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.698572] systemd[1]: Started Journal Service. worker2 # [ 7.649884] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.697106] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.657564] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.646282] systemd[1]: Finished Load Kernel Modules. lb # [ 7.699265] systemd[1]: Mounted Huge Pages File System. lb # [ 7.701944] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.702865] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.652853] EXT4-fs (vda): re-mounted 6791fd92-16a0-4211-9718-3e507c5a93dc. lb # [ 7.703773] systemd[1]: Mounted Kernel Trace File System. client # [ 7.665119] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.666470] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.657275] systemd[1]: Starting Firewall... client # [ 7.670837] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.668670] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.669152] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.669539] systemd[1]: Mounted Kernel Debug File System. lb # [ 7.713294] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.673303] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.714201] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.673799] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.676557] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.664236] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.674179] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.673379] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.674001] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.674297] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.693964] systemd-journald[271]: Collecting audit messages is enabled. lb # [ 7.735857] systemd-modules-load[271]: Module 'atkbd' is built in client # [ 7.699291] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.703155] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.691789] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.743523] systemd-modules-load[271]: Module 'loop' is built in client # [ 7.690289] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.704216] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.694280] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.691580] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.749092] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 7.709794] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.701575] systemd-journald[272]: Collecting audit messages is enabled. client # [ 7.716952] systemd[1]: Started Journal Service. lb # [ 7.759827] systemd[1]: Finished Load Kernel Modules. lb # [ 7.762899] systemd-oomd[272]: No swap; memory pressure usage will be degraded lb # [ 7.770289] systemd[1]: Starting Firewall... client # [ 7.716193] systemd[1]: Mounted Huge Pages File System. client # [ 7.728694] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.785723] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.786596] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.739455] systemd[1]: Started Journal Service. client # [ 7.736944] systemd[1]: Mounted Kernel Debug File System. client # [ 7.739013] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.812390] EXT4-fs (vda): re-mounted 403c5b88-34c7-4837-963b-b9ae6e158bd5. worker2 # [ 7.758742] systemd[1]: Finished Load Kernel Modules. client # [ 7.748242] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.765559] systemd[1]: Starting Firewall... client # [ 7.752270] systemd-modules-load[272]: Module 'atkbd' is built in client # [ 7.756366] systemd-modules-load[272]: Module 'loop' is built in worker2 # [ 7.773118] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 7.752723] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.753893] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.778656] systemd-journald[272]: Collecting audit messages is enabled. lb # [ 7.822451] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.823552] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.769930] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.762662] systemd-modules-load[273]: Module 'atkbd' is built in client # [ 7.792501] EXT4-fs (vda): re-mounted 8e1e7877-f7b8-4126-bb0a-62d6c498188f. lb # [ 7.831169] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.778004] systemd[1]: Finished Load Kernel Modules. client # [ 7.778769] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.774798] systemd-modules-load[273]: Module 'loop' is built in lb # [ 7.836310] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.790483] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 7.785499] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.847867] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.794518] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.798582] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 7.813805] EXT4-fs (vda): re-mounted 92c65040-f127-4aa2-8d3f-abf260cc7c64. client # [ 7.799546] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 7.856378] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.857587] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.801713] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.813957] systemd-oomd[274]: No swap; memory pressure usage will be degraded worker2 # [ 7.828177] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 7.828823] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.816598] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 7.829122] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.817554] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.810873] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.836087] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.838702] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.820903] systemd-oomd[274]: No swap; memory pressure usage will be degraded client # [ 7.835378] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.837593] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.838914] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.856648] systemd[1]: Started Journal Service. worker1 # [ 7.835448] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.846079] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.856339] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.857517] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 7.859233] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.863122] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.874401] systemd-modules-load[273]: Module 'atkbd' is built in worker1 # [ 7.880979] systemd-journald[272]: Received client request to flush runtime journal. lb # [ 7.946672] systemd-journald[269]: Received client request to flush runtime journal. worker2 # [ 7.880430] systemd-modules-load[273]: Module 'loop' is built in worker2 # [ 7.890391] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.925751] systemd-journald[271]: Received client request to flush runtime journal. worker1 # [ 7.928445] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.004431] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.005353] systemd[1]: Reached target First Boot Complete. lb # [ 8.006162] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.007159] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.955524] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.963213] systemd-oomd[274]: No swap; memory pressure usage will be degraded lb # [ 8.016476] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.017390] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.968525] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.969453] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.988014] systemd-journald[272]: Received client request to flush runtime journal. client # [ 7.976462] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 8.001078] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.028385] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.029332] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.030201] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.039037] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.042383] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.043343] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.042761] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.046919] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.056362] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.161187] systemd-udevd[310]: Using default interface naming scheme 'v261'. worker2 # [ 8.174936] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.176608] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.180530] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.196817] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.252796] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.254183] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.200407] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.261174] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.290699] systemd-udevd[315]: Using default interface naming scheme 'v261'. worker1 # [ 8.285402] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.298195] systemd[1]: Mounting /run/wrappers... lb # [ 8.393908] systemd[1]: Mounting /run/wrappers... lb # [ 8.441309] systemd-udevd[306]: Using default interface naming scheme 'v261'. lb # [ 8.444606] systemd[1]: Mounted /run/wrappers. lb # [ 8.445347] systemd[1]: Reached target Local File Systems. lb # [ 8.455556] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.393996] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.396236] systemd[1]: Reached target Local File Systems. worker2 # [ 8.406625] systemd[1]: Mounting /run/wrappers... lb # [ 8.460263] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.402523] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 8.468240] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.469169] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.410112] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.411040] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.411828] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.483598] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.436212] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.437483] systemd[1]: Starting Create System Files and Directories... lb # [ 8.501401] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.461791] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.463135] systemd[1]: Reached target Local File Systems. worker2 # [ 8.466583] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.474997] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.484907] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.501320] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.504344] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.511105] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.513806] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.520356] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.528457] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.614685] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.627958] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.580942] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.587893] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.590553] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.595084] systemd[1]: Mounting /run/wrappers... client # [ 8.597007] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.603210] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.615880] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.673016] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.627791] systemd[1]: Finished Create System Files and Directories. client # [ 8.637049] systemd[1]: Mounted /run/wrappers. client # [ 8.638015] systemd[1]: Reached target Local File Systems. client # [ 8.642240] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.636906] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.645538] systemd[1]: Starting register-nix-paths.service... client # [ 8.655287] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.656307] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.649539] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.659242] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.724284] systemd[1]: Finished Create System Files and Directories. client # [ 8.677272] systemd[1]: Starting Create System Files and Directories... lb # [ 8.736236] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.687396] systemd-udevd[301]: Using default interface naming scheme 'v261'. lb # [ 8.746875] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.714428] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.723216] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.736808] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.735163] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.746198] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.752730] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.871639] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.814081] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.822562] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.816882] systemd[1]: Starting Update is Completed... client # [ 8.831544] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.839525] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.842393] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.857206] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.944100] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 8.957803] systemd[1]: Starting Update is Completed... worker1 # [ 8.904087] systemd[1]: Finished Update is Completed. worker2 # [ 8.910415] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.917203] systemd[1]: Starting Update is Completed... client # [ 8.941760] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.975800] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.986094] systemd[1]: Starting Update is Completed... lb # [ 9.048200] systemd[1]: Finished Update is Completed. worker2 # [ 9.003494] systemd[1]: Finished Update is Completed. client # [ 9.046029] systemd[1]: Finished Update is Completed. client # [ 9.260505] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.322784] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 9.304187] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.306025] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.322505] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.392657] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 9.380485] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.385405] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.451788] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 9.495832] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.499601] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.507229] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.526302] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 9.547240] systemd[1]: Finished Firewall. lb # [ 9.663007] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 9.626771] systemd[1]: Finished Firewall. lb # [ 9.717355] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.725828] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.726743] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 9.682116] systemd[1]: Finished Coldplug All udev Devices. client # [ 9.715700] (udev-worker)[350]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 9.724485] (udev-worker)[350]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.728804] (udev-worker)[356]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 9.759115] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.841726] systemd[1]: Finished register-nix-paths.service. lb # [ 9.845351] systemd[1]: Reached target System Initialization. lb # [ 9.846276] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.847282] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.854751] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 9.855714] systemd[1]: Reached target Timer Units. worker1 # [ 9.793316] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.858380] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 9.862665] systemd[1]: Listening on niks3 server socket. lb # [ 9.863537] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.868218] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.869426] systemd[1]: Reached target Socket Units. lb # [ 9.870143] systemd[1]: Reached target Basic System. lb # [ 9.870882] systemd[1]: Started backdoor.service. lb # [ 9.878383] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 9.888641] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.901610] systemd[1]: Starting Post-Boot Actions... worker2 # [ 9.861774] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.942200] systemd[1]: Started Reset console on configuration changes. lb # [ 9.959478] systemd[1]: Starting resolvconf update... lb # [ 9.964349] (udev-worker)[405]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 9.966473] (udev-worker)[405]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 9.918796] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.987016] systemd[1]: Started rustfs.service. lb # [ 9.995872] (udev-worker)[346]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 9.951255] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 9.942784] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.967965] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 9.969288] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 9.974331] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 9.975218] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.979034] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 9.988120] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 10.042353] nsncd[452]: Sep 23 05:33:02.671 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 9.989716] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 9.990969] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 9.983900] systemd[1]: Finished register-nix-paths.service. worker1 # [ 9.987486] systemd[1]: Reached target System Initialization. worker1 # [ 9.992439] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.057416] systemd[1]: Starting rustfs-bucket.service... lb # [ 10.058358] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 9.999067] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.011517] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.008662] systemd[1]: Reached target Timer Units. lb # connecting to host... worker1 # [ 10.023628] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.030956] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.095897] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.038479] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.044984] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.052324] systemd[1]: Reached target Socket Units. worker1 # [ 10.057991] systemd[1]: Reached target Basic System. worker1 # [ 10.068681] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.081861] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.094258] systemd[1]: Starting Post-Boot Actions... lb # [ 10.162946] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.101439] systemd[1]: Started Reset console on configuration changes. lb # [ 10.167552] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.118331] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.119721] systemd[1]: Reached target System Initialization. worker2 # [ 10.121745] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.124396] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.113181] systemd[1]: Starting resolvconf update... lb # [ 10.180211] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.131105] systemd[1]: Reached target Timer Units. worker1 # [ 10.124465] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.140369] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.141329] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.142082] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 10.194664] systemd[1]: Starting User Login Management... worker2 # [ 10.142966] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.136109] systemd[1]: Started backdoor.service. worker1 # [ 10.143431] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.173113] mousedev: PS/2 mouse device common for all mice worker1 # [ 10.153061] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.170472] systemd[1]: Reached target Socket Units. worker2 # [ 10.177400] systemd[1]: Reached target Basic System. worker2 # [ 10.179043] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.186013] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.179140] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.245477] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.196196] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.187462] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.206797] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.200532] nsncd[490]: Sep 23 05:33:02.816 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.216568] systemd[1]: Starting resolvconf update... worker1 # connecting to host... worker2 # [ 10.223407] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 10.214811] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.222591] systemd[1]: Starting User Login Management... worker2 # [ 10.233623] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.247738] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.276248] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.268199] dbus-broker-launch[495]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.279498] dbus-broker-launch[495]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.284290] dbus-broker-launch[495]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.306540] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.298792] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.316322] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.322629] nsncd[491]: Sep 23 05:33:02.929 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.335063] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.339643] systemd[1]: Starting User Login Management... worker2 # [ 10.349266] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 10.347651] dbus-broker-launch[495]: Ready worker2 # [ 10.359597] systemd[1]: Started backdoor.service. worker2 # [ 10.363524] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.374622] dbus-broker-launch[495]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.388407] dbus-broker-launch[495]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.396566] dbus-broker-launch[495]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.391052] systemd-logind[511]: New seat seat0. worker1 # [ 10.396754] systemd[1]: Started User Login Management. worker1 # [ 10.403288] systemd[1]: Starting linger-users.service... worker2 # [ 10.414730] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.417542] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.418498] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.419326] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.428886] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.429775] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.430652] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.448241] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.507462] systemd-logind[489]: New seat seat0. worker1 # [ 10.449231] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.511958] dbus-broker-launch[487]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.470395] dbus-broker-launch[495]: Ready lb # [ 10.526876] dbus-broker-launch[487]: NSS returned no entry for 'systemd-timesync' lb # [ 10.527883] dbus-broker-launch[487]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # connecting to host... client # [ 10.484751] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 10.488216] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 10.544047] systemd[1]: Started User Login Management. lb # [ 10.544808] systemd[1]: Starting linger-users.service... client # [ 10.492293] systemd[1]: Reached target System Initialization. client # [ 10.493952] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.497486] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.499966] systemd[1]: Reached target Timer Units. client # [ 10.502322] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.495279] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.497992] systemd[1]: Finished linger-users.service. lb # [ 10.560438] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.561411] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.562226] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.508797] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.563070] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.509629] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.510773] systemd[1]: Reached target Socket Units. client # [ 10.511492] systemd[1]: Reached target Basic System. client # [ 10.516204] systemd[1]: Started backdoor.service. client # [ 10.518274] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.575390] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.512922] (udev-worker)[337]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.523100] systemd[1]: Starting nftables firewall... worker1 # [ 10.515011] (udev-worker)[337]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.581622] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.582476] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.534103] systemd-logind[503]: New seat seat0. worker1 # [ 10.527449] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.538643] systemd[1]: Started User Login Management. worker1 # [ 10.533256] nsncd[566]: Sep 23 05:33:03.162 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.535224] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.545397] systemd[1]: Starting linger-users.service... lb # [ 10.600308] systemd[1]: Started D-Bus System Message Bus. client # [ 10.548000] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.542392] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.543388] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.554137] systemd[1]: Finished resolvconf update. worker1 # [ 10.554851] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.555672] systemd[1]: Starting DHCP Client... client # [ 10.564128] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.570180] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.571171] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.624448] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.567690] systemd[1]: Starting Extra networking commands.... client # [ 10.579917] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.584765] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.593426] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.594384] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.595236] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.594289] systemd[1]: Starting resolvconf update... lb # [ 10.651575] systemd-logind[489]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 10.604550] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.605483] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.666849] dbus-broker-launch[487]: Ready worker2 # [ 10.631592] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.636907] systemd[1]: Finished linger-users.service. lb # [ 10.694808] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.699764] systemd[1]: Finished linger-users.service. worker2 # [ 10.665964] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.667223] nsncd[565]: Sep 23 05:33:03.294 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # connecting to host... worker2 # [ 10.677076] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.678071] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.737200] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.738428] nsncd[562]: Sep 23 05:33:03.367 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.693844] (udev-worker)[354]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 10.747575] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.754387] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.755271] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.709219] systemd[1]: Finished resolvconf update. worker2 # [ 10.709958] systemd[1]: Reached target Preparation for Network. worker2 # [ 10.710766] systemd[1]: Starting DHCP Client... worker2 # [ 10.711416] systemd[1]: Starting Extra networking commands.... client: Guest shell says: b'Spawning backdoor root shell...\n' client # [ 10.714434] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.722251] (udev-worker)[357]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.726752] systemd[1]: Finished Post-Boot Actions. lb # [ 10.803093] systemd[1]: Finished resolvconf update. client # [ 10.741247] nsncd[453]: Sep 23 05:33:03.371 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.723355] (udev-worker)[354]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.817215] systemd[1]: Starting DHCP Client... client # [ 10.754999] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.764469] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.767845] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.772797] dhcpcd[598]: dhcpcd-10.3.2 starting worker1 # [ 10.783916] dhcpcd[638]: dev: loaded udev client # [ 10.793428] systemd[1]: Starting User Login Management... client # [ 10.800281] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.834426] 8021q: 802.1Q VLAN Support v1.8 client: connected to guest root shell client: (connecting took 11.26 seconds) client: (finished: waiting for the VM to finish booting, in 11.26 seconds) worker1 # [ 10.884587] systemd[1]: Finished Extra networking commands.. worker1 # [ 10.885457] systemd[1]: Reached target Network. worker1 # [ 10.900631] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.913855] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.937455] dhcpcd[599]: dhcpcd-10.3.2 starting worker1 # [ 10.928466] systemd[1]: Starting Address configuration of eth1... worker1 # [ 10.946772] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 10.949864] dhcpcd[640]: dev: loaded udev worker1 # [ 10.983854] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 10.984337] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 10.996294] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 10.987450] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 10.987776] cfg80211: failed to load regulatory.db client # [ 10.997650] dbus-broker-launch[466]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.990510] systemd[1]: Finished Permit User Sessions. worker1 # [ 10.995922] systemd[1]: Started Getty on tty1. client # [ 11.007573] systemd-logind[485]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 11.000859] systemd[1]: Reached target Login Prompts. client # [ 11.015119] systemd-logind[485]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.017645] systemd-logind[485]: New seat seat0. client # [ 11.034718] systemd[1]: Started User Login Management. lb # [ 11.090638] dhcpcd[614]: dhcpcd-10.3.2 starting worker2 # [ 11.042905] systemd[1]: Finished Extra networking commands.. client # [ 11.045667] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.046591] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 11.115443] mousedev: PS/2 mouse device common for all mice client # [ 11.047412] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 11.053073] systemd[1]: Reached target Network. worker2 # [ 11.053833] systemd[1]: Starting Permit User Sessions... lb # [ 11.109087] dhcpcd[629]: dev: loaded udev client # [ 11.060097] systemd[1]: Stopping User and Group Name Lookups... client # [ 11.061003] dbus-broker-launch[466]: NSS returned no entry for 'systemd-timesync' worker1 # [ 11.065755] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.061963] dbus-broker-launch[466]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 11.056234] systemd-logind[511]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.081930] systemd[1]: Starting linger-users.service... client # [ 11.082753] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.083615] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.078052] network-addresses-eth1-start[653]: adding address 192.168.1.3/24... done worker1 # [ 11.093596] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.084405] dhcpcd[638]: eth0: waiting for carrier worker2 # [ 11.094537] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.087952] dhcpcd[638]: eth0: waiting for carrier client # [ 11.097489] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.151752] systemd[1]: Finished Firewall. worker1 # [ 11.089996] dhcpcd[638]: eth0: carrier acquired worker2 # [ 11.113974] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.155889] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.103963] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.096331] network-addresses-eth1-start[653]: adding address 2001:db8:1::3/64... done client # [ 11.111643] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.105734] dhcpcd[638]: DUID 00:01:00:01:32:46:21:0f:52:54:00:12:34:56 lb # [ 11.168133] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.106695] dhcpcd[638]: eth0: IAID 00:12:34:56 worker1 # [ 11.107322] dhcpcd[638]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.192989] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.116605] systemd[1]: Finished Address configuration of eth1. lb # [ 11.183303] systemd[1]: Starting Extra networking commands.... worker2 # [ 11.150229] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.150731] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.153410] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.153777] cfg80211: failed to load regulatory.db client # [ 11.144931] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.160128] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.166245] systemd[1]: Started Getty on tty1. worker2 # [ 11.166958] systemd[1]: Reached target Login Prompts. client # [ 11.168664] dbus-broker-launch[466]: Ready worker1 # [ 11.172657] mousedev: PS/2 mouse device common for all mice worker2 # [ 11.175514] systemd-logind[503]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 11.229916] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.220641] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.224250] systemd[1]: Finished linger-users.service. worker2 # [ 11.238624] network-addresses-eth1-start[654]: adding address 192.168.1.4/24... done client # [ 11.243805] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 11.259607] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.248375] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 11.250507] dhcpcd[640]: eth0: waiting for carrier client # [ 11.249653] nsncd[550]: Sep 23 05:33:03.877 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.251509] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 11.255420] dhcpcd[640]: eth0: waiting for carrier worker2 # [ 11.256624] network-addresses-eth1-start[654]: adding address 2001:db8:1::4/64... done worker2 # [ 11.257666] dhcpcd[640]: eth0: carrier acquired client # [ 11.261889] systemd[1]: Finished resolvconf update. lb # [ 11.318579] systemd-logind[489]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.272900] dhcpcd[640]: DUID 00:01:00:01:32:46:21:0f:52:54:00:12:34:56 worker2 # [ 11.273850] dhcpcd[640]: eth0: IAID 00:12:34:56 worker2 # [ 11.274488] dhcpcd[640]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.280930] systemd[1]: Finished Address configuration of eth1. client # [ 11.279233] systemd[1]: Starting DHCP Client... lb # [ 11.386277] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.334175] systemd-logind[511]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.432835] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 11.391618] mousedev: PS/2 mouse device common for all mice lb # [ 11.452418] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.452881] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.457084] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.457415] cfg80211: failed to load regulatory.db lb # [ 11.454211] network-addresses-eth1-start[634]: adding address 192.168.1.2/24... done worker2 # [ 11.427791] systemd-logind[503]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.490394] network-addresses-eth1-start[634]: adding address 2001:db8:1::2/64... done client # [ 11.460542] dhcpcd[580]: dhcpcd-10.3.2 starting client # [ 11.470079] dhcpcd[587]: dev: loaded udev lb # [ 11.527657] systemd[1]: Finished Address configuration of eth1. lb # [ 11.555913] dhcpcd[668]: /nix/store/1ki7bgdr0niilkg9jbkpcdaggbv0kb8a-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (638) - Operation not permitted client # [ 11.522100] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.566061] dhcpcd[668]: .resolvconf-wrapped: clearing stale lock pid 638 lb # [ 11.629010] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.617955] dhcpcd[629]: eth0: waiting for carrier lb # [ 11.620161] dhcpcd[629]: libudev: received NULL device lb # [ 11.621025] dhcpcd[629]: libudev: received NULL device lb # [ 11.621720] dhcpcd[629]: eth0: carrier acquired lb # [ 11.645470] dhcpcd[629]: DUID 00:01:00:01:32:46:21:10:52:54:00:12:34:56 lb # [ 11.648537] dhcpcd[629]: eth0: IAID 00:12:34:56 lb # [ 11.649205] dhcpcd[629]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.628006] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.691921] systemd[1]: Finished Extra networking commands.. lb # [ 11.696672] systemd[1]: Reached target Network. lb # [ 11.706370] systemd[1]: Starting PostgreSQL Server... lb # [ 11.712145] systemd[1]: Starting Permit User Sessions... client # [ 11.690670] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.691152] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.694855] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.695170] cfg80211: failed to load regulatory.db client # [ 11.726654] systemd[1]: Finished nftables firewall. client # [ 11.730369] systemd[1]: Reached target Preparation for Network. client # [ 11.739007] systemd[1]: Starting Address configuration of eth1... lb # [ 11.797127] systemd[1]: Finished Permit User Sessions. client # [ 11.745570] systemd[1]: Starting Extra networking commands.... lb # [ 11.807392] systemd[1]: Started Getty on tty1. lb # [ 11.808501] systemd[1]: Reached target Login Prompts. client # [ 11.786474] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.776744] dhcpcd[587]: eth0: waiting for carrier client # [ 11.778530] dhcpcd[587]: libudev: received NULL device client # [ 11.779270] dhcpcd[587]: libudev: received NULL device client # [ 11.787194] dhcpcd[587]: eth0: carrier acquired client # [ 11.797400] dhcpcd[587]: DUID 00:01:00:01:32:46:21:10:52:54:00:12:34:56 client # [ 11.798358] dhcpcd[587]: eth0: IAID 00:12:34:56 client # [ 11.798988] dhcpcd[587]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.865647] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.875487] network-addresses-eth1-start[608]: adding address 192.168.1.1/24... done client # [ 11.903649] network-addresses-eth1-start[608]: adding address 2001:db8:1::1/64... done worker1 # [ 11.923814] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 11.930587] systemd[1]: Finished Address configuration of eth1. lb # [ 11.993371] postgresql-pre-start[729]: The files belonging to this database system will be owned by user "postgres". worker2 # [ 11.942895] dhcpcd[640]: eth0: soliciting a DHCP lease lb # [ 11.994808] postgresql-pre-start[729]: This user must also own the server process. worker2 # [ 11.948647] dhcpcd[640]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 11.956538] dhcpcd[640]: eth0: probing address 10.0.2.15/24 lb # [ 12.011576] postgresql-pre-start[729]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 12.015121] postgresql-pre-start[729]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.017952] postgresql-pre-start[729]: The default text search configuration will be set to "english". lb # [ 12.019186] postgresql-pre-start[729]: Data page checksums are enabled. lb # [ 12.023339] postgresql-pre-start[729]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.027308] postgresql-pre-start[729]: creating subdirectories ... ok lb # [ 12.030403] postgresql-pre-start[729]: selecting dynamic shared memory implementation ... posix client # [ 12.032711] systemd[1]: Finished Extra networking commands.. client # [ 12.038088] systemd[1]: Reached target Network. client # [ 12.040393] systemd[1]: Starting Permit User Sessions... client # [ 12.073290] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 12.101085] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.145377] postgresql-pre-start[729]: selecting default "max_connections" ... 100 client # [ 12.116128] systemd[1]: Finished Permit User Sessions. client # [ 12.119696] systemd[1]: Started Getty on tty1. client # [ 12.125280] systemd[1]: Reached target Login Prompts. lb # [ 12.269102] postgresql-pre-start[729]: selecting default "shared_buffers" ... 128MB worker1 # [ 12.241539] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.257514] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 12.267622] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 12.273605] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.274393] systemd[1]: Starting Virtual Console Setup... client: (finished: waiting for unit network-addresses-eth1.service, in 12.71 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } worker1 # [ 12.344396] systemd-logind[511]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2 # [ 12.398698] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 12.415293] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.416438] systemd-vconsole-setup[695]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 12.420341] systemd[1]: Finished Virtual Console Setup. worker2 # [ 12.434621] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.438356] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.442336] systemd[1]: Starting Virtual Console Setup... client # [ 12.451659] systemd[1]: Starting Virtual Console Setup... lb: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker1 # [ 12.467974] dhcpcd[638]: eth0: soliciting a DHCP lease worker1 # [ 12.469645] dhcpcd[638]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 12.476244] dhcpcd[638]: eth0: probing address 10.0.2.15/24 client # [ 12.486028] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.488837] systemd[1]: Stopped Virtual Console Setup. client # [ 12.492685] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 12.496734] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.502552] systemd-logind[503]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.548092] systemd-logind[485]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker2 # [ 12.580672] systemd-vconsole-setup[695]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 12.585102] systemd[1]: Finished Virtual Console Setup. worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.03 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.03 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } client # [ 12.642938] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.646827] systemd[1]: Finished Virtual Console Setup. lb # [ 12.717565] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service worker1 # [ 12.707953] dhcpcd[638]: eth0: soliciting an IPv6 router worker1 # [ 12.711461] dhcpcd[638]: eth0: Router Advertisement from fe80::2 worker1 # [ 12.714161] dhcpcd[638]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 12.717013] dhcpcd[638]: eth0: adding route to fec0::/64 worker1 # [ 12.719218] dhcpcd[638]: eth0: adding default route via fe80::2 lb # [ 13.110033] systemd[1]: Starting Virtual Console Setup... lb # [ 13.148419] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.159297] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.165476] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.170542] systemd[1]: Starting Virtual Console Setup... lb # [ 13.215481] systemd-logind[489]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 13.226227] dhcpcd[587]: eth0: soliciting a DHCP lease client # [ 13.233237] dhcpcd[587]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 13.236512] dhcpcd[587]: eth0: probing address 10.0.2.15/24 lb # [ 13.385200] systemd-vconsole-setup[760]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 13.389178] systemd[1]: Finished Virtual Console Setup. lb # [ 13.392203] dhcpcd[629]: eth0: soliciting a DHCP lease lb # [ 13.396501] dhcpcd[629]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.404146] dhcpcd[629]: eth0: probing address 10.0.2.15/24 worker2 # [ 13.554948] dhcpcd[640]: eth0: soliciting an IPv6 router worker2 # [ 13.558334] dhcpcd[640]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.561136] dhcpcd[640]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.563834] dhcpcd[640]: eth0: adding route to fec0::/64 worker2 # [ 13.566224] dhcpcd[640]: eth0: adding default route via fe80::2 lb # [ 13.762311] postgresql-pre-start[729]: selecting default time zone ... UTC lb # [ 13.764967] postgresql-pre-start[729]: creating configuration files ... ok client # [ 13.815664] dhcpcd[587]: eth0: soliciting an IPv6 router client # [ 13.819516] dhcpcd[587]: eth0: Router Advertisement from fe80::2 client # [ 13.822754] dhcpcd[587]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 13.825616] dhcpcd[587]: eth0: adding route to fec0::/64 client # [ 13.827827] dhcpcd[587]: eth0: adding default route via fe80::2 lb # [ 14.010983] dhcpcd[629]: eth0: soliciting an IPv6 router lb # [ 14.017140] dhcpcd[629]: eth0: Router Advertisement from fe80::2 lb # [ 14.017993] dhcpcd[629]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.018901] dhcpcd[629]: eth0: adding route to fec0::/64 lb # [ 14.019628] dhcpcd[629]: eth0: adding default route via fe80::2 lb # [ 14.032175] postgresql-pre-start[729]: running bootstrap script ... ok lb # [ 14.519887] postgresql-pre-start[729]: performing post-bootstrap initialization ... ok lb # [ 14.676555] postgresql-pre-start[729]: syncing data to disk ... ok lb # [ 14.677627] postgresql-pre-start[729]: initdb: warning: enabling "trust" authentication for local connections lb # [ 14.679983] postgresql-pre-start[729]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 14.682064] postgresql-pre-start[729]: Success. You can now start the database server using: lb # [ 14.683150] postgresql-pre-start[729]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 14.772874] postgres[779]: [779] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 14.775765] postgres[779]: [779] LOG: listening on IPv6 address "::1", port 5432 lb # [ 14.777141] postgres[779]: [779] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 14.778815] postgres[779]: [779] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 14.791223] postgres[788]: [788] LOG: database system was shut down at 2026-09-23 05:33:07 GMT lb # [ 14.795694] postgres[779]: [779] LOG: database system is ready to accept connections lb # [ 14.801464] systemd[1]: Started PostgreSQL Server. lb # [ 14.808647] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.001616] postgresql-setup-start[800]: CREATE DATABASE lb # [ 15.038139] postgresql-setup-start[805]: CREATE ROLE lb # [ 15.057052] postgresql-setup-start[807]: ALTER DATABASE lb # [ 15.067212] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.068776] systemd[1]: Reached target PostgreSQL. worker1 # [ 16.638884] dhcpcd[638]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 16.642712] dhcpcd[638]: eth0: adding route to 10.0.2.0/24 worker1 # [ 16.647764] dhcpcd[638]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.723712] dhcpcd[640]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 16.726806] dhcpcd[640]: eth0: adding route to 10.0.2.0/24 worker2 # [ 16.733681] dhcpcd[640]: eth0: adding default route via 10.0.2.2 worker1 # [ 16.783918] systemd[1]: Started DHCP Client. worker1 # [ 16.786221] systemd[1]: Reached target Multi-User System. worker1 # [ 16.787223] systemd[1]: Startup finished in 930ms (kernel) + 4.965s (initrd) + 10.890s (userspace) = 16.786s. worker2 # [ 16.871706] systemd[1]: Started DHCP Client. worker2 # [ 16.874612] systemd[1]: Reached target Multi-User System. worker2 # [ 16.875714] systemd[1]: Startup finished in 940ms (kernel) + 5.021s (initrd) + 10.912s (userspace) = 16.873s. client # [ 18.134889] dhcpcd[587]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.138038] dhcpcd[587]: eth0: adding route to 10.0.2.0/24 client # [ 18.142379] dhcpcd[587]: eth0: adding default route via 10.0.2.2 client # [ 18.277437] systemd[1]: Started DHCP Client. client # [ 18.280958] systemd[1]: Reached target Multi-User System. client # [ 18.282328] systemd[1]: Startup finished in 935ms (kernel) + 5.027s (initrd) + 12.318s (userspace) = 18.282s. lb # [ 18.628708] dhcpcd[629]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 18.631902] dhcpcd[629]: eth0: adding route to 10.0.2.0/24 lb # [ 18.636341] dhcpcd[629]: eth0: adding default route via 10.0.2.2 lb # [ 18.764877] systemd[1]: Started DHCP Client. lb # [ 18.767462] systemd[1]: Reached target Network is Online. lb # [ 18.771782] systemd[1]: Started Envoy reverse proxy. lb # [ 18.780544] systemd[1]: Started mock-oidc.service. lb # [ 18.939388] mock-oidc-server[908]: Mock OIDC Server running lb # [ 18.942108] mock-oidc-server[908]: OIDC Address: 192.168.1.2:8080 lb # [ 18.943708] mock-oidc-server[908]: Issue Address: 0.0.0.0:8081 lb # [ 18.945350] mock-oidc-server[908]: Issuer: http://192.168.1.2:8080/oidc lb # [ 18.947042] mock-oidc-server[908]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 18.949214] mock-oidc-server[908]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 18.952078] mock-oidc-server[908]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 19.135338] envoy[907]: [2026-09-23 05:33:11.766][907][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 19.138116] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 19.139555] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 19.142049] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 19.165407] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 19.172722] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 19.175857] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 19.181063] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 19.183931] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 19.186373] envoy[907]: [2026-09-23 05:33:11.768][907][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 19.189685] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 19.222761] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 19.225763] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 19.233530] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 19.235572] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 19.237716] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 19.239464] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 19.242666] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 19.245209] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 19.248104] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 19.255267] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 19.257836] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 19.259555] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 19.265081] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 19.266911] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 19.269375] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 19.271632] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 19.276389] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 19.278152] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 19.279943] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 19.282833] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 19.285871] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 19.287753] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 19.295446] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 19.302201] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 19.303982] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 19.309212] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 19.311438] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 19.314715] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 19.316454] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 19.323448] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 19.326701] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 19.330571] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.332773] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 19.335333] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 19.338322] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 19.339815] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 19.344470] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 19.352758] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 19.355130] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 19.357274] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 19.359983] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 19.362602] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 19.367105] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 19.369685] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 19.372544] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 19.376680] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 19.378932] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 19.381658] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 19.385538] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 19.388415] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction worker2 # [ 19.338079] systemd[1]: Reached target Network is Online. lb # [ 19.394130] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default worker1 # [ 19.333295] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 19.397389] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default worker2 # [ 19.348191] systemd[1]: Starting scheduler-token.service... lb # [ 19.399424] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 19.404305] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.406739] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 19.408371] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 19.411700] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 19.414662] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 19.416599] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 19.418230] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 19.421636] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 19.423419] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 19.425925] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 19.427616] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 19.430633] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 19.436190] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 19.438206] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 19.439846] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 19.451217] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 19.456351] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 19.458289] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 19.460324] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 19.473684] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 19.475399] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 19.478098] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 19.479918] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 19.481854] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 19.484527] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 19.487420] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 19.493058] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 19.495942] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 19.499048] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret worker2 # [ 19.448359] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 19.501275] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 19.503595] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 19.506810] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid worker2 # [ 19.456624] systemd[1]: Finished scheduler-token.service. lb # [ 19.509451] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 19.512818] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action worker2 # [ 19.458603] systemd[1]: scheduler-token.service: Consumed 63ms CPU time over 104ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 19.514821] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 19.517495] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation worker2 # [ 19.466592] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 19.519483] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 19.530622] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 19.533358] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.535904] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 19.537722] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 19.539491] envoy[907]: [2026-09-23 05:33:11.769][907][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 19.541532] envoy[907]: [2026-09-23 05:33:11.785][907][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 19.542909] envoy[907]: [2026-09-23 05:33:11.785][907][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 19.558982] envoy[907]: [2026-09-23 05:33:11.785][907][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 19.560550] envoy[907]: [2026-09-23 05:33:11.785][907][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 19.569479] envoy[907]: [2026-09-23 05:33:11.785][907][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 19.571393] envoy[907]: [2026-09-23 05:33:11.821][907][info][main] [source/server/server.cc:968] runtime: {} worker1 # [ 19.510662] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 19.572763] envoy[907]: [2026-09-23 05:33:11.853][907][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 19.574416] envoy[907]: [2026-09-23 05:33:11.853][907][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 19.575975] envoy[907]: [2026-09-23 05:33:11.853][907][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 19.577641] envoy[907]: [2026-09-23 05:33:11.853][907][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 19.579219] envoy[907]: [2026-09-23 05:33:11.853][907][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 19.580873] envoy[907]: [2026-09-23 05:33:11.853][907][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) worker1 # [ 19.519071] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 19.582496] envoy[907]: [2026-09-23 05:33:12.016][907][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker1 # [ 19.521407] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 19.583976] envoy[907]: [2026-09-23 05:33:12.028][907][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 19.585714] envoy[907]: [2026-09-23 05:33:12.028][907][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 19.588849] envoy[907]: [2026-09-23 05:33:12.028][907][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 19.531175] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 19.534349] nix-grpc-daemon[769]: E0000 00:00:1790141592.166472 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 19.539173] nix-grpc-daemon[769]: E0000 00:00:1790141592.174626 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 19.557859] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 19.560254] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 19.626583] envoy[907]: [2026-09-23 05:33:12.257][907][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization worker1 # [ 19.564359] nix-grpc-daemon[769]: E0000 00:00:1790141592.197510 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 19.628624] envoy[907]: [2026-09-23 05:33:12.257][907][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker1 # [ 19.581986] nix-grpc-daemon[769]: E0000 00:00:1790141592.217699 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 19.588093] nix-grpc-daemon[769]: E0000 00:00:1790141592.220435 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.608503] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 19.603840] nix-grpc-daemon[769]: E0000 00:00:1790141592.238581 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.616352] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 19.607658] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.618517] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 19.630487] nix-grpc-daemon[772]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 19.633380] nix-grpc-daemon[772]: E0000 00:00:1790141592.258428 772 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.638347] nix-grpc-daemon[772]: E0000 00:00:1790141592.266450 772 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.654393] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 19.657217] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 19.659209] nix-grpc-daemon[772]: E0000 00:00:1790141592.287212 814 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.688067] nix-grpc-daemon[772]: E0000 00:00:1790141592.307981 814 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.690338] nix-grpc-daemon[772]: E0000 00:00:1790141592.308374 814 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.700967] nix-grpc-daemon[772]: E0000 00:00:1790141592.328441 814 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.704522] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 19.808337] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.905236] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 20.208910] nix-grpc-daemon[769]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.305803] nix-grpc-daemon[772]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 21.010470] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 21.107378] nix-grpc-daemon[772]: ts=2026-09-23T05:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 22.611119] nix-grpc-daemon[769]: ts=2026-09-23T05:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 22.707886] nix-grpc-daemon[772]: ts=2026-09-23T05:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 25.811655] nix-grpc-daemon[769]: ts=2026-09-23T05:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 25.908577] nix-grpc-daemon[772]: ts=2026-09-23T05:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 28.513112] rustfs-bucket-start[1001]: mb s3://farm lb # [ 28.525244] systemd[1]: Finished rustfs-bucket.service. lb # [ 28.537007] systemd[1]: Starting niks3 server... lb # [ 28.672763] postgres[1021]: [1021] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 28.674044] postgres[1021]: [1021] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 28.701006] niks3-server[1016]: 2026/09/23 05:33:21 OK 20241026095416_initial_model.sql (17.07ms) lb # [ 28.704645] niks3-server[1016]: 2026/09/23 05:33:21 OK 20251210153512_drop_unused_gin_index.sql (3.91ms) lb # [ 28.711154] niks3-server[1016]: 2026/09/23 05:33:21 OK 20251218171726_add_pins.sql (6.43ms) lb # [ 28.715984] niks3-server[1016]: 2026/09/23 05:33:21 OK 20260628120000_add_object_size_and_stats.sql (4.8ms) lb # [ 28.721071] niks3-server[1016]: 2026/09/23 05:33:21 OK 20260905000000_add_claims.sql (5.02ms) lb # [ 28.725188] niks3-server[1016]: 2026/09/23 05:33:21 OK 20260920000000_drop_claims.sql (3.96ms) lb # [ 28.726466] niks3-server[1016]: 2026/09/23 05:33:21 goose: successfully migrated database to version: 20260920000000 lb # [ 28.731001] niks3-server[1016]: 2026/09/23 05:33:21 OK 1_commit_pending_closure.sql (5.77ms) lb # [ 28.734025] niks3-server[1016]: 2026/09/23 05:33:21 OK 2_object_stats_trigger.sql (2.86ms) lb # [ 28.736874] niks3-server[1016]: 2026/09/23 05:33:21 goose: up to current file version: 2 lb # [ 28.739359] niks3-server[1016]: 2026/09/23 05:33:21 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 28.759103] niks3-server[1016]: 2026/09/23 05:33:21 INFO Read proxy enabled — serving cache objects from S3 lb # [ 28.763781] niks3-server[1016]: 2026/09/23 05:33:21 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 28.769291] niks3-server[1016]: 2026/09/23 05:33:21 INFO systemd watchdog enabled interval=15s lb # [ 28.771651] niks3-server[1016]: 2026/09/23 05:33:21 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 28.773159] systemd[1]: Started niks3 server. lb # [ 28.773977] systemd[1]: Reached target Multi-User System. lb # [ 28.775006] systemd[1]: Startup finished in 938ms (kernel) + 5.008s (initrd) + 22.823s (userspace) = 28.770s. lb: (finished: waiting for unit niks3.service, in 16.90 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.04 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 30.812343] nix-grpc-daemon[769]: ts=2026-09-23T05:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 30.909137] nix-grpc-daemon[772]: ts=2026-09-23T05:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 34.626237] envoy[907]: [2026-09-23 05:33:27.256][907][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 34.626500] envoy[907]: [2026-09-23 05:33:27.256][907][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 34.626604] envoy[907]: [2026-09-23 05:33:27.256][907][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 34.650647] envoy[907]: [2026-09-23 05:33:27.280][907][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 34.997078] envoy[907]: conn peer=[::1]:44328 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=8 lb: (finished: waiting for TCP port 50051 on localhost, in 5.29 seconds) worker2 # [ 35.120887] systemd[1]: Starting scheduler-token.service... worker2 # [ 35.197560] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 35.202001] systemd[1]: Finished scheduler-token.service. worker2 # [ 35.203429] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 86ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 35.813701] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.818789] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.911360] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.911754] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.019841] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.025096] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.112149] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.117169] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.225847] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.231023] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 36.318497] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.323783] nix-grpc-daemon[772]: ts=2026-09-23T05:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 36.431786] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.437105] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.524816] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.529875] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.637723] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.642945] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.731058] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.736808] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.845422] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.849571] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.939351] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.943518] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.045970] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.051058] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.139927] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.144956] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.251703] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.256912] nix-grpc-daemon[769]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.345910] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.350878] nix-grpc-daemon[772]: ts=2026-09-23T05:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 37.457632] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.462804] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.551887] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.556977] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.663344] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.667701] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.757816] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.762834] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.870190] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.874326] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.965624] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.969670] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.070645] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.075917] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.166273] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.171431] nix-grpc-daemon[772]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.276730] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.281755] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.372468] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.377506] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 38.482444] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.487410] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.578633] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.583887] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.688123] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.693387] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.709556] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_take_over lb # [ 38.773582] niks3-server[1016]: 2026/09/23 05:33:31 INFO lead: acquired remote=192.168.1.3:52234 worker2 # [ 38.785010] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.790268] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.896491] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.900608] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.995121] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.999492] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.098401] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.104630] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.195455] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.200890] nix-grpc-daemon[772]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.301918] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.307487] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.403112] nix-grpc-daemon[772]: ts=2026-09-23T05:33:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.409476] nix-grpc-daemon[772]: ts=2026-09-23T05:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 39.508519] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.513980] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker2 # [ 39.610255] nix-grpc-daemon[772]: ts=2026-09-23T05:33:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.616369] nix-grpc-daemon[772]: ts=2026-09-23T05:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.715782] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.737988] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45520 worker1 # [ 39.744216] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 39.818495] nix-grpc-daemon[772]: ts=2026-09-23T05:33:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.820317] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 39.825513] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45520 worker1 # [ 39.831046] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 49.569568] nix-grpc-daemon[769]: E0000 00:00:1790141622.201611 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 49.623799] nix-grpc-daemon[769]: E0000 00:00:1790141622.257851 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 49.664959] nix-grpc-daemon[772]: E0000 00:00:1790141622.291211 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 49.660461] nix-grpc-daemon[769]: E0000 00:00:1790141622.265577 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 49.665117] nix-grpc-daemon[769]: E0000 00:00:1790141622.294211 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 49.719414] nix-grpc-daemon[772]: E0000 00:00:1790141622.347391 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 49.753218] nix-grpc-daemon[772]: E0000 00:00:1790141622.352982 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 49.756484] nix-grpc-daemon[772]: E0000 00:00:1790141622.379105 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 58.573925] nix-grpc-daemon[769]: ts=2026-09-23T05:33:51Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:35862 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 23.63 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.73 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 59.883066] nix-grpc-daemon[772]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 59.898569] systemd[1]: Started Nix Daemon. worker2 # [ 60.088655] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 60.108595] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5) worker2 # [ 60.177634] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 2/5) worker2 # [ 60.303484] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 350 ms (attempt 3/5) worker2 # [ 60.655974] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 586 ms (attempt 4/5) worker2 # [ 61.255031] nix-grpc-daemon[772]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 61.260870] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 61.278426] nix-grpc-daemon[772]: ts=2026-09-23T05:33:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 61.293007] nix-grpc-daemon[772]: ts=2026-09-23T05:33:53Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 61.312962] nix-grpc-daemon[769]: ts=2026-09-23T05:33:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 61.330794] systemd[1]: Started Nix Daemon. worker1 # [ 61.506916] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 61.530682] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5) worker1 # [ 61.624357] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5) worker1 # [ 61.708508] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5) worker1 # [ 61.982566] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 335 ms (attempt 4/5) worker1 # [ 62.322475] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 62.327419] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 62.337926] nix-grpc-daemon[769]: ts=2026-09-23T05:33:54Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker1 # [ 62.667490] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:50732 worker2 # [ 62.707159] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 62.712937] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40618 worker2 # [ 62.724110] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 62.738618] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40618 worker2 # [ 62.746519] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds) worker1 # [ 62.981270] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 63.156844] nix-grpc-daemon[772]: ts=2026-09-23T05:33:55Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.58 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 63.312734] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 63.323136] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 63.353426] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 63.360658] nix-grpc-daemon[769]: ts=2026-09-23T05:33:55Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 63.402932] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 63.420846] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying 1 paths to 'lb:50051'... worker1 # [ 63.436979] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 63.458515] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 63.505671] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=info event=niks3_push_started pid=869 worker1 # [ 63.687576] nix-grpc-daemon[869]: time=2026-09-23T05:33:56.322Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 63.804112] nix-daemon[855]: accepted connection from pid 882, user nix-grpc-daemon (trusted) worker1 # [ 63.821024] nix-daemon[855]: reaped child process 885, status = succeeded lb # [ 63.892979] niks3-server[1016]: 2026/09/23 05:33:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 63.852066] nix-grpc-daemon[869]: time=2026-09-23T05:33:56.487Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 63.853589] nix-grpc-daemon[869]: time=2026-09-23T05:33:56.487Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 63.964667] niks3-server[1016]: 2026/09/23 05:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 63.997677] niks3-server[1016]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 64.007895] niks3-server[1016]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 64.011133] niks3-server[1016]: 2026/09/23 05:33:56 INFO Signed narinfos id=1 count=1 worker1 # [ 63.948214] nix-grpc-daemon[869]: time=2026-09-23T05:33:56.581Z level=INFO msg="Uploading 1 narinfos" lb # [ 64.027476] niks3-server[1016]: 2026/09/23 05:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 64.040042] niks3-server[1016]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo lb # [ 64.049333] niks3-server[1016]: 2026/09/23 05:33:56 INFO Completed upload id=1 worker1 # [ 63.987830] nix-grpc-daemon[869]: time=2026-09-23T05:33:56.623Z level=INFO msg="Upload complete. (297ms)" worker1 # [ 63.998783] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 64.013078] nix-grpc-daemon[769]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 64.029485] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 64.034218] nix-daemon[855]: reaped child process 867, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker1 # [ 64.046038] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 64.938458] nix-daemon[855]: reaped child process 889, status = succeeded worker1 # [ 65.024836] nix-daemon[855]: accepted connection from pid 905, user nix-grpc-daemon (trusted) worker1 # [ 65.042671] nix-daemon[855]: reaped child process 908, status = succeeded lb # [ 65.109358] niks3-server[1016]: 2026/09/23 05:33:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 65.062015] nix-grpc-daemon[869]: time=2026-09-23T05:33:57.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 65.063730] nix-grpc-daemon[869]: time=2026-09-23T05:33:57.695Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 65.158594] niks3-server[1016]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 65.171719] niks3-server[1016]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 65.117974] nix-grpc-daemon[869]: time=2026-09-23T05:33:57.752Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.183123] niks3-server[1016]: 2026/09/23 05:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 65.190287] niks3-server[1016]: 2026/09/23 05:33:57 INFO Signed narinfos id=2 count=1 lb # [ 65.194594] niks3-server[1016]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 65.201367] niks3-server[1016]: 2026/09/23 05:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 65.211250] niks3-server[1016]: 2026/09/23 05:33:57 INFO Completed upload id=2 worker1 # [ 65.149593] nix-grpc-daemon[869]: time=2026-09-23T05:33:57.785Z level=INFO msg="Upload complete. (291ms)" lb # [ 65.215166] niks3-server[1016]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 65.176510] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 65.191978] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 65.197063] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 65.202275] nix-daemon[855]: reaped child process 887, status = succeeded worker1 # [ 65.224164] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 65.236909] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 65.251439] nix-grpc-daemon[869]: time=2026-09-23T05:33:57.887Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 65.262083] nix-grpc-daemon[769]: ts=2026-09-23T05:33:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 65.284731] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 65.440410] nix-daemon[855]: accepted connection from pid 914, user nix-grpc-daemon (trusted) worker1 # [ 65.461965] nix-daemon[855]: reaped child process 917, status = succeeded lb # [ 65.526829] niks3-server[1016]: 2026/09/23 05:33:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 65.474322] nix-grpc-daemon[869]: time=2026-09-23T05:33:58.109Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 65.476106] nix-grpc-daemon[869]: time=2026-09-23T05:33:58.109Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 65.556993] niks3-server[1016]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker1 # [ 65.515128] nix-grpc-daemon[869]: time=2026-09-23T05:33:58.150Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.569176] niks3-server[1016]: 2026/09/23 05:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 65.582849] niks3-server[1016]: 2026/09/23 05:33:58 INFO Signed narinfos id=3 count=1 lb # [ 65.589962] niks3-server[1016]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 65.606352] niks3-server[1016]: 2026/09/23 05:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 65.612938] niks3-server[1016]: 2026/09/23 05:33:58 INFO Completed upload id=3 lb # [ 65.617273] niks3-server[1016]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 65.550870] nix-grpc-daemon[869]: time=2026-09-23T05:33:58.186Z level=INFO msg="Upload complete. (249ms)" worker1 # [ 65.560629] nix-grpc-daemon[769]: ts=2026-09-23T05:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 65.580767] nix-grpc-daemon[769]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 65.590676] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 65.593831] nix-daemon[855]: reaped child process 912, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 65.606068] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 67.279559] nix-daemon[855]: reaped child process 921, status = succeeded worker1 # [ 67.397722] nix-daemon[855]: accepted connection from pid 937, user nix-grpc-daemon (trusted) worker1 # [ 67.414689] nix-daemon[855]: reaped child process 940, status = succeeded lb # [ 67.485508] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 67.438086] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.072Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 67.440591] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.072Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 67.527550] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker1 # [ 67.474496] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.109Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.548607] niks3-server[1016]: 2026/09/23 05:34:00 INFO Signed narinfos id=4 count=1 lb # [ 67.557658] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 67.566279] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker1 # [ 67.516959] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.152Z level=INFO msg="Upload complete. (291ms)" lb # [ 67.579561] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 67.583515] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 67.588153] niks3-server[1016]: 2026/09/23 05:34:00 INFO Completed upload id=4 lb # [ 67.591062] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 67.531688] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 67.554685] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 67.559276] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 67.564914] nix-daemon[855]: reaped child process 919, status = succeeded worker1 # [ 67.594634] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 67.609915] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 67.625892] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.261Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 67.636740] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 67.659872] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 67.819522] nix-daemon[855]: accepted connection from pid 946, user nix-grpc-daemon (trusted) worker1 # [ 67.840598] nix-daemon[855]: reaped child process 949, status = succeeded lb # [ 67.901165] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 67.849031] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.484Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 67.850931] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.484Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 67.930683] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls worker1 # [ 67.880759] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.516Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.945939] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 67.956348] niks3-server[1016]: 2026/09/23 05:34:00 INFO Signed narinfos id=5 count=1 lb # [ 67.965117] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker1 # [ 67.910284] nix-grpc-daemon[869]: time=2026-09-23T05:34:00.545Z level=INFO msg="Upload complete. (231ms)" lb # [ 67.974078] niks3-server[1016]: 2026/09/23 05:34:00 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 67.982032] niks3-server[1016]: 2026/09/23 05:34:00 INFO Completed upload id=5 worker1 # [ 67.920483] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 67.984719] niks3-server[1016]: 2026/09/23 05:34:00 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 67.931831] nix-grpc-daemon[769]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 67.941181] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 67.944191] nix-daemon[855]: reaped child process 944, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 67.958297] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 69.224732] nix-daemon[855]: reaped child process 953, status = succeeded worker1 # [ 69.330448] nix-daemon[855]: accepted connection from pid 969, user nix-grpc-daemon (trusted) lb # [ 69.390802] envoy[907]: conn peer=[2001:db8:1::1]:46792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=2508 lb # [ 69.396083] envoy[907]: conn peer=[2001:db8:1::1]:60006 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=84 lb # [ 69.400595] envoy[907]: conn peer=[2001:db8:1::1]:60018 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=13 lb # [ 69.405163] envoy[907]: conn peer=[2001:db8:1::1]:60032 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=13 worker1 # [ 69.350170] nix-daemon[855]: reaped child process 972, status = succeeded lb # [ 69.417998] niks3-server[1016]: 2026/09/23 05:34:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 69.367122] nix-grpc-daemon[869]: time=2026-09-23T05:34:02.000Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 69.369054] nix-grpc-daemon[869]: time=2026-09-23T05:34:02.000Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 69.460091] niks3-server[1016]: 2026/09/23 05:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 69.404728] nix-grpc-daemon[869]: time=2026-09-23T05:34:02.040Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.471047] niks3-server[1016]: 2026/09/23 05:34:02 INFO Signed narinfos id=6 count=1 lb # [ 69.478872] niks3-server[1016]: 2026/09/23 05:34:02 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 69.487675] niks3-server[1016]: 2026/09/23 05:34:02 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 69.498979] niks3-server[1016]: 2026/09/23 05:34:02 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 69.505468] niks3-server[1016]: 2026/09/23 05:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 69.510041] niks3-server[1016]: 2026/09/23 05:34:02 INFO Completed upload id=6 worker1 # [ 69.443352] nix-grpc-daemon[869]: time=2026-09-23T05:34:02.078Z level=INFO msg="Upload complete. (274ms)" lb # [ 69.514130] niks3-server[1016]: 2026/09/23 05:34:02 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 69.458280] nix-grpc-daemon[769]: ts=2026-09-23T05:34:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 69.477142] nix-daemon[855]: reaped child process 951, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.34 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.88 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 70.719593] nix-grpc-daemon[772]: ts=2026-09-23T05:34:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 70.713677] nix-grpc-daemon[769]: ts=2026-09-23T05:34:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 70.725353] nix-grpc-daemon[869]: time=2026-09-23T05:34:03.360Z level=INFO msg="All 2 paths already cached" worker1 # [ 70.743125] nix-grpc-daemon[769]: ts=2026-09-23T05:34:03Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.29 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.15 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 worker1 # [ 71.576859] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 71.596979] nix-grpc-daemon[772]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 71.615369] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 71.622265] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 71.650994] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 71.665971] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 71.681825] nix-grpc-daemon[869]: time=2026-09-23T05:34:04.317Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 71.694122] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 71.709442] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 71.866482] nix-daemon[855]: accepted connection from pid 1001, user nix-grpc-daemon (trusted) worker1 # [ 71.884171] nix-daemon[855]: reaped child process 1004, status = succeeded lb # [ 71.954465] niks3-server[1016]: 2026/09/23 05:34:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 71.903704] nix-grpc-daemon[869]: time=2026-09-23T05:34:04.538Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 71.905547] nix-grpc-daemon[869]: time=2026-09-23T05:34:04.538Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 71.987571] niks3-server[1016]: 2026/09/23 05:34:04 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 71.937607] nix-grpc-daemon[869]: time=2026-09-23T05:34:04.573Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.002761] niks3-server[1016]: 2026/09/23 05:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 72.012939] niks3-server[1016]: 2026/09/23 05:34:04 INFO Signed narinfos id=7 count=1 worker1 # [ 71.959585] nix-grpc-daemon[869]: time=2026-09-23T05:34:04.595Z level=INFO msg="Upload complete. (232ms)" lb # [ 72.021503] niks3-server[1016]: 2026/09/23 05:34:04 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 72.030642] niks3-server[1016]: 2026/09/23 05:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker1 # [ 71.969842] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 72.035434] niks3-server[1016]: 2026/09/23 05:34:04 INFO Completed upload id=7 lb # [ 72.038456] niks3-server[1016]: 2026/09/23 05:34:04 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 71.984003] nix-grpc-daemon[769]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 71.992170] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 71.995783] nix-daemon[855]: reaped child process 999, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 72.006302] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 73.516169] nix-daemon[855]: reaped child process 1008, status = succeeded worker1 # [ 73.588663] nix-daemon[855]: accepted connection from pid 1024, user nix-grpc-daemon (trusted) worker1 # [ 73.603971] nix-daemon[855]: reaped child process 1027, status = succeeded lb # [ 73.674688] niks3-server[1016]: 2026/09/23 05:34:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 73.624409] nix-grpc-daemon[869]: time=2026-09-23T05:34:06.258Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 73.626112] nix-grpc-daemon[869]: time=2026-09-23T05:34:06.258Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" worker1 # [ 73.669451] nix-grpc-daemon[869]: time=2026-09-23T05:34:06.304Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.722000] niks3-server[1016]: 2026/09/23 05:34:06 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 73.736212] niks3-server[1016]: 2026/09/23 05:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 73.746630] niks3-server[1016]: 2026/09/23 05:34:06 INFO Signed narinfos id=8 count=1 lb # [ 73.752277] niks3-server[1016]: 2026/09/23 05:34:06 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls worker1 # [ 73.703147] nix-grpc-daemon[869]: time=2026-09-23T05:34:06.338Z level=INFO msg="Upload complete. (283ms)" lb # [ 73.765173] niks3-server[1016]: 2026/09/23 05:34:06 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 73.778893] niks3-server[1016]: 2026/09/23 05:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete worker1 # [ 73.717061] nix-grpc-daemon[769]: ts=2026-09-23T05:34:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 lb # [ 73.784969] niks3-server[1016]: 2026/09/23 05:34:06 INFO Completed upload id=8 lb # [ 73.787717] niks3-server[1016]: 2026/09/23 05:34:06 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 73.734478] nix-daemon[855]: reaped child process 1006, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.40 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.90 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 74.297298] nix-grpc-daemon[772]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 74.305166] nix-grpc-daemon[772]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 74.315005] nix-grpc-daemon[769]: ts=2026-09-23T05:34:06Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker2 # [ 74.328324] nix-grpc-daemon[772]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 74.321117] nix-grpc-daemon[769]: ts=2026-09-23T05:34:06Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 74.343198] nix-grpc-daemon[772]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 74.359397] nix-grpc-daemon[772]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 74.374855] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 74.403504] nix-grpc-daemon[772]: ts=2026-09-23T05:34:07Z level=info event=niks3_push_started pid=869 worker2 # [ 74.571911] nix-grpc-daemon[869]: time=2026-09-23T05:34:07.199Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 74.685056] nix-daemon[844]: accepted connection from pid 882, user nix-grpc-daemon (trusted) worker2 # [ 74.702208] nix-daemon[844]: reaped child process 885, status = succeeded lb # [ 74.762622] niks3-server[1016]: 2026/09/23 05:34:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 74.722061] nix-grpc-daemon[869]: time=2026-09-23T05:34:07.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 74.725424] nix-grpc-daemon[869]: time=2026-09-23T05:34:07.349Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 74.805217] niks3-server[1016]: 2026/09/23 05:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign worker2 # [ 74.761502] nix-grpc-daemon[869]: time=2026-09-23T05:34:07.388Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.818534] niks3-server[1016]: 2026/09/23 05:34:07 INFO Signed narinfos id=9 count=1 lb # [ 74.825524] niks3-server[1016]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 74.836660] niks3-server[1016]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst worker2 # [ 74.794327] nix-grpc-daemon[869]: time=2026-09-23T05:34:07.421Z level=INFO msg="Upload complete. (217ms)" lb # [ 74.848779] niks3-server[1016]: 2026/09/23 05:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker2 # [ 74.800389] nix-grpc-daemon[772]: ts=2026-09-23T05:34:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 74.854779] niks3-server[1016]: 2026/09/23 05:34:07 INFO Completed upload id=9 lb # [ 74.858112] niks3-server[1016]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 74.813002] nix-grpc-daemon[772]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 74.818366] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 74.823004] nix-daemon[844]: reaped child process 867, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker2 # [ 74.833199] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 75.663551] nix-daemon[844]: reaped child process 889, status = succeeded worker2 # [ 75.759485] nix-daemon[844]: accepted connection from pid 905, user nix-grpc-daemon (trusted) worker2 # [ 75.782339] nix-daemon[844]: reaped child process 908, status = succeeded lb # [ 75.832680] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 75.792539] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 75.794032] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.419Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 75.883337] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 75.898390] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker2 # [ 75.853296] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.481Z level=INFO msg="Uploading 1 narinfos" lb # [ 75.908322] niks3-server[1016]: 2026/09/23 05:34:08 INFO Signed narinfos id=10 count=1 lb # [ 75.910380] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 75.922060] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 75.931239] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker2 # [ 75.886068] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.514Z level=INFO msg="Upload complete. (241ms)" lb # [ 75.939675] niks3-server[1016]: 2026/09/23 05:34:08 INFO Completed upload id=10 lb # [ 75.943030] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker2 # [ 75.897157] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker1 # [ 75.895123] nix-grpc-daemon[769]: ts=2026-09-23T05:34:08Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 75.901148] nix-grpc-daemon[769]: ts=2026-09-23T05:34:08Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 75.917610] nix-daemon[844]: reaped child process 887, status = succeeded worker2 # [ 75.953175] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 75.973722] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 75.991755] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.619Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 76.005113] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 76.020139] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 76.206535] nix-daemon[844]: accepted connection from pid 912, user nix-grpc-daemon (trusted) worker2 # [ 76.227732] nix-daemon[844]: reaped child process 915, status = succeeded lb # [ 76.286415] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 76.244115] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.872Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 76.245607] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.872Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 76.320449] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls worker2 # [ 76.280112] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.907Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.333084] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 76.345706] niks3-server[1016]: 2026/09/23 05:34:08 INFO Signed narinfos id=11 count=1 lb # [ 76.351683] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 76.365293] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 76.371062] niks3-server[1016]: 2026/09/23 05:34:09 INFO Completed upload id=11 worker2 # [ 76.318887] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.947Z level=INFO msg="Upload complete. (281ms)" lb # [ 76.377361] niks3-server[1016]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 76.328348] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 76.343924] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 76.350308] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 76.356490] nix-daemon[844]: reaped child process 910, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker2 # [ 76.369618] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 77.868595] nix-daemon[844]: reaped child process 919, status = succeeded worker2 # [ 77.936348] nix-daemon[844]: accepted connection from pid 935, user nix-grpc-daemon (trusted) worker2 # [ 77.953960] nix-daemon[844]: reaped child process 938, status = succeeded lb # [ 78.013611] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 77.973057] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 77.974606] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.600Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 78.052904] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker2 # [ 78.012182] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.640Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.069985] niks3-server[1016]: 2026/09/23 05:34:10 INFO Signed narinfos id=12 count=1 lb # [ 78.078273] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 78.090841] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker2 # [ 78.050556] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.678Z level=INFO msg="Upload complete. (267ms)" lb # [ 78.102889] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 78.106670] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 78.111633] niks3-server[1016]: 2026/09/23 05:34:10 INFO Completed upload id=12 lb # [ 78.115017] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker1 # [ 78.061084] nix-grpc-daemon[769]: ts=2026-09-23T05:34:10Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 78.061106] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker2 # [ 78.080332] nix-daemon[844]: reaped child process 917, status = succeeded worker1 # [ 78.066931] nix-grpc-daemon[769]: ts=2026-09-23T05:34:10Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 78.116914] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 78.132311] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 78.145702] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.773Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 78.156615] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 78.170163] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 78.324328] nix-daemon[844]: accepted connection from pid 942, user nix-grpc-daemon (trusted) worker2 # [ 78.342951] nix-daemon[844]: reaped child process 945, status = succeeded lb # [ 78.401414] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 78.358104] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 78.360264] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.988Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 78.435267] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker2 # [ 78.390905] nix-grpc-daemon[869]: time=2026-09-23T05:34:11.018Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.449484] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 78.462066] niks3-server[1016]: 2026/09/23 05:34:11 INFO Signed narinfos id=13 count=1 worker2 # [ 78.415672] nix-grpc-daemon[869]: time=2026-09-23T05:34:11.043Z level=INFO msg="Upload complete. (229ms)" lb # [ 78.468236] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 78.471999] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 78.478578] niks3-server[1016]: 2026/09/23 05:34:11 INFO Completed upload id=13 lb # [ 78.482418] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 78.424345] nix-grpc-daemon[772]: ts=2026-09-23T05:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 78.438912] nix-grpc-daemon[772]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 78.444819] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 78.449719] nix-daemon[844]: reaped child process 940, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker2 # [ 78.462495] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 79.392837] envoy[907]: conn peer=[2001:db8:1::1]:60042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=4414 ms=6167 lb # [ 79.397961] envoy[907]: conn peer=[2001:db8:1::1]:60056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=56 lb # [ 79.402656] envoy[907]: conn peer=[2001:db8:1::1]:41948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1895 ms=2159 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 80.152730] nix-daemon[844]: reaped child process 949, status = succeeded worker2 # [ 80.248942] nix-daemon[844]: accepted connection from pid 965, user nix-grpc-daemon (trusted) worker2 # [ 80.267067] nix-daemon[844]: reaped child process 968, status = succeeded lb # [ 80.326736] niks3-server[1016]: 2026/09/23 05:34:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 80.287291] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.913Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 80.289157] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.913Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" worker2 # [ 80.324505] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.952Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.366336] niks3-server[1016]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 80.382609] niks3-server[1016]: 2026/09/23 05:34:13 INFO Signed narinfos id=14 count=1 lb # [ 80.389857] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 80.400305] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 80.403429] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker2 # [ 80.355385] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.983Z level=INFO msg="Upload complete. (280ms)" lb # [ 80.411107] niks3-server[1016]: 2026/09/23 05:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 80.415031] niks3-server[1016]: 2026/09/23 05:34:13 INFO Completed upload id=14 lb # [ 80.418501] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 80.369170] nix-grpc-daemon[772]: ts=2026-09-23T05:34:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 80.386190] nix-daemon[844]: reaped child process 947, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.31 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 80.729119] nix-grpc-daemon[769]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:42728 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 80.776869] nix-grpc-daemon[769]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:42728 worker1 # [ 80.786310] nix-grpc-daemon[769]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 80.803266] nix-grpc-daemon[769]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 80.819337] nix-grpc-daemon[769]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 81.050397] nix-grpc-daemon[772]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:40828 worker2 # [ 81.092924] nix-grpc-daemon[772]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:40828 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/905" client # removing stale temporary roots file "/nix/var/nix/temproots/864" client # removing stale temporary roots file "/nix/var/nix/temproots/885" client # removing stale temporary roots file "/nix/var/nix/temproots/926" client # removing stale temporary roots file "/nix/var/nix/temproots/918" client # removing stale temporary roots file "/nix/var/nix/temproots/849" client # removing stale temporary roots file "/nix/var/nix/temproots/897" client # removing stale temporary roots file "/nix/var/nix/temproots/941" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 81.479706] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.486532] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.501596] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.135Z level=INFO msg="All 1 paths already cached" worker2 # [ 81.518417] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 81.510162] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying 3 paths... worker1 # [ 81.524957] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 81.561484] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 81.569208] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 81.571093] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.579910] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 81.589790] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.594912] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.54 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.60 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 81.743113] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: e384347af7454a1e9eff4cadf0c2e3ff client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 81.926800] nix[994]: don't know how to build these paths: client # [ 81.927800] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 81.917019] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.930517] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 81.937880] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 81.978167] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 81.993356] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 client # [ 82.016124] nix[994]: copying 1 paths to 'lb:50051'... worker1 # [ 82.007716] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 client # [ 82.018425] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 82.030226] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 82.197872] nix-daemon[855]: accepted connection from pid 1047, user nix-grpc-daemon (trusted) worker1 # [ 82.219712] nix-daemon[855]: reaped child process 1050, status = succeeded lb # [ 82.286787] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.235573] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 82.237337] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.870Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 82.317972] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 82.266640] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.901Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.331996] niks3-server[1016]: 2026/09/23 05:34:14 INFO Signed narinfos id=15 count=1 lb # [ 82.340804] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker1 # [ 82.294004] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.929Z level=INFO msg="Upload complete. (243ms)" lb # [ 82.356971] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 82.366315] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete worker1 # [ 82.304531] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 82.370962] niks3-server[1016]: 2026/09/23 05:34:14 INFO Completed upload id=15 lb # [ 82.375650] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 82.316278] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 82.326857] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 82.329758] nix-daemon[855]: reaped child process 1045, status = succeeded client # [ 82.348169] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 82.343518] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # [ 82.830496] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: e3cdea24b11544de9131a083f34115dd client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds) worker2 # [ 83.033364] nix-grpc-daemon[772]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40618 client # [ 83.037096] nix[1010]: don't know how to build these paths: client # [ 83.038147] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 83.041345] nix-grpc-daemon[769]: ts=2026-09-23T05:34:15Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 83.057374] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 83.048772] nix-grpc-daemon[769]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 86.408756] nix-daemon[855]: reaped child process 1054, status = succeeded worker1 # [ 86.502253] nix-daemon[855]: accepted connection from pid 1086, user nix-grpc-daemon (trusted) worker1 # [ 86.518863] nix-daemon[855]: reaped child process 1089, status = succeeded lb # [ 86.591574] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 86.540397] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 86.542041] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.173Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 86.647464] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 86.594877] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.230Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.660877] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 86.670433] niks3-server[1016]: 2026/09/23 05:34:19 INFO Signed narinfos id=16 count=1 lb # [ 86.676841] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 86.688344] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 86.691855] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 86.698558] niks3-server[1016]: 2026/09/23 05:34:19 INFO Completed upload id=16 worker1 # [ 86.635794] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.271Z level=INFO msg="Upload complete. (279ms)" lb # [ 86.701959] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 86.653137] nix-grpc-daemon[769]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 86.670195] nix-grpc-daemon[769]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 86.678033] nix-daemon[855]: reaped child process 1052, status = succeeded client # [ 86.691907] systemd[1]: dedup1.service: Deactivated successfully. client # [ 86.697189] systemd[1]: dedup1.service: Consumed 171ms CPU time over 4.950s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 86.736640] systemd[1]: dedup2.service: Deactivated successfully. client # [ 86.739516] systemd[1]: dedup2.service: Consumed 182ms CPU time over 3.909s wall clock time, 33.3M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 88.016971] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 88.020286] nix-grpc-daemon[769]: ts=2026-09-23T05:34:20Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 88.031697] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 88.020630] nix-grpc-daemon[769]: ts=2026-09-23T05:34:20Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 88.047024] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 88.061453] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 88.076788] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 88.238982] nix-daemon[844]: accepted connection from pid 993, user nix-grpc-daemon (trusted) worker2 # [ 88.260612] nix-daemon[844]: reaped child process 996, status = succeeded lb # [ 88.311755] niks3-server[1016]: 2026/09/23 05:34:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 88.269961] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.897Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 88.272119] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.899Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 88.346061] niks3-server[1016]: 2026/09/23 05:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 88.302869] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.930Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.360096] niks3-server[1016]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 88.370623] niks3-server[1016]: 2026/09/23 05:34:20 INFO Signed narinfos id=17 count=1 lb # [ 88.382320] niks3-server[1016]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 88.336282] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.964Z level=INFO msg="Upload complete. (243ms)" lb # [ 88.393833] niks3-server[1016]: 2026/09/23 05:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker2 # [ 88.344235] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 88.399696] niks3-server[1016]: 2026/09/23 05:34:21 INFO Completed upload id=17 lb # [ 88.402720] niks3-server[1016]: 2026/09/23 05:34:21 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 88.360106] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 88.370556] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 88.373473] nix-daemon[844]: reaped child process 991, status = succeeded worker2 # [ 88.387717] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) lb # [ 89.390562] envoy[907]: conn peer=[2001:db8:1::1]:41950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4378 ms=6085 lb # [ 89.395818] envoy[907]: conn peer=[2001:db8:1::1]:33534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=122 lb # [ 89.400961] envoy[907]: conn peer=[2001:db8:1::1]:33556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=88 lb # [ 89.405718] envoy[907]: conn peer=[2001:db8:1::1]:33550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=107 lb # [ 89.410360] envoy[907]: conn peer=[2001:db8:1::1]:33518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2915 ms=196 lb # [ 89.415168] envoy[907]: conn peer=[2001:db8:1::1]:33564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1539 ms=4747 lb # [ 89.420784] envoy[907]: conn peer=[2001:db8:1::1]:33566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=3695 worker2 # [ 94.804845] nix-grpc-daemon[772]: ts=2026-09-23T05:34:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 94.845452] nix-daemon[844]: reaped child process 998, status = succeeded worker2 # [ 94.847893] nix-daemon[844]: reaped child process 1000, status = succeeded worker1 # [ 95.039255] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 95.053949] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 95.063747] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 95.072795] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 95.080290] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 95.090344] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 95.105050] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 95.256056] nix-daemon[855]: accepted connection from pid 1106, user nix-grpc-daemon (trusted) worker1 # [ 95.273537] nix-daemon[855]: reaped child process 1109, status = succeeded lb # [ 95.343183] niks3-server[1016]: 2026/09/23 05:34:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 95.291682] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 95.293693] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.928Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 95.383233] niks3-server[1016]: 2026/09/23 05:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 95.332691] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.968Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.400965] niks3-server[1016]: 2026/09/23 05:34:28 INFO Signed narinfos id=18 count=1 lb # [ 95.407437] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 95.414893] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst worker1 # [ 95.364175] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.999Z level=INFO msg="Upload complete. (244ms)" lb # [ 95.425994] niks3-server[1016]: 2026/09/23 05:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 95.430801] niks3-server[1016]: 2026/09/23 05:34:28 INFO Completed upload id=18 lb # [ 95.433748] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 95.376130] nix-grpc-daemon[769]: ts=2026-09-23T05:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 95.391283] nix-grpc-daemon[769]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 95.398508] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 95.404245] nix-daemon[855]: reaped child process 1104, status = succeeded worker1 # [ 95.419307] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 99.386161] envoy[907]: conn peer=[2001:db8:1::1]:33574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=953 ms=6819 worker1 # [ 101.034100] nix-grpc-daemon[769]: ts=2026-09-23T05:34:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 101.051620] nix-daemon[855]: reaped child process 1111, status = succeeded worker1 # [ 101.052918] nix-daemon[855]: reaped child process 1113, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.48 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 101.330822] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: c198c844f6e246449ac97f7a576ed51e client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 101.537979] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 client # [ 101.553224] nix[1083]: don't know how to build these paths: client # [ 101.554192] nix[1083]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 101.556845] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 101.563158] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 101.578257] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 101.593751] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 client # [ 101.603484] nix[1083]: copying 1 paths to 'lb:50051'... client # [ 101.606410] nix[1083]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 101.609644] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 101.624359] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 101.781870] nix-daemon[844]: accepted connection from pid 1023, user nix-grpc-daemon (trusted) worker2 # [ 101.799919] nix-daemon[844]: reaped child process 1026, status = succeeded lb # [ 101.859267] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.817314] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.445Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 101.818855] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.445Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 101.886441] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 101.842236] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.470Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.900095] niks3-server[1016]: 2026/09/23 05:34:34 INFO Signed narinfos id=19 count=1 lb # [ 101.909291] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 101.920073] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 101.932117] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 101.941906] niks3-server[1016]: 2026/09/23 05:34:34 INFO Completed upload id=19 lb # [ 101.946623] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 101.889398] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.517Z level=INFO msg="Upload complete. (248ms)" worker2 # [ 101.900535] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 101.918342] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 101.928787] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 101.932355] nix-daemon[844]: reaped child process 1021, status = succeeded client # [ 101.942304] nix[1083]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 101.947885] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # [ 102.445314] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 8d0a91c46f4741c49c0a9a3d6e81ad72 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds) worker1 # [ 102.667510] nix-grpc-daemon[769]: ts=2026-09-23T05:34:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 client # [ 102.683275] nix[1099]: don't know how to build these paths: client # [ 102.685137] nix[1099]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 102.698674] nix-grpc-daemon[772]: ts=2026-09-23T05:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 102.685524] nix-grpc-daemon[769]: ts=2026-09-23T05:34:35Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 client # [ 102.700540] nix[1099]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 105.817467] systemd[1]: share1.service: Sent signal SIGINT to main process 1083 (nix) on client request. client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.08 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 105.827663] nix[1083]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 105.830238] nix[1083]: error: interrupted by the user client # [ 105.899444] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 105.908633] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 105.910370] systemd[1]: share1.service: Consumed 179ms CPU time over 4.575s wall clock time, 43.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.14 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 107.399457] nix-daemon[844]: reaped child process 1030, status = succeeded worker2 # [ 107.526853] nix-daemon[844]: accepted connection from pid 1064, user nix-grpc-daemon (trusted) worker2 # [ 107.545928] nix-daemon[844]: reaped child process 1067, status = succeeded lb # [ 107.603798] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 107.563356] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 107.565040] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.189Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 107.642449] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker2 # [ 107.598134] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.226Z level=INFO msg="Uploading 1 narinfos" lb # [ 107.657084] niks3-server[1016]: 2026/09/23 05:34:40 INFO Signed narinfos id=20 count=1 lb # [ 107.663448] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 107.669133] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 107.674549] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 107.683659] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 107.690132] niks3-server[1016]: 2026/09/23 05:34:40 INFO Completed upload id=20 worker2 # [ 107.637754] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.266Z level=INFO msg="Upload complete. (276ms)" lb # [ 107.692902] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 107.649202] nix-grpc-daemon[772]: ts=2026-09-23T05:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 107.668374] nix-grpc-daemon[772]: ts=2026-09-23T05:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 107.676485] nix-daemon[844]: reaped child process 1028, status = succeeded client # [ 107.725759] systemd[1]: share2.service: Deactivated successfully. client # [ 107.730890] systemd[1]: share2.service: Consumed 202ms CPU time over 5.281s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.13 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 108.537951] systemd[1]: Started Nix Daemon. client # [ 108.651677] nix-daemon[1151]: accepted connection from pid 1148, user root (trusted) worker1 # [ 108.792583] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 108.798860] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 108.816134] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 108.823586] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 108.832998] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 108.844612] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 108.859642] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 108.874766] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 109.035783] nix-daemon[855]: accepted connection from pid 1158, user nix-grpc-daemon (trusted) worker1 # [ 109.054531] nix-daemon[855]: reaped child process 1161, status = succeeded lb # [ 109.125292] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 109.073311] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.708Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 109.075430] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.708Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 109.155865] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst worker1 # [ 109.107034] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.742Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.170219] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 109.183119] niks3-server[1016]: 2026/09/23 05:34:41 INFO Signed narinfos id=21 count=1 lb # [ 109.190882] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls worker1 # [ 109.140125] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.775Z level=INFO msg="Upload complete. (250ms)" lb # [ 109.204801] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 109.212917] niks3-server[1016]: 2026/09/23 05:34:41 INFO Completed upload id=21 worker1 # [ 109.152336] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 109.218180] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 109.168151] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 109.175329] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 109.181677] nix-daemon[855]: reaped child process 1156, status = succeeded worker1 # [ 109.198125] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 109.391201] envoy[907]: conn peer=[2001:db8:1::1]:43720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=951 ms=6056 lb # [ 109.396332] envoy[907]: conn peer=[2001:db8:1::1]:46206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2986 tx=773 ms=4295 lb # [ 109.401105] envoy[907]: conn peer=[2001:db8:1::1]:46218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=5030 worker1 # [ 110.063908] nix-daemon[855]: reaped child process 1165, status = succeeded worker1 # [ 110.173481] nix-daemon[855]: accepted connection from pid 1181, user nix-grpc-daemon (trusted) worker1 # [ 110.191909] nix-daemon[855]: reaped child process 1184, status = succeeded lb # [ 110.253724] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 110.204431] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 110.205995] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.837Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 110.298061] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker1 # [ 110.244817] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.880Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.312751] niks3-server[1016]: 2026/09/23 05:34:42 INFO Signed narinfos id=22 count=1 lb # [ 110.315444] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 110.335854] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls worker1 # [ 110.279562] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.915Z level=INFO msg="Upload complete. (270ms)" lb # [ 110.343426] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 110.348701] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 110.353973] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo lb # [ 110.358133] niks3-server[1016]: 2026/09/23 05:34:42 INFO Completed upload id=22 worker1 # [ 110.293086] nix-grpc-daemon[769]: ts=2026-09-23T05:34:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker1 # [ 110.308950] nix-daemon[855]: reaped child process 1163, status = succeeded worker2 # [ 110.346457] nix-grpc-daemon[772]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 110.360789] nix-grpc-daemon[772]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 110.401763] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 110.408561] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 110.698451] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50732 worker2 # [ 110.712861] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 110.724825] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.352Z level=INFO msg="All 1 paths already cached" worker1 # [ 110.732729] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 110.740084] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 110.749358] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 110.760908] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 110.775586] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.411Z level=INFO msg="All 1 paths already cached" worker1 # [ 110.786016] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 110.800442] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 110.966250] nix-daemon[855]: accepted connection from pid 1190, user nix-grpc-daemon (trusted) worker1 # [ 110.987633] nix-daemon[855]: reaped child process 1193, status = succeeded lb # [ 111.046853] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 110.994336] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.629Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 110.995801] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.629Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 111.078184] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 111.028206] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.663Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.093855] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 111.105130] niks3-server[1016]: 2026/09/23 05:34:43 INFO Signed narinfos id=23 count=1 lb # [ 111.115541] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls worker1 # [ 111.061706] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.697Z level=INFO msg="Upload complete. (244ms)" lb # [ 111.126501] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 111.133026] niks3-server[1016]: 2026/09/23 05:34:43 INFO Completed upload id=23 worker1 # [ 111.070646] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 111.136861] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 111.088253] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 111.095591] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 111.101032] nix-daemon[855]: reaped child process 1188, status = succeeded worker1 # [ 111.114779] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 112.208483] nix-daemon[855]: reaped child process 1197, status = succeeded worker1 # [ 112.296269] nix-daemon[855]: accepted connection from pid 1213, user nix-grpc-daemon (trusted) worker1 # [ 112.314925] nix-daemon[855]: reaped child process 1216, status = succeeded lb # [ 112.381638] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 112.332431] nix-grpc-daemon[869]: time=2026-09-23T05:34:44.964Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 112.334116] nix-grpc-daemon[869]: time=2026-09-23T05:34:44.964Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 112.425441] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign worker1 # [ 112.371485] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.006Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.438744] niks3-server[1016]: 2026/09/23 05:34:45 INFO Signed narinfos id=24 count=1 lb # [ 112.446841] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 112.456793] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 112.461164] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 112.468882] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete worker1 # [ 112.406462] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.042Z level=INFO msg="Upload complete. (280ms)" lb # [ 112.471964] niks3-server[1016]: 2026/09/23 05:34:45 INFO Completed upload id=24 lb # [ 112.475322] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 112.420719] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 112.439265] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 112.445371] nix-daemon[855]: reaped child process 1195, status = succeeded worker2 # [ 112.479918] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 112.487529] nix-grpc-daemon[772]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 112.505556] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 112.788882] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 112.796290] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 112.804319] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.430Z level=INFO msg="All 2 paths already cached" worker1 # [ 112.806610] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 112.814696] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 112.823857] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 112.835029] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 112.849816] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.485Z level=INFO msg="All 2 paths already cached" worker1 # [ 112.859959] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 112.874684] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 113.027239] nix-daemon[855]: accepted connection from pid 1222, user nix-grpc-daemon (trusted) worker1 # [ 113.046128] nix-daemon[855]: reaped child process 1225, status = succeeded lb # [ 113.114059] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 113.062382] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.697Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 113.063953] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.697Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 113.142692] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 113.086644] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.722Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.155331] niks3-server[1016]: 2026/09/23 05:34:45 INFO Signed narinfos id=25 count=1 lb # [ 113.164575] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 113.173771] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst worker1 # [ 113.126677] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.762Z level=INFO msg="Upload complete. (235ms)" lb # [ 113.190397] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 113.200712] niks3-server[1016]: 2026/09/23 05:34:45 INFO Completed upload id=25 worker1 # [ 113.136392] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 113.203749] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 113.154986] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 113.164363] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 113.167527] nix-daemon[855]: reaped child process 1220, status = succeeded worker1 # [ 113.181290] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 114.426926] nix-daemon[855]: reaped child process 1229, status = succeeded worker1 # [ 114.518390] nix-daemon[855]: accepted connection from pid 1245, user nix-grpc-daemon (trusted) worker1 # [ 114.533510] nix-daemon[855]: reaped child process 1248, status = succeeded lb # [ 114.603178] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 114.551740] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 114.553386] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.186Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" worker1 # [ 114.592595] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.227Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.642944] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 114.658069] niks3-server[1016]: 2026/09/23 05:34:47 INFO Signed narinfos id=26 count=1 lb # [ 114.667451] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 114.677795] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 114.688692] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst worker1 # [ 114.632268] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.267Z level=INFO msg="Upload complete. (238ms)" lb # [ 114.697300] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 114.703344] niks3-server[1016]: 2026/09/23 05:34:47 INFO Completed upload id=26 lb # [ 114.707018] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 114.646219] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 114.665640] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 114.671211] nix-daemon[855]: reaped child process 1227, status = succeeded worker1 # [ 114.692533] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 114.699244] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=232 client # [ 114.836573] nix-daemon[1151]: reaped child process 1153, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.46 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.58 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 115.801444] nix-daemon[1151]: accepted connection from pid 1233, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 115.952340] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 115.958871] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 115.966776] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 115.980320] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 116.157086] nix-daemon[855]: accepted connection from pid 1257, user nix-grpc-daemon (trusted) worker1 # [ 116.178078] nix-daemon[855]: reaped child process 1260, status = succeeded lb # [ 116.239574] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 116.188274] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.823Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 116.189984] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.823Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" worker1 # [ 116.215743] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.851Z level=INFO msg="Uploading 1 narinfos" lb # [ 116.269480] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 116.285431] niks3-server[1016]: 2026/09/23 05:34:48 INFO Signed narinfos id=27 count=1 lb # [ 116.291787] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 116.244488] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.880Z level=INFO msg="Upload complete. (249ms)" lb # [ 116.306353] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 116.310494] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 116.315036] niks3-server[1016]: 2026/09/23 05:34:48 INFO Completed upload id=27 worker1 # [ 116.256144] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=120 lb # [ 116.318558] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 116.269586] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker2 # [ 116.283301] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 116.276800] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 116.281669] nix-daemon[855]: reaped child process 1255, status = succeeded worker2 # [ 116.298253] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying 2 paths to 'lb:50051'... worker2 # [ 116.317193] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 116.334113] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 116.498073] nix-daemon[844]: accepted connection from pid 1076, user nix-grpc-daemon (trusted) worker2 # [ 116.515495] nix-daemon[844]: reaped child process 1079, status = succeeded lb # [ 116.575171] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 116.584165] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 116.542726] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 116.545605] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 116.548211] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 116.616597] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 116.636839] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 116.648279] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign worker2 # [ 116.602946] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.230Z level=INFO msg="Uploading 3 narinfos" lb # [ 116.657324] niks3-server[1016]: 2026/09/23 05:34:49 INFO Signed narinfos id=29 count=2 lb # [ 116.661879] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 116.668035] niks3-server[1016]: 2026/09/23 05:34:49 INFO Signed narinfos id=28 count=1 lb # [ 116.669742] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 116.677560] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 116.690491] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 116.698214] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 116.701622] niks3-server[1016]: 2026/09/23 05:34:49 INFO Completed upload id=28 lb # [ 116.703468] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 116.708382] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker2 # [ 116.656412] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.284Z level=INFO msg="Upload complete. (304ms)" lb # [ 116.710732] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 116.713240] niks3-server[1016]: 2026/09/23 05:34:49 INFO Completed upload id=29 worker2 # [ 116.667344] nix-grpc-daemon[772]: ts=2026-09-23T05:34:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 116.680688] nix-grpc-daemon[772]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 116.696195] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 116.703686] nix-daemon[844]: reaped child process 1074, status = succeeded worker2 # [ 116.720953] nix-grpc-daemon[772]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 116.741542] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 117.846019] nix-daemon[844]: reaped child process 1084, status = succeeded worker2 # [ 117.916144] nix-daemon[844]: accepted connection from pid 1100, user nix-grpc-daemon (trusted) worker2 # [ 117.933306] nix-daemon[844]: reaped child process 1103, status = succeeded lb # [ 117.991886] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 117.950511] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.577Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 118.019955] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 117.979383] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.607Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.035345] niks3-server[1016]: 2026/09/23 05:34:50 INFO Signed narinfos id=30 count=1 lb # [ 118.043234] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 118.054593] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls worker2 # [ 118.008139] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.636Z level=INFO msg="Upload complete. (253ms)" lb # [ 118.060840] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 118.070292] niks3-server[1016]: 2026/09/23 05:34:50 INFO Completed upload id=30 lb # [ 118.073562] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 118.019968] nix-grpc-daemon[772]: ts=2026-09-23T05:34:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 118.033629] nix-daemon[844]: reaped child process 1081, status = succeeded client # copying 1 paths... worker2 # [ 118.066243] nix-grpc-daemon[772]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker1 # [ 118.091245] nix-grpc-daemon[769]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 118.099243] nix-grpc-daemon[769]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... worker1 # [ 118.115264] nix-grpc-daemon[769]: ts=2026-09-23T05:34:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=120 client # [ 118.248509] nix-daemon[1151]: reaped child process 1236, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.65 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 118.422474] nix-daemon[1151]: accepted connection from pid 1255, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 118.560678] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50732 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 118.577258] nix-grpc-daemon[772]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 118.586810] nix-grpc-daemon[869]: time=2026-09-23T05:34:51.215Z level=INFO msg="All 1 paths already cached" worker1 # [ 118.590172] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 118.601083] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 118.610369] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 118.618806] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 118.632775] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 118.647663] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 118.667251] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5) worker1 # [ 118.737592] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 2/5) worker1 # [ 118.906440] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 210 ms (attempt 3/5) worker1 # [ 119.119495] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 370 ms (attempt 4/5) lb # [ 119.389797] envoy[907]: conn peer=[2001:db8:1::1]:46230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=71 lb # [ 119.394681] envoy[907]: conn peer=[2001:db8:1::1]:46222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1943 ms=1667 lb # [ 119.399531] envoy[907]: conn peer=[2001:db8:1::1]:46234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3913 tx=2155 ms=1857 lb # [ 119.404395] envoy[907]: conn peer=[2001:db8:1::1]:55264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=91 lb # [ 119.408955] envoy[907]: conn peer=[2001:db8:1::1]:55276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=68 lb # [ 119.413534] envoy[907]: conn peer=[2001:db8:1::1]:55266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4337 tx=2390 ms=1980 lb # [ 119.418367] envoy[907]: conn peer=[2001:db8:1::1]:55286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4278 tx=2076 ms=2233 lb # [ 119.423174] envoy[907]: conn peer=[2001:db8:1::1]:55300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=90 worker1 # [ 119.491157] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 119.497359] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 119.505439] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 119.682017] nix-daemon[855]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker1 # [ 119.702963] nix-daemon[855]: reaped child process 1277, status = succeeded lb # [ 119.769856] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.717269] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.352Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 119.719523] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.352Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 119.800413] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 119.812811] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker1 # [ 119.757025] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.392Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.825185] niks3-server[1016]: 2026/09/23 05:34:52 INFO Signed narinfos id=31 count=1 lb # [ 119.831542] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 119.846799] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete worker1 # [ 119.784467] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.420Z level=INFO msg="Upload complete. (259ms)" lb # [ 119.850462] niks3-server[1016]: 2026/09/23 05:34:52 INFO Completed upload id=31 lb # [ 119.853407] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 119.796668] nix-grpc-daemon[769]: ts=2026-09-23T05:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 paths=1 nar_bytes_in=568 worker1 # [ 119.808737] nix-grpc-daemon[769]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 119.822918] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker1 # [ 119.828572] nix-daemon[855]: reaped child process 1267, status = succeeded worker1 # [ 119.840545] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 120.938401] nix-daemon[855]: reaped child process 1281, status = succeeded worker1 # [ 121.003954] nix-daemon[855]: accepted connection from pid 1297, user nix-grpc-daemon (trusted) worker1 # [ 121.024380] nix-daemon[855]: reaped child process 1300, status = succeeded lb # [ 121.085928] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.034574] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.668Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 121.111928] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 121.127220] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker1 # [ 121.073674] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.709Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.141948] niks3-server[1016]: 2026/09/23 05:34:53 INFO Signed narinfos id=32 count=1 lb # [ 121.147635] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 121.099441] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.735Z level=INFO msg="Upload complete. (246ms)" lb # [ 121.163062] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 121.171552] niks3-server[1016]: 2026/09/23 05:34:53 INFO Completed upload id=32 worker1 # [ 121.109755] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 lb # [ 121.176191] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 121.124336] nix-daemon[855]: reaped child process 1279, status = succeeded client # copying 1 paths... worker1 # [ 121.129861] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 121.159625] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 121.166448] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=120 client # [ 121.302505] nix-daemon[1151]: reaped child process 1258, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.05 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 121.473973] nix-daemon[1151]: accepted connection from pid 1277, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 121.613021] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50732 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 121.620469] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 121.627586] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.261Z level=INFO msg="All 1 paths already cached" worker1 # [ 121.638952] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 121.653089] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker1 # [ 121.645625] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 121.668453] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40618 client # copying 1 paths to 'lb:50051'... worker2 # [ 121.685101] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.310Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 121.690425] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 121.705773] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 121.864195] nix-daemon[844]: accepted connection from pid 1107, user nix-grpc-daemon (trusted) worker2 # [ 121.882550] nix-daemon[844]: reaped child process 1110, status = succeeded lb # [ 121.940792] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 121.897641] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.525Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 121.899111] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.525Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 121.970594] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker2 # [ 121.925089] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.553Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.985991] niks3-server[1016]: 2026/09/23 05:34:54 INFO Signed narinfos id=33 count=1 lb # [ 121.993881] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 121.957958] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.586Z level=INFO msg="Upload complete. (233ms)" lb # [ 122.010969] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 122.016223] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker2 # [ 121.968272] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 122.022428] niks3-server[1016]: 2026/09/23 05:34:54 INFO Completed upload id=33 lb # [ 122.028701] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 121.982981] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 121.989773] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 121.994609] nix-daemon[844]: reaped child process 1105, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 122.007789] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 123.316890] nix-daemon[844]: reaped child process 1114, status = succeeded worker2 # [ 123.385602] nix-daemon[844]: accepted connection from pid 1130, user nix-grpc-daemon (trusted) worker2 # [ 123.403143] nix-daemon[844]: reaped child process 1133, status = succeeded lb # [ 123.467087] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 123.423460] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.050Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 123.494259] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker2 # [ 123.448276] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.076Z level=INFO msg="Uploading 1 narinfos" lb # [ 123.508213] niks3-server[1016]: 2026/09/23 05:34:56 INFO Signed narinfos id=34 count=1 lb # [ 123.516141] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 123.525506] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 123.483515] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.111Z level=INFO msg="Upload complete. (264ms)" lb # [ 123.535940] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 123.545019] niks3-server[1016]: 2026/09/23 05:34:56 INFO Completed upload id=34 client # copying 1 paths... lb # [ 123.548952] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 123.497005] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 123.512411] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 123.518265] nix-daemon[844]: reaped child process 1112, status = succeeded client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 123.544852] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 worker2 # [ 123.551477] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=120 client # [ 123.673447] nix-daemon[1151]: reaped child process 1280, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.37 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 8.71 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds) worker2 # [ 124.665818] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 124.700961] nix-grpc-daemon[772]: ts=2026-09-23T05:34:57Z level=info event=signal_exit worker2 # [ 125.720295] nix-grpc-daemon[772]: ts=2026-09-23T05:34:58Z level=info event=scheduler_restarting worker1 # [ 125.755338] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45520 worker1 # [ 125.763154] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 125.791151] nix-grpc-daemon[772]: ts=2026-09-23T05:34:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 125.811290] nix-daemon[844]: reaped child process 846, status = succeeded worker2 # [ 125.850481] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 125.856705] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 125.858998] systemd[1]: nix-grpc-daemon.service: Consumed 2.746s CPU time over 1min 46.387s wall clock time, 61.6M memory peak, 71.2K incoming IP traffic, 137.2K outgoing IP traffic. worker2 # [ 125.877045] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 125.882820] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 125.884523] systemd[1]: nix-grpc-daemon.socket: Consumed 539us CPU time over 1min 55.754s wall clock time, 256K memory peak, 91.7K incoming IP traffic, 80.7K outgoing IP traffic. lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 126.143514] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50732 worker1 # [ 126.152425] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50732 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.23 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 126.286235] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 126.346885] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 126.602866] nix-grpc-daemon[769]: ts=2026-09-23T05:34:59Z level=info event=signal_exit lb # [ 126.777334] niks3-server[1016]: 2026/09/23 05:34:59 INFO lead: released remote=192.168.1.3:52234 worker2 # [ 126.930658] systemd[1]: Starting scheduler-token.service... worker2 # [ 127.009844] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 127.015891] systemd[1]: Finished scheduler-token.service. worker2 # [ 127.022931] systemd[1]: scheduler-token.service: Consumed 55ms CPU time over 83ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 127.030244] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 127.160595] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 127.162478] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 127.166293] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 127.172057] nix-grpc-daemon[1157]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 127.173498] nix-grpc-daemon[1157]: E0000 00:00:1790141699.799150 1157 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 127.237781] niks3-server[1016]: 2026/09/23 05:34:59 INFO lead: acquired remote=192.168.1.4:52284 worker2 # [ 127.216100] nix-grpc-daemon[1157]: E0000 00:00:1790141699.805345 1157 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 127.218363] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 127.219948] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_take_over worker2 # [ 127.224808] nix-grpc-daemon[1157]: E0000 00:00:1790141699.814636 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 127.227689] nix-grpc-daemon[1157]: E0000 00:00:1790141699.832012 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 127.230370] nix-grpc-daemon[1157]: E0000 00:00:1790141699.832144 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 127.235579] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 127.238400] nix-grpc-daemon[1157]: E0000 00:00:1790141699.852197 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 127.240955] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.243386] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 127.439597] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.445061] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 127.646193] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.651612] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 127.717828] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=scheduler_restarting lb # [ 127.778187] envoy[907]: [2026-09-23 05:35:00.406][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 127.726922] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 127.731306] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45520 worker1 # [ 127.736333] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 127.753163] nix-grpc-daemon[769]: E0000 00:00:1790141700.388595 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker1 # [ 127.772759] nix-daemon[855]: reaped child process 857, status = succeeded worker1 # [ 127.818737] nix-grpc-daemon[769]: E0000 00:00:1790141700.454253 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 127.852786] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.858812] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 127.944891] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 128.059677] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.064918] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.221980] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:59514 worker2 # [ 128.265817] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.282525] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 128.287303] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:59528 worker2 # [ 128.300200] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb # [ 129.388867] envoy[907]: conn peer=[2001:db8:1::1]:55314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=2147 ms=2665 lb # [ 129.393905] envoy[907]: conn peer=[2001:db8:1::1]:57624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=69 lb # [ 129.399509] envoy[907]: conn peer=[2001:db8:1::1]:57638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3770 tx=2101 ms=1985 lb # [ 129.404869] envoy[907]: conn peer=[2001:db8:1::1]:57640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=68 lb # [ 129.409879] envoy[907]: conn peer=192.168.1.4:59178 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=17036 tx=1569 ms=90595 lb # [ 129.414851] envoy[907]: conn peer=[2001:db8:1::1]:57654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=28 lb # [ 129.419621] envoy[907]: conn peer=[2001:db8:1::3]:43278 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5116 tx=1842 ms=92260 worker1 # [ 129.794373] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 129.799125] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 129.807402] systemd[1]: nix-grpc-daemon.service: Consumed 3.944s CPU time over 1min 50.465s wall clock time, 71.8M memory peak, 106.6K incoming IP traffic, 214.8K outgoing IP traffic. worker1 # [ 129.820883] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 129.827792] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 129.830229] systemd[1]: nix-grpc-daemon.socket: Consumed 541us CPU time over 1min 59.836s wall clock time, 256K memory peak, 306.9K incoming IP traffic, 114.3K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 130.145585] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 130.158879] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 130.185655] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5) worker2 # [ 130.206119] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5) worker2 # [ 130.364095] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 3/5) worker2 # [ 130.496958] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 691 ms (attempt 4/5) worker2 # [ 131.199802] nix-grpc-daemon[1157]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 131.206359] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 131.231058] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 131.249797] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 131.282776] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:03Z level=info event=niks3_push_started pid=1218 worker2 # [ 131.429710] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.057Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 131.539633] nix-daemon[844]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 131.557417] nix-daemon[844]: reaped child process 1234, status = succeeded lb # [ 131.616585] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 131.625727] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.583184] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 131.588613] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 131.591003] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 131.683202] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls worker2 # [ 131.642687] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.270Z level=INFO msg="Uploading 3 narinfos" lb # [ 131.695394] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 131.705418] niks3-server[1016]: 2026/09/23 05:35:04 INFO Signed narinfos id=35 count=1 lb # [ 131.712895] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 131.715328] niks3-server[1016]: 2026/09/23 05:35:04 INFO Signed narinfos id=36 count=2 lb # [ 131.725160] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 131.727676] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 131.745717] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 131.750765] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 131.754395] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 131.756342] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 131.759390] niks3-server[1016]: 2026/09/23 05:35:04 INFO Completed upload id=35 lb # [ 131.761742] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 131.763450] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 131.766855] niks3-server[1016]: 2026/09/23 05:35:04 INFO Completed upload id=36 worker2 # [ 131.715187] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.343Z level=INFO msg="Upload complete. (281ms)" worker2 # [ 131.728620] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 131.745335] nix-daemon[844]: reaped child process 1216, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.83 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 131.855177] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 131.868406] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 132.018499] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 132.020840] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 132.024970] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 132.030509] nix-grpc-daemon[1325]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 132.031955] nix-grpc-daemon[1325]: E0000 00:00:1790141704.665579 1325 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 132.068166] nix-grpc-daemon[1325]: E0000 00:00:1790141704.671872 1325 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.070464] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 132.081545] nix-grpc-daemon[1325]: E0000 00:00:1790141704.681660 1368 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 132.084581] nix-grpc-daemon[1325]: E0000 00:00:1790141704.696703 1368 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.086859] nix-grpc-daemon[1325]: E0000 00:00:1790141704.696846 1368 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 132.093585] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 132.096295] nix-grpc-daemon[1325]: E0000 00:00:1790141704.716262 1368 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.098810] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 132.109783] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:59528 worker2 # [ 132.115774] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.44 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 132.241603] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=scheduler_lock_lost status=200 lb # [ 132.307777] niks3-server[1016]: 2026/09/23 05:35:04 INFO Shutdown signal received, draining in-flight requests timeout=10s worker2 # [ 132.260921] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=scheduler_lock_lost status=200 lb # [ 132.323457] niks3-server[1016]: 2026/09/23 05:35:04 INFO lead: released remote=192.168.1.4:52284 lb # [ 132.328931] systemd[1]: Stopping niks3 server... lb # [ 132.333530] systemd[1]: niks3.service: Deactivated successfully. lb # [ 132.334462] systemd[1]: Stopped niks3 server. lb # [ 132.335114] systemd[1]: niks3.service: Consumed 625ms CPU time over 1min 43.795s wall clock time, 27.7M memory peak, 92.8K incoming IP traffic, 135.8K outgoing IP traffic. lb # [ 132.344249] systemd[1]: Starting niks3 server... lb # [ 132.408457] niks3-server[1211]: 2026/09/23 05:35:05 goose: no migrations to run. current version: 20260920000000 lb # [ 132.414186] niks3-server[1211]: 2026/09/23 05:35:05 OK 1_commit_pending_closure.sql (6.51ms) lb # [ 132.417788] niks3-server[1211]: 2026/09/23 05:35:05 OK 2_object_stats_trigger.sql (3.25ms) lb # [ 132.418927] niks3-server[1211]: 2026/09/23 05:35:05 goose: up to current file version: 2 lb # [ 132.420560] niks3-server[1211]: 2026/09/23 05:35:05 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 132.430927] niks3-server[1211]: 2026/09/23 05:35:05 INFO Read proxy enabled — serving cache objects from S3 lb # [ 132.436663] niks3-server[1211]: 2026/09/23 05:35:05 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 132.437984] niks3-server[1211]: 2026/09/23 05:35:05 INFO systemd watchdog enabled interval=15s lb # [ 132.439075] niks3-server[1211]: 2026/09/23 05:35:05 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 132.440466] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 132.861065] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:59546 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 132.872739] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 132.902945] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 132.910299] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 132.948750] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 132.964317] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59546 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 132.980105] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 132.994838] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 133.157213] nix-daemon[844]: accepted connection from pid 1251, user nix-grpc-daemon (trusted) worker2 # [ 133.177061] nix-daemon[844]: reaped child process 1254, status = succeeded lb # [ 133.232328] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.194717] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 133.196668] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.822Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 133.274270] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 133.304660] niks3-server[1211]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 133.320315] niks3-server[1211]: 2026/09/23 05:35:05 INFO lead: acquired remote=192.168.1.4:39428 lb # [ 133.331423] niks3-server[1211]: 2026/09/23 05:35:05 INFO Signed narinfos id=37 count=1 worker2 # [ 133.283459] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.910Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.347556] niks3-server[1211]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 133.358795] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 133.367484] niks3-server[1211]: 2026/09/23 05:35:05 INFO Completed upload id=37 worker2 # [ 133.315116] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.943Z level=INFO msg="Upload complete. (303ms)" lb # [ 133.374594] niks3-server[1211]: 2026/09/23 05:35:06 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 133.329337] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 133.346995] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 133.357704] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 133.367176] nix-daemon[844]: reaped child process 1249, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 133.374839] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 134.828428] nix-daemon[844]: reaped child process 1260, status = succeeded worker2 # [ 134.923419] nix-daemon[844]: accepted connection from pid 1276, user nix-grpc-daemon (trusted) worker2 # [ 134.939436] nix-daemon[844]: reaped child process 1279, status = succeeded lb # [ 134.999186] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.960384] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 134.961942] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.588Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 135.050618] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker2 # [ 135.008112] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.636Z level=INFO msg="Uploading 1 narinfos" lb # [ 135.063517] niks3-server[1211]: 2026/09/23 05:35:07 INFO Signed narinfos id=38 count=1 lb # [ 135.073716] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 135.082043] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 135.097644] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 135.104734] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 135.111355] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo lb # [ 135.123508] niks3-server[1211]: 2026/09/23 05:35:07 INFO Completed upload id=38 worker2 # [ 135.072060] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.700Z level=INFO msg="Upload complete. (286ms)" worker2 # [ 135.093314] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 135.112248] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 135.116691] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 135.120940] nix-daemon[844]: reaped child process 1258, status = succeeded worker2 # [ 135.136417] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 135.151466] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 135.166864] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.795Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 135.177759] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 135.192078] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 135.344789] nix-daemon[844]: accepted connection from pid 1285, user nix-grpc-daemon (trusted) worker2 # [ 135.364869] nix-daemon[844]: reaped child process 1288, status = succeeded lb # [ 135.415196] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 135.375932] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.003Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 135.377893] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.003Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 135.449713] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 135.408822] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.036Z level=INFO msg="Uploading 1 narinfos" lb # [ 135.468065] niks3-server[1211]: 2026/09/23 05:35:08 INFO Signed narinfos id=39 count=1 lb # [ 135.471032] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls worker2 # [ 135.432733] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.061Z level=INFO msg="Upload complete. (225ms)" lb # [ 135.486753] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 135.493568] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 135.496814] niks3-server[1211]: 2026/09/23 05:35:08 INFO Completed upload id=39 worker2 # [ 135.441956] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=656 lb # [ 135.498956] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 135.454957] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 135.460618] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 135.465195] nix-daemon[844]: reaped child process 1283, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 135.477799] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 136.309032] nix-daemon[844]: reaped child process 1292, status = succeeded worker2 # [ 136.404152] nix-daemon[844]: accepted connection from pid 1308, user nix-grpc-daemon (trusted) worker2 # [ 136.422879] nix-daemon[844]: reaped child process 1311, status = succeeded lb # [ 136.482742] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 136.441367] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.068Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 136.443011] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.068Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 136.525379] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls worker2 # [ 136.482121] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.110Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.539816] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 136.548226] niks3-server[1211]: 2026/09/23 05:35:09 INFO Signed narinfos id=40 count=1 lb # [ 136.550455] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 136.563323] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 136.568387] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete worker2 # [ 136.516215] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.144Z level=INFO msg="Upload complete. (229ms)" lb # [ 136.571011] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo lb # [ 136.574945] niks3-server[1211]: 2026/09/23 05:35:09 INFO Completed upload id=40 worker2 # [ 136.532991] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 136.546771] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 136.550441] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 136.554206] nix-daemon[844]: reaped child process 1290, status = succeeded worker2 # [ 136.576685] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 136.590910] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 136.605824] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.234Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 136.616780] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 136.634266] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 136.777830] nix-daemon[844]: accepted connection from pid 1317, user nix-grpc-daemon (trusted) worker2 # [ 136.796202] nix-daemon[844]: reaped child process 1320, status = succeeded lb # [ 136.854318] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 136.814844] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 136.817026] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.442Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 136.885460] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker2 # [ 136.848229] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.476Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.899926] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 136.910848] niks3-server[1211]: 2026/09/23 05:35:09 INFO Signed narinfos id=41 count=1 lb # [ 136.917490] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 136.930288] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker2 # [ 136.884204] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.512Z level=INFO msg="Upload complete. (238ms)" lb # [ 136.938147] niks3-server[1211]: 2026/09/23 05:35:09 INFO Completed upload id=41 lb # [ 136.941025] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 136.897218] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 136.912594] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 worker2 # [ 136.920380] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 136.925733] nix-daemon[844]: reaped child process 1315, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 136.939466] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 138.212412] nix-daemon[844]: reaped child process 1324, status = succeeded worker2 # [ 138.311225] nix-daemon[844]: accepted connection from pid 1340, user nix-grpc-daemon (trusted) worker2 # [ 138.328996] nix-daemon[844]: reaped child process 1343, status = succeeded lb # [ 138.390393] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.348928] nix-grpc-daemon[1218]: time=2026-09-23T05:35:10.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 138.350554] nix-grpc-daemon[1218]: time=2026-09-23T05:35:10.976Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 138.431713] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 138.393587] nix-grpc-daemon[1218]: time=2026-09-23T05:35:11.021Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.449311] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 138.459090] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 138.471290] niks3-server[1211]: 2026/09/23 05:35:11 INFO Signed narinfos id=42 count=1 worker2 # [ 138.424173] nix-grpc-daemon[1218]: time=2026-09-23T05:35:11.052Z level=INFO msg="Upload complete. (273ms)" lb # [ 138.477724] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 138.484738] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 138.488569] niks3-server[1211]: 2026/09/23 05:35:11 INFO Completed upload id=42 lb # [ 138.491987] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 138.437983] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 138.456731] nix-daemon[844]: reaped child process 1322, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.86 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 6.54 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 138.912679] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 139.187926] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:11Z level=info event=signal_exit lb # [ 139.324253] niks3-server[1211]: 2026/09/23 05:35:11 INFO lead: released remote=192.168.1.4:39428 lb # [ 139.389689] envoy[907]: conn peer=[2001:db8:1::1]:57656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=305 ms=1598 lb # [ 139.395222] envoy[907]: conn peer=[2001:db8:1::1]:48604 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4426 ms=5604 worker2 # [ 140.273296] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting lb # [ 140.323137] envoy[907]: [2026-09-23 05:35:12.954][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 140.283241] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59528 worker1 # [ 140.273790] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 140.292974] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 140.299464] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 140.304153] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59528 worker2 # [ 140.309456] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 140.332529] nix-daemon[844]: reaped child process 1210, status = succeeded worker2 # [ 140.351437] nix-grpc-daemon[1157]: E0000 00:00:1790141712.979607 1203 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 140.473397] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.478729] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 140.511502] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 140.679387] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.684822] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 140.885443] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.890587] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 141.091514] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.096828] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 141.234099] nix-grpc-daemon[1157]: E0000 00:00:1790141713.862201 1203 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 141.299260] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.303530] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 141.500066] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.505396] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 141.706135] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.711456] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 141.912329] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 141.917506] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 142.118193] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.123407] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 142.325965] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.330189] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 142.526585] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.531991] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 142.546298] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 142.547958] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 142.556511] systemd[1]: nix-grpc-daemon.service: Consumed 1.348s CPU time over 15.517s wall clock time, 39.5M memory peak, 47.5K incoming IP traffic, 83.3K outgoing IP traffic. worker2 # [ 142.571332] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2 # [ 142.585225] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 142.586869] systemd[1]: nix-grpc-daemon.socket: Consumed 844us CPU time over 16.301s wall clock time, 256K memory peak, 270.7K incoming IP traffic, 31.6K outgoing IP traffic. worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 142.732683] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.737884] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 142.938506] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.943646] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.144277] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.149254] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 143.312853] niks3-server[1211]: 2026/09/23 05:35:15 INFO lead: acquired remote=192.168.1.3:35964 worker1 # [ 143.251708] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_take_over worker1 # [ 143.351466] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.355632] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.552122] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.557208] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 143.758257] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:50051 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 143.765498] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.964906] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.974138] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:50442 worker1 # [ 143.979441] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker1 # [ 145.050418] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 145.063767] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 145.085510] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 145.106929] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5) worker1 # [ 145.213998] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 2/5) worker1 # [ 145.225883] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 371 ms (attempt 3/5) worker1 # [ 145.606896] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 4/5) worker1 # [ 145.727795] nix-grpc-daemon[1325]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 145.734862] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 145.765917] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 145.773144] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 145.811382] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 145.827986] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50458 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 145.842714] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 145.857600] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 145.891994] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=info event=niks3_push_started pid=1405 worker1 # [ 146.037723] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.672Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 146.147534] nix-daemon[855]: accepted connection from pid 1418, user nix-grpc-daemon (trusted) worker1 # [ 146.166099] nix-daemon[855]: reaped child process 1421, status = succeeded lb # [ 146.237873] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 146.188069] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 146.189589] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.822Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" worker1 # [ 146.224094] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.858Z level=INFO msg="Uploading 1 narinfos" lb # [ 146.277354] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 146.288811] niks3-server[1211]: 2026/09/23 05:35:18 INFO Signed narinfos id=43 count=1 lb # [ 146.298717] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 146.312897] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 146.325578] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete worker1 # [ 146.271456] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.907Z level=INFO msg="Upload complete. (229ms)" lb # [ 146.335709] niks3-server[1211]: 2026/09/23 05:35:18 INFO Completed upload id=43 lb # [ 146.345284] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 146.281086] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 146.296267] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 146.305411] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 146.310801] nix-daemon[855]: reaped child process 1403, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker1 # [ 146.323699] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 147.620352] nix-daemon[855]: reaped child process 1425, status = succeeded worker1 # [ 147.706311] nix-daemon[855]: accepted connection from pid 1441, user nix-grpc-daemon (trusted) worker1 # [ 147.722917] nix-daemon[855]: reaped child process 1444, status = succeeded lb # [ 147.792946] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 147.743553] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.376Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 147.746773] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.376Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" worker1 # [ 147.781671] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.416Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.834204] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 147.856672] niks3-server[1211]: 2026/09/23 05:35:20 INFO Signed narinfos id=44 count=1 lb # [ 147.859593] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 147.883205] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls worker1 # [ 147.830493] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.466Z level=INFO msg="Upload complete. (292ms)" lb # [ 147.894555] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 147.900868] niks3-server[1211]: 2026/09/23 05:35:20 INFO Completed upload id=44 lb # [ 147.903523] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo lb # [ 147.907435] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker1 # [ 147.853332] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 147.869639] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 147.873749] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 147.877832] nix-daemon[855]: reaped child process 1423, status = succeeded worker1 # [ 147.898610] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 147.913435] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 147.927032] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.562Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 147.937668] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 147.953697] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 148.104894] nix-daemon[855]: accepted connection from pid 1450, user nix-grpc-daemon (trusted) worker1 # [ 148.124583] nix-daemon[855]: reaped child process 1453, status = succeeded lb # [ 148.185155] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 148.132859] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.768Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 148.134420] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.768Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 148.216073] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 148.162138] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.797Z level=INFO msg="Uploading 1 narinfos" lb # [ 148.230241] niks3-server[1211]: 2026/09/23 05:35:20 INFO Signed narinfos id=45 count=1 lb # [ 148.238326] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 148.254986] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls worker1 # [ 148.203979] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.839Z level=INFO msg="Upload complete. (234ms)" lb # [ 148.266250] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 148.270474] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo lb # [ 148.277050] niks3-server[1211]: 2026/09/23 05:35:20 INFO Completed upload id=45 worker1 # [ 148.215682] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 148.231122] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 148.237387] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 148.242563] nix-daemon[855]: reaped child process 1448, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker1 # [ 148.254103] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 148.910219] nix-daemon[855]: reaped child process 1457, status = succeeded worker1 # [ 149.026587] nix-daemon[855]: accepted connection from pid 1473, user nix-grpc-daemon (trusted) worker1 # [ 149.041661] nix-daemon[855]: reaped child process 1476, status = succeeded lb # [ 149.112720] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.062571] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.697Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 149.064450] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.697Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 149.159180] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker1 # [ 149.104654] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.740Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.171475] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 149.180934] niks3-server[1211]: 2026/09/23 05:35:21 INFO Signed narinfos id=46 count=1 lb # [ 149.183186] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 149.194062] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 149.198135] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete worker1 # [ 149.135350] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.770Z level=INFO msg="Upload complete. (269ms)" lb # [ 149.201513] niks3-server[1211]: 2026/09/23 05:35:21 INFO Completed upload id=46 lb # [ 149.203206] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 149.156461] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 149.170278] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 149.174161] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 149.178223] nix-daemon[855]: reaped child process 1455, status = succeeded worker1 # [ 149.198594] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 149.213373] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 149.228693] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.863Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 149.240574] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 149.254557] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) lb # [ 149.391100] envoy[907]: conn peer=[2001:db8:1::4]:55842 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=13262 worker1 # [ 149.404856] nix-daemon[855]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) worker1 # [ 149.422236] nix-daemon[855]: reaped child process 1485, status = succeeded lb # [ 149.491324] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.439553] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.074Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 149.441308] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.074Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" worker1 # [ 149.468765] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.104Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.522225] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 149.536967] niks3-server[1211]: 2026/09/23 05:35:22 INFO Signed narinfos id=47 count=1 lb # [ 149.539847] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 149.556482] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls worker1 # [ 149.498875] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.134Z level=INFO msg="Upload complete. (229ms)" lb # [ 149.561612] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 149.568198] niks3-server[1211]: 2026/09/23 05:35:22 INFO Completed upload id=47 lb # [ 149.571308] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 149.508263] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 149.521872] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 149.530441] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 149.533595] nix-daemon[855]: reaped child process 1480, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker1 # [ 149.546738] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 150.622237] nix-daemon[855]: reaped child process 1489, status = succeeded worker1 # [ 150.706150] nix-daemon[855]: accepted connection from pid 1505, user nix-grpc-daemon (trusted) worker1 # [ 150.723010] nix-daemon[855]: reaped child process 1508, status = succeeded lb # [ 150.793875] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 150.743199] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 150.744997] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.376Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 150.832468] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker1 # [ 150.781759] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.417Z level=INFO msg="Uploading 1 narinfos" lb # [ 150.845439] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 150.854971] niks3-server[1211]: 2026/09/23 05:35:23 INFO Signed narinfos id=48 count=1 lb # [ 150.864615] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 150.816304] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.451Z level=INFO msg="Upload complete. (259ms)" lb # [ 150.878268] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 150.883996] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 150.889036] niks3-server[1211]: 2026/09/23 05:35:23 INFO Completed upload id=48 lb # [ 150.892822] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 150.828743] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 150.849109] nix-daemon[855]: reaped child process 1487, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.06 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker2 # [ 151.150833] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 151.159011] systemd[1]: Starting scheduler-token.service... worker2 # [ 151.230674] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 151.236250] systemd[1]: Finished scheduler-token.service. worker2 # [ 151.237723] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 75ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 151.254411] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 151.365345] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 151.367550] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 151.371393] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 151.376051] nix-grpc-daemon[1364]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 151.377507] nix-grpc-daemon[1364]: E0000 00:00:1790141724.004290 1364 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 151.424110] nix-grpc-daemon[1364]: E0000 00:00:1790141724.010363 1364 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 151.426404] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 151.431268] nix-grpc-daemon[1364]: E0000 00:00:1790141724.018918 1407 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 151.434220] nix-grpc-daemon[1364]: E0000 00:00:1790141724.038953 1407 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 151.436707] nix-grpc-daemon[1364]: E0000 00:00:1790141724.039088 1407 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 151.439087] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 151.439880] nix-grpc-daemon[1364]: E0000 00:00:1790141724.058796 1407 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 151.447975] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 151.448137] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 151.452909] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:50442 worker1 # [ 151.458248] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 151.537750] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:38416 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.05 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 151.789428] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: d82ceea537064fe3a9067e1eb904666a client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker1 # [ 151.987198] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50458 client # [ 152.003848] nix[1404]: don't know how to build these paths: client # [ 152.005238] nix[1404]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 152.003372] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 152.010443] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 152.049952] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 152.065838] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50458 client # [ 152.088275] nix[1404]: copying 1 paths to 'lb:50051'... worker1 # [ 152.080580] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 client # [ 152.090559] nix[1404]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 152.097208] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 152.258247] nix-daemon[855]: accepted connection from pid 1521, user nix-grpc-daemon (trusted) worker1 # [ 152.275796] nix-daemon[855]: reaped child process 1524, status = succeeded lb # [ 152.348905] niks3-server[1211]: 2026/09/23 05:35:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 152.297338] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.932Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 152.299454] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.932Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 152.378964] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst worker1 # [ 152.330113] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.965Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.395052] niks3-server[1211]: 2026/09/23 05:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 152.403738] niks3-server[1211]: 2026/09/23 05:35:25 INFO Signed narinfos id=49 count=1 lb # [ 152.411696] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker1 # [ 152.354358] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.989Z level=INFO msg="Upload complete. (240ms)" lb # [ 152.419227] niks3-server[1211]: 2026/09/23 05:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 152.423948] niks3-server[1211]: 2026/09/23 05:35:25 INFO Completed upload id=49 lb # [ 152.427104] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 152.364292] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 152.380310] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 worker1 # [ 152.388512] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) worker1 # [ 152.391407] nix-daemon[855]: reaped child process 1519, status = succeeded worker1 # [ 152.402406] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted) client # [ 152.409483] nix[1404]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 154.596056] systemd-journald[272]: Received client request to rotate journal, rotating. worker2 # [ 154.621810] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5bab130533d944929e2e9da27f3fa86e. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 154.658721] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 worker1 # [ 155.046737] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:27Z level=info event=draining lb # [ 159.391413] envoy[907]: conn peer=[2001:db8:1::1]:35466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9576 tx=4373 ms=5812 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 160.069140] nix-daemon[855]: reaped child process 1528, status = succeeded worker1 # [ 160.193224] nix-daemon[855]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker1 # [ 160.209312] nix-daemon[855]: reaped child process 1572, status = succeeded lb # [ 160.278595] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.231771] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.863Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.233722] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.863Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 160.315012] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 160.272819] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.908Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.335400] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 160.345228] niks3-server[1211]: 2026/09/23 05:35:32 INFO Signed narinfos id=50 count=1 lb # [ 160.352191] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 160.355895] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 160.363842] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 160.374468] niks3-server[1211]: 2026/09/23 05:35:33 INFO Completed upload id=50 lb # [ 160.378173] niks3-server[1211]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 160.312271] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.947Z level=INFO msg="Upload complete. (292ms)" worker1 # [ 160.325395] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 160.345965] nix-daemon[855]: reaped child process 1526, status = succeeded client # [ 160.367346] systemd[1]: rs.service: Deactivated successfully. client # [ 160.372136] systemd[1]: rs.service: Consumed 193ms CPU time over 8.579s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic. worker1 # [ 161.049535] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:33Z level=info event=signal_exit lb # [ 161.317309] niks3-server[1211]: 2026/09/23 05:35:33 INFO lead: released remote=192.168.1.3:35964 lb # [ 161.443333] niks3-server[1211]: 2026/09/23 05:35:34 INFO lead: acquired remote=192.168.1.4:38688 worker2 # [ 161.396885] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:34Z level=info event=scheduler_take_over worker2 # [ 162.267513] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 162.259974] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting lb # [ 162.320159] envoy[907]: [2026-09-23 05:35:34.948][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 162.262345] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 162.262477] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50442 worker1 # [ 162.262590] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 162.262701] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50442 worker1 # [ 162.262810] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 162.288583] nix-grpc-daemon[1325]: E0000 00:00:1790141734.923592 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 162.319853] nix-daemon[855]: reaped child process 1395, status = succeeded worker2 # [ 162.470083] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 162.461126] nix-grpc-daemon[1325]: E0000 00:00:1790141735.096601 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 162.488795] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 162.867761] nix-grpc-daemon[1325]: E0000 00:00:1790141735.503247 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true worker1 # [ 163.166154] nix-grpc-daemon[1325]: E0000 00:00:1790141735.801513 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 163.596499] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 163.603102] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 163.617222] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 163.608901] systemd[1]: nix-grpc-daemon.service: Consumed 1.632s CPU time over 31.735s wall clock time, 39.3M memory peak, 49.6K incoming IP traffic, 94.8K outgoing IP traffic. worker2 # [ 163.622485] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:47536 worker2 # [ 163.627848] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 163.638467] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 163.775788] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 163.778272] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 163.782222] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 163.788062] nix-grpc-daemon[1579]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 163.789515] nix-grpc-daemon[1579]: E0000 00:00:1790141736.422353 1579 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 163.828057] nix-grpc-daemon[1579]: E0000 00:00:1790141736.428568 1579 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 163.830376] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 163.833820] nix-grpc-daemon[1579]: E0000 00:00:1790141736.434608 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 163.836692] nix-grpc-daemon[1579]: E0000 00:00:1790141736.449833 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 163.839135] nix-grpc-daemon[1579]: E0000 00:00:1790141736.449967 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 163.844725] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 163.847381] nix-grpc-daemon[1579]: E0000 00:00:1790141736.468522 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 163.849985] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.858161] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:47536 worker2 # [ 163.864383] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 164.144961] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:47544 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.92 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 164.704970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: faca667cf90e4b389da40e980f95d5f7 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 164.919064] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47556 worker2 # [ 164.933297] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted) client # [ 164.949854] nix[1440]: don't know how to build these paths: client # [ 164.951836] nix[1440]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 164.965694] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 164.972529] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 165.020562] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47556 worker2 # [ 165.035671] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5) worker2 # [ 165.062799] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 2/5) worker2 # [ 165.261095] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 290 ms (attempt 3/5) worker2 # [ 165.553078] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5) worker2 # [ 166.197470] nix-grpc-daemon[1364]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 166.201757] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 166.222417] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47556 client # [ 166.236824] nix[1440]: copying 1 paths to 'lb:50051'... worker2 # [ 166.237452] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47556 client # [ 166.237074] nix[1440]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 166.253057] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted) worker2 # [ 166.286570] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=info event=niks3_push_started pid=1497 worker2 # [ 166.435935] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.063Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 166.541358] nix-daemon[844]: accepted connection from pid 1510, user nix-grpc-daemon (trusted) worker2 # [ 166.560853] nix-daemon[844]: reaped child process 1513, status = succeeded lb # [ 166.613896] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 166.571042] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.198Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 166.574861] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.198Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 166.659932] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 166.678295] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls worker2 # [ 166.648164] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.276Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.700779] niks3-server[1211]: 2026/09/23 05:35:39 INFO Signed narinfos id=51 count=1 lb # [ 166.708206] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 166.721248] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 166.731258] niks3-server[1211]: 2026/09/23 05:35:39 INFO Completed upload id=51 worker2 # [ 166.679744] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.307Z level=INFO msg="Upload complete. (240ms)" lb # [ 166.740097] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 166.692670] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47556 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 166.708469] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47556 worker2 # [ 166.715382] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted) worker2 # [ 166.724323] nix-daemon[844]: reaped child process 1495, status = succeeded worker2 # [ 166.732204] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted) client # [ 166.729012] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 168.581421] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1364 (nix-grpc-daemon) on client request. worker2 # [ 168.583166] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. client # [ 168.588625] nix[1440]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker1 # [ 168.580490] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker2 # [ 168.598079] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL lb # [ 168.648901] niks3-server[1211]: 2026/09/23 05:35:41 INFO lead: released remote=192.168.1.4:38688 client # [ 168.588967] nix[1440]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 168.599597] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. lb # [ 168.654937] envoy[907]: [2026-09-23 05:35:41.284][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} worker2 # [ 168.611175] systemd[1]: nix-grpc-daemon.service: Consumed 507ms CPU time over 17.350s wall clock time, 35.1M memory peak, 15K incoming IP traffic, 23.5K outgoing IP traffic. worker2 # [ 168.625828] systemd[1]: Starting scheduler-token.service... worker2 # [ 168.649323] nix-daemon[844]: reaped child process 1515, status = succeeded worker2 # [ 168.651628] nix-daemon[844]: reaped child process 1483, status = succeeded worker2 # [ 168.657645] nix-daemon[1517]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 168.668465] nix-daemon[844]: reaped child process 1517, status = failed with exit code 1 worker2 # [ 168.691254] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 168.694228] systemd[1]: Finished scheduler-token.service. worker2 # [ 168.695002] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 168.700554] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 168.705592] systemd[1]: Starting scheduler-token.service... worker2 # [ 168.745266] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 168.748070] systemd[1]: Finished scheduler-token.service. worker2 # [ 168.748910] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 47ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 168.753071] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 168.780571] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 lb # [ 168.862086] niks3-server[1211]: 2026/09/23 05:35:41 INFO lead: acquired remote=192.168.1.3:42838 worker1 # [ 168.804346] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_take_over worker2 # [ 168.853627] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 168.855799] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 168.859831] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 168.866489] nix-grpc-daemon[1560]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 168.867937] nix-grpc-daemon[1560]: E0000 00:00:1790141741.492713 1560 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.35 seconds) client: must succeed: sleep 5.0 worker1 # [ 168.875843] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 168.892801] nix-grpc-daemon[1560]: E0000 00:00:1790141741.494700 1560 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 168.895577] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 168.922990] nix-grpc-daemon[1560]: E0000 00:00:1790141741.514120 1600 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 168.925952] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 168.926909] nix-grpc-daemon[1560]: E0000 00:00:1790141741.536056 1600 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 168.929358] nix-grpc-daemon[1560]: E0000 00:00:1790141741.536194 1600 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 168.934352] nix-grpc-daemon[1560]: E0000 00:00:1790141741.550562 1600 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 168.939114] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 168.941016] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 169.076827] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 client # [ 169.092254] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting worker1 # [ 169.082226] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 169.142216] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.148793] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 169.282836] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 169.288188] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" lb # [ 169.389616] envoy[907]: conn peer=[2001:db8:1::1]:44124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1515 ms=8350 lb # [ 169.394928] envoy[907]: conn peer=[2001:db8:1::3]:60940 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3797 tx=1219 ms=30377 lb # [ 169.399952] envoy[907]: conn peer=[2001:db8:1::4]:50528 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1948 tx=328 ms=17153 worker2 # [ 169.349823] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.356283] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 169.488917] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 169.494246] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 169.557328] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.563646] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 169.696626] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 169.700681] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 169.764847] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.769950] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 169.896914] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 169.902002] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 169.972674] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.976743] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 170.095840] nix[1440]: scheduler at lb:50051: no healthy upstream, reconnecting worker1 # [ 170.102997] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.108226] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.173161] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.178158] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 170.309025] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.314228] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.379168] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.384362] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 170.514748] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.520188] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.585375] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.590372] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 170.727126] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.735840] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:35242 worker1 # [ 170.740920] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 170.791286] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.791029] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 170.795666] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:35242 worker1 # [ 170.801938] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 172.101830] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 172.102127] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.160136] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 worker1 # [ 172.174809] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 172.198010] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5) worker1 # [ 172.232829] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5) worker1 # [ 172.328864] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 257 ms (attempt 3/5) worker1 # [ 172.587958] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 769 ms (attempt 4/5) worker1 # [ 173.074904] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:45Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:50704 worker1 # [ 173.361149] nix-grpc-daemon[1579]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 173.369622] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 173.378911] nix-grpc-daemon[1579]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'... worker1 # [ 173.408777] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 173.428174] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) client # [ 173.436336] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 264.948495] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:17Z level=info event=niks3_push_started pid=1688 worker1 # [ 265.032375] nix-daemon[855]: reaped child process 1663, status = succeeded worker1 # [ 265.107938] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.743Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 265.214236] nix-daemon[855]: accepted connection from pid 1700, user nix-grpc-daemon (trusted) worker1 # [ 265.230653] nix-daemon[855]: reaped child process 1703, status = succeeded lb # [ 265.301593] niks3-server[1211]: 2026/09/23 05:37:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 265.258349] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 265.259832] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.892Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 265.378167] niks3-server[1211]: 2026/09/23 05:37:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign worker1 # [ 265.323641] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.957Z level=INFO msg="Uploading 1 narinfos" lb # [ 265.387715] niks3-server[1211]: 2026/09/23 05:37:18 INFO Signed narinfos id=52 count=1 lb # [ 265.399774] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 265.407857] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst worker1 # [ 265.351663] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.987Z level=INFO msg="Upload complete. (239ms)" lb # [ 265.415443] niks3-server[1211]: 2026/09/23 05:37:18 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 265.419097] niks3-server[1211]: 2026/09/23 05:37:18 INFO Completed upload id=52 lb # [ 265.420858] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 265.425275] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo worker1 # [ 265.372380] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=93 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 265.388604] nix-daemon[855]: reaped child process 1661, status = succeeded client # [ 265.436266] systemd[1]: mid.service: Deactivated successfully. client # [ 265.440700] systemd[1]: mid.service: Consumed 273ms CPU time over 1min 40.734s wall clock time, 41.3M memory peak, 4K written to disk, 10.7K incoming IP traffic, 12.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 92.26 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 101.90 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 266.583038] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 7cd8f1b1e6b44cf8b51567e069602f7f client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 266.793652] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 266.807426] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # [ 266.821915] nix[1905]: don't know how to build these paths: client # [ 266.823228] nix[1905]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 266.825088] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:19Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 266.831316] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:19Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 266.882023] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 266.902332] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5) worker2 # [ 266.925560] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5) worker2 # [ 266.943603] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5) worker2 # [ 267.240592] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 474 ms (attempt 4/5) worker2 # [ 267.719620] nix-grpc-daemon[1560]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 267.725033] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 267.744459] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 267.757398] nix[1905]: copying 1 paths to 'lb:50051'... worker2 # [ 267.759476] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 267.760556] nix[1905]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 267.780963] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 267.813551] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=info event=niks3_push_started pid=1638 worker2 # [ 267.999493] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.627Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 268.108451] nix-daemon[844]: accepted connection from pid 1654, user nix-grpc-daemon (trusted) worker2 # [ 268.128837] nix-daemon[844]: reaped child process 1657, status = succeeded lb # [ 268.180796] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 268.140097] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 268.141559] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.767Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 268.218490] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker2 # [ 268.180759] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.808Z level=INFO msg="Uploading 1 narinfos" lb # [ 268.233527] niks3-server[1211]: 2026/09/23 05:37:20 INFO Signed narinfos id=53 count=1 lb # [ 268.240343] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker2 # [ 268.205556] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.833Z level=INFO msg="Upload complete. (202ms)" lb # [ 268.253162] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 268.260807] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 268.265507] niks3-server[1211]: 2026/09/23 05:37:20 INFO Completed upload id=53 worker2 # [ 268.212126] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 268.268533] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 268.224423] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 268.227813] nix-daemon[844]: reaped child process 1636, status = succeeded worker2 # [ 268.234489] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 268.244753] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # [ 268.243127] nix[1905]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... lb # [ 269.392833] envoy[907]: conn peer=[2001:db8:1::1]:43630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4454 tx=2266 ms=100496 client: must succeed: systemctl kill -s INT intr client # [ 270.375178] systemd[1]: intr.service: Sent signal SIGINT to main process 1905 (nix) on client request. worker2 # [ 270.387475] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:23Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 270.384583] nix[1905]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 270.396757] nix[1905]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 270.423726] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 270.428953] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 270.431094] systemd[1]: intr.service: Consumed 198ms CPU time over 3.844s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 270.471261] nix-daemon[1661]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 270.476683] nix-daemon[844]: reaped child process 1659, status = succeeded worker2 # [ 270.482098] nix-daemon[844]: reaped child process 1661, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 3.96 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 270.570300] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 1f9e5e12dd924c92a6ac092a9707eeaa client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) worker1 # [ 270.763824] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50688 client # [ 270.780565] nix[1926]: don't know how to build these paths: client # [ 270.781515] nix[1926]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 270.783841] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 270.791311] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 270.830703] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 worker1 # [ 270.853231] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50688 client # [ 270.876142] nix[1926]: copying 1 paths to 'lb:50051'... worker1 # [ 270.870100] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50688 client # [ 270.880930] nix[1926]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 270.885195] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 271.043919] nix-daemon[855]: accepted connection from pid 1737, user nix-grpc-daemon (trusted) worker1 # [ 271.062560] nix-daemon[855]: reaped child process 1740, status = succeeded lb # [ 271.129760] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 271.078498] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.713Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 271.080103] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.713Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 271.162870] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 271.111556] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.746Z level=INFO msg="Uploading 1 narinfos" lb # [ 271.178784] niks3-server[1211]: 2026/09/23 05:37:23 INFO Signed narinfos id=54 count=1 lb # [ 271.188509] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 271.199322] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 271.209312] niks3-server[1211]: 2026/09/23 05:37:23 INFO Completed upload id=54 worker1 # [ 271.142841] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.778Z level=INFO msg="Upload complete. (239ms)" worker1 # [ 271.151888] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 271.214867] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo lb # [ 271.224891] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker1 # [ 271.164444] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 worker1 # [ 271.174492] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 271.179939] nix-daemon[855]: reaped child process 1735, status = succeeded client # [ 271.199204] nix[1926]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 271.194882] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 273.172280] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 273.211820] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 273.496930] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 273.504692] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 273.520782] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 273.524053] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 273.591209] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 273.605900] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 client # copying 1 paths to 'lb:50051'... worker2 # [ 273.621831] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 273.645154] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 273.810974] nix-daemon[844]: accepted connection from pid 1708, user nix-grpc-daemon (trusted) worker2 # [ 273.832185] nix-daemon[844]: reaped child process 1711, status = succeeded lb # [ 273.885767] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 273.846803] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 273.848708] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.474Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" worker1 # [ 273.840503] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=info event=draining lb # [ 273.927672] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker2 # [ 273.884583] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.512Z level=INFO msg="Uploading 1 narinfos" lb # [ 273.940971] niks3-server[1211]: 2026/09/23 05:37:26 INFO Signed narinfos id=55 count=1 lb # [ 273.949886] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 273.958466] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 273.968596] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 273.977076] niks3-server[1211]: 2026/09/23 05:37:26 INFO Completed upload id=55 worker2 # [ 273.924790] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.553Z level=INFO msg="Upload complete. (262ms)" lb # [ 273.984170] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 273.936357] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 273.952334] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 273.962081] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 273.968723] nix-daemon[844]: reaped child process 1706, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker2 # [ 273.984196] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 274.618011] nix-daemon[844]: reaped child process 1715, status = succeeded worker2 # [ 274.733628] nix-daemon[844]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker2 # [ 274.750180] nix-daemon[844]: reaped child process 1734, status = succeeded lb # [ 274.809061] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 274.768074] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 274.769604] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.395Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 274.852362] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker2 # [ 274.807124] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.435Z level=INFO msg="Uploading 1 narinfos" lb # [ 274.869418] niks3-server[1211]: 2026/09/23 05:37:27 INFO Signed narinfos id=56 count=1 lb # [ 274.876330] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker2 # [ 274.837126] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.465Z level=INFO msg="Upload complete. (242ms)" lb # [ 274.889453] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 274.898598] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker2 # [ 274.847391] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 274.842503] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:27Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 lb # [ 274.904571] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete worker1 # [ 274.844381] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:27Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 lb # [ 274.907691] niks3-server[1211]: 2026/09/23 05:37:27 INFO Completed upload id=56 lb # [ 274.910778] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 274.864522] nix-daemon[844]: reaped child process 1713, status = succeeded worker2 # [ 274.900267] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 274.914392] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 client # copying 1 paths to 'lb:50051'... worker2 # [ 274.930192] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.556Z level=INFO msg="All 1 paths already cached" worker2 # [ 274.935540] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 274.957000] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 275.107623] nix-daemon[844]: accepted connection from pid 1738, user nix-grpc-daemon (trusted) worker2 # [ 275.123808] nix-daemon[844]: reaped child process 1741, status = succeeded lb # [ 275.181992] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 275.139366] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.767Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 275.141008] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.767Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" worker2 # [ 275.177195] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.805Z level=INFO msg="Uploading 1 narinfos" lb # [ 275.221736] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 275.233942] niks3-server[1211]: 2026/09/23 05:37:27 INFO Signed narinfos id=57 count=1 lb # [ 275.240455] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 275.247406] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 275.255291] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete worker2 # [ 275.203149] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.831Z level=INFO msg="Upload complete. (229ms)" lb # [ 275.263582] niks3-server[1211]: 2026/09/23 05:37:27 INFO Completed upload id=57 lb # [ 275.267619] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 275.212616] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 275.226607] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 275.234929] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 275.238161] nix-daemon[844]: reaped child process 1736, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker2 # [ 275.248136] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 276.113357] nix-daemon[844]: reaped child process 1745, status = succeeded worker2 # [ 276.224933] nix-daemon[844]: accepted connection from pid 1761, user nix-grpc-daemon (trusted) worker2 # [ 276.242336] nix-daemon[844]: reaped child process 1764, status = succeeded lb # [ 276.300823] niks3-server[1211]: 2026/09/23 05:37:28 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 276.259424] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.886Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 276.261275] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.886Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 276.336234] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 276.298035] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.926Z level=INFO msg="Uploading 1 narinfos" lb # [ 276.349850] niks3-server[1211]: 2026/09/23 05:37:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 276.362738] niks3-server[1211]: 2026/09/23 05:37:28 INFO Signed narinfos id=58 count=1 lb # [ 276.369738] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 276.378364] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 276.389975] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete worker2 # [ 276.338421] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.966Z level=INFO msg="Upload complete. (278ms)" lb # [ 276.393582] niks3-server[1211]: 2026/09/23 05:37:29 INFO Completed upload id=58 lb # [ 276.396173] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 276.344080] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:28Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 276.349135] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 276.345907] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:28Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 276.371281] nix-daemon[844]: reaped child process 1743, status = succeeded worker2 # [ 276.400101] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 276.414686] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 276.429693] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.057Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 276.440776] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 276.457836] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 276.615219] nix-daemon[844]: accepted connection from pid 1768, user nix-grpc-daemon (trusted) worker2 # [ 276.634016] nix-daemon[844]: reaped child process 1771, status = succeeded lb # [ 276.690872] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 276.647351] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.275Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 276.649422] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.277Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 276.723474] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls worker2 # [ 276.682661] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.310Z level=INFO msg="Uploading 1 narinfos" lb # [ 276.738821] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 276.749209] niks3-server[1211]: 2026/09/23 05:37:29 INFO Signed narinfos id=59 count=1 lb # [ 276.755600] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 276.766389] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 276.772163] niks3-server[1211]: 2026/09/23 05:37:29 INFO Completed upload id=59 worker2 # [ 276.720133] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.348Z level=INFO msg="Upload complete. (245ms)" lb # [ 276.776117] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 276.732428] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 276.747204] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 276.756579] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 276.759541] nix-daemon[844]: reaped child process 1766, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 276.771074] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 277.878928] nix-daemon[844]: reaped child process 1775, status = succeeded worker2 # [ 277.952088] nix-daemon[844]: accepted connection from pid 1791, user nix-grpc-daemon (trusted) worker2 # [ 277.969755] nix-daemon[844]: reaped child process 1794, status = succeeded lb # [ 278.026904] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 277.986571] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.613Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 277.988351] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.613Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 278.068757] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker2 # [ 278.030890] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.658Z level=INFO msg="Uploading 1 narinfos" lb # [ 278.085316] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 278.094219] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 278.102948] niks3-server[1211]: 2026/09/23 05:37:30 INFO Signed narinfos id=60 count=1 worker2 # [ 278.053648] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.681Z level=INFO msg="Upload complete. (269ms)" lb # [ 278.107930] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 278.110660] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 278.114889] niks3-server[1211]: 2026/09/23 05:37:30 INFO Completed upload id=60 lb # [ 278.118054] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 278.068250] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker2 # [ 278.081440] nix-daemon[844]: reaped child process 1773, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 4.84 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 278.460965] nix-daemon[855]: reaped child process 1744, status = succeeded worker1 # [ 278.549654] nix-daemon[855]: accepted connection from pid 1791, user nix-grpc-daemon (trusted) worker1 # [ 278.566327] nix-daemon[855]: reaped child process 1794, status = succeeded lb # [ 278.635714] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 278.583643] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 278.585308] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.218Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 278.673271] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker1 # [ 278.617088] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.252Z level=INFO msg="Uploading 1 narinfos" lb # [ 278.689040] niks3-server[1211]: 2026/09/23 05:37:31 INFO Signed narinfos id=61 count=1 lb # [ 278.691836] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst worker1 # [ 278.644560] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.280Z level=INFO msg="Upload complete. (234ms)" lb # [ 278.706714] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 278.715278] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 278.720632] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete worker1 # [ 278.656850] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=7 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 lb # [ 278.724704] niks3-server[1211]: 2026/09/23 05:37:31 INFO Completed upload id=61 lb # [ 278.726601] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 278.672557] nix-daemon[855]: reaped child process 1742, status = succeeded client # [ 278.702486] systemd[1]: sw.service: Deactivated successfully. client # [ 278.707540] systemd[1]: sw.service: Consumed 188ms CPU time over 8.132s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 278.842743] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:31Z level=info event=signal_exit lb # [ 279.387093] envoy[907]: conn peer=[2001:db8:1::1]:59936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=829 ms=3627 client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service lb # [ 279.392048] envoy[907]: conn peer=[2001:db8:1::1]:47682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=4407 ms=4588 lb # [ 279.392174] envoy[907]: conn peer=[2001:db8:1::1]:59950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3004 tx=1518 ms=7910 lb # [ 279.864230] niks3-server[1211]: 2026/09/23 05:37:32 INFO lead: released remote=192.168.1.3:42838 worker1 # [ 280.805529] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting worker2 # [ 280.817714] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 280.866826] envoy[907]: [2026-09-23 05:37:33.496][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 280.816698] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:35242 worker1 # [ 280.827720] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 280.833090] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 280.837650] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:35242 worker1 # [ 280.843849] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 280.868277] nix-daemon[855]: reaped child process 1654, status = succeeded worker1 # [ 280.891698] nix-grpc-daemon[1579]: E0000 00:00:1790141853.527058 1625 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 280.905025] nix-grpc-daemon[1579]: E0000 00:00:1790141853.540450 1625 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 281.018680] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 281.023966] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 281.045627] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 281.224877] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 281.229889] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 281.430844] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 281.435980] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 281.636947] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 281.641951] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 281.844769] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 281.848860] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 281.923276] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 281.928326] systemd[1]: nix-grpc-daemon.service: Consumed 1.054s CPU time over 1min 58.298s wall clock time, 38.2M memory peak, 30.4K incoming IP traffic, 38.7K outgoing IP traffic. worker1 # [ 281.951513] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 282.045248] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 282.050368] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 282.099911] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 282.102324] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 282.106361] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 282.112057] nix-grpc-daemon[1808]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 282.113475] nix-grpc-daemon[1808]: E0000 00:00:1790141854.746710 1808 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 282.123673] nix-grpc-daemon[1808]: E0000 00:00:1790141854.753249 1808 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 282.186406] niks3-server[1211]: 2026/09/23 05:37:34 INFO lead: acquired remote=192.168.1.3:56924 worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.66 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 282.140188] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 282.141893] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 282.142688] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_take_over worker1 # [ 282.143772] nix-grpc-daemon[1808]: E0000 00:00:1790141854.766115 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 282.168285] nix-grpc-daemon[1808]: E0000 00:00:1790141854.788476 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 282.171053] nix-grpc-daemon[1808]: E0000 00:00:1790141854.788635 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 282.173667] nix-grpc-daemon[1808]: E0000 00:00:1790141854.802934 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 282.184269] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 282.186144] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 282.251443] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 282.256618] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 282.385836] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 282.390972] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 282.457642] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 282.462720] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 282.591693] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 282.597058] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 282.663600] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 282.668717] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 282.797586] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 282.802633] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 282.871180] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 282.875349] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 283.003251] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 283.011689] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40770 worker1 # [ 283.017562] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 283.071579] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 283.072136] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 283.077418] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40770 worker1 # [ 283.082750] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 286.008751] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:40786 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 4.34 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.09 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 286.841397] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 7c9c3de658c74a1daba61813ecf27f7f client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 287.044814] nix[1982]: don't know how to build these paths: client # [ 287.045763] nix[1982]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client # [ 287.046810] nix[1982]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv client # [ 287.047789] nix[1982]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker2 # [ 287.046368] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 287.049359] nix[1982]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker1 # [ 287.054554] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=0 worker1 # [ 287.060047] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 287.065922] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=0 worker1 # [ 287.071220] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 287.076580] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=0 worker1 # [ 287.080750] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=0 worker2 # [ 287.108773] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 287.103067] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker2 # [ 287.125068] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 287.117753] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) client # [ 287.136261] nix[1982]: copying 1 paths to 'lb:50051'... worker2 # [ 287.139954] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 287.141188] nix[1982]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'... worker1 # [ 287.140687] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5) worker2 # [ 287.155050] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker1 # [ 287.147595] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5) worker1 # [ 287.202186] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 370 ms (attempt 3/5) worker2 # [ 287.323101] nix-daemon[844]: accepted connection from pid 1798, user nix-grpc-daemon (trusted) worker2 # [ 287.345382] nix-daemon[844]: reaped child process 1801, status = succeeded lb # [ 287.399360] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 287.358575] nix-grpc-daemon[1638]: time=2026-09-23T05:37:39.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 287.360463] nix-grpc-daemon[1638]: time=2026-09-23T05:37:39.986Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)" lb # [ 287.433020] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst worker2 # [ 287.394205] nix-grpc-daemon[1638]: time=2026-09-23T05:37:40.022Z level=INFO msg="Uploading 1 narinfos" lb # [ 287.448338] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 287.456207] niks3-server[1211]: 2026/09/23 05:37:40 INFO Signed narinfos id=62 count=1 lb # [ 287.465845] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls lb # [ 287.480735] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 287.486874] niks3-server[1211]: 2026/09/23 05:37:40 INFO Completed upload id=62 worker2 # [ 287.434794] nix-grpc-daemon[1638]: time=2026-09-23T05:37:40.063Z level=INFO msg="Upload complete. (261ms)" lb # [ 287.498436] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo worker2 # [ 287.445367] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 287.460509] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 287.474871] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 287.480396] nix-daemon[844]: reaped child process 1796, status = succeeded client # [ 287.489557] nix[1982]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051... worker2 # [ 287.495174] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker1 # [ 287.574232] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 4/5) worker1 # [ 287.899325] nix-grpc-daemon[1808]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 287.904482] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 287.923677] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 client # [ 287.947545] nix[1982]: copying 1 paths to 'lb:50051'... worker1 # [ 287.939377] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 client # [ 287.950552] nix[1982]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'... worker1 # [ 287.956339] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 287.998900] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=info event=niks3_push_started pid=1880 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 288.134055] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.769Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 288.257713] nix-daemon[855]: accepted connection from pid 1892, user nix-grpc-daemon (trusted) worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.12 seconds) worker1 # [ 288.291233] nix-daemon[855]: reaped child process 1898, status = succeeded lb # [ 288.351623] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 288.300482] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.934Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 288.301948] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.934Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)" worker1 # [ 288.337677] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.972Z level=INFO msg="Uploading 1 narinfos" lb # [ 288.392720] niks3-server[1211]: 2026/09/23 05:37:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 288.406953] niks3-server[1211]: 2026/09/23 05:37:41 INFO Signed narinfos id=63 count=1 worker1 # [ 288.361686] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.997Z level=INFO msg="Upload complete. (224ms)" lb # [ 288.413788] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst lb # [ 288.427827] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls worker1 # [ 288.366868] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 288.371363] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 lb # [ 288.432753] niks3-server[1211]: 2026/09/23 05:37:41 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 288.436128] niks3-server[1211]: 2026/09/23 05:37:41 INFO Completed upload id=63 worker1 # [ 288.378365] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) lb # [ 288.438666] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo worker1 # [ 288.381324] nix-daemon[855]: reaped child process 1878, status = succeeded worker1 # [ 288.388125] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) client # [ 288.397008] nix[1982]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051... lb # [ 289.392271] envoy[907]: conn peer=[2001:db8:1::3]:52016 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2431 tx=1013 ms=117188 worker2 # [ 289.797749] nix-daemon[844]: reaped child process 1805, status = succeeded worker2 # [ 289.871680] nix-daemon[844]: accepted connection from pid 1830, user nix-grpc-daemon (trusted) worker2 # [ 289.890236] nix-daemon[844]: reaped child process 1833, status = succeeded lb # [ 289.948925] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 289.908429] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.534Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 289.910706] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.534Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)" lb # [ 289.990606] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker2 # [ 289.950988] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.579Z level=INFO msg="Uploading 1 narinfos" lb # [ 290.007006] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls lb # [ 290.016547] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 290.029475] niks3-server[1211]: 2026/09/23 05:37:42 INFO Signed narinfos id=64 count=1 worker2 # [ 289.976480] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.604Z level=INFO msg="Upload complete. (269ms)" worker1 # [ 289.974827] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:42Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=1 lb # [ 290.035774] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst lb # [ 290.040491] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 290.044845] niks3-server[1211]: 2026/09/23 05:37:42 INFO Completed upload id=64 lb # [ 290.047367] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo worker2 # [ 289.989646] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=1 worker2 # [ 290.005367] nix-daemon[844]: reaped child process 1803, status = succeeded worker2 # [ 290.033120] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 290.048572] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 290.061611] nix[1982]: copying 1 paths to 'lb:50051'... worker2 # [ 290.062847] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 client # [ 290.067003] nix[1982]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'... worker2 # [ 290.082759] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 290.244892] nix-daemon[844]: accepted connection from pid 1837, user nix-grpc-daemon (trusted) worker2 # [ 290.267350] nix-daemon[844]: reaped child process 1840, status = succeeded lb # [ 290.317457] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 290.274111] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.902Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 290.275546] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.902Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)" lb # [ 290.359379] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign worker2 # [ 290.315672] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.943Z level=INFO msg="Uploading 1 narinfos" lb # [ 290.374916] niks3-server[1211]: 2026/09/23 05:37:42 INFO Signed narinfos id=65 count=1 lb # [ 290.382372] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst lb # [ 290.393011] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls lb # [ 290.398565] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete worker2 # [ 290.353090] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.981Z level=INFO msg="Upload complete. (254ms)" lb # [ 290.405149] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo lb # [ 290.409173] niks3-server[1211]: 2026/09/23 05:37:43 INFO Completed upload id=65 worker2 # [ 290.363421] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 290.380352] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 290.387495] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 290.393373] nix-daemon[844]: reaped child process 1835, status = succeeded client # [ 290.402637] nix[1982]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051... worker2 # [ 290.408461] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker1 # [ 290.844873] nix-daemon[855]: reaped child process 1906, status = succeeded worker1 # [ 290.956445] nix-daemon[855]: accepted connection from pid 1931, user nix-grpc-daemon (trusted) worker1 # [ 290.976201] nix-daemon[855]: reaped child process 1934, status = succeeded lb # [ 291.037841] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 290.987174] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.620Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 290.989830] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.620Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)" lb # [ 291.070000] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv lb # [ 291.092422] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst worker1 # [ 291.040737] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.675Z level=INFO msg="Uploading 1 narinfos" lb # [ 291.102218] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 291.108195] niks3-server[1211]: 2026/09/23 05:37:43 INFO Signed narinfos id=66 count=1 lb # [ 291.110366] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls lb # [ 291.121869] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 291.126501] niks3-server[1211]: 2026/09/23 05:37:43 INFO Completed upload id=66 worker1 # [ 291.064526] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.700Z level=INFO msg="Upload complete. (266ms)" lb # [ 291.129576] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo worker1 # [ 291.077003] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=1 worker1 # [ 291.092343] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 291.097667] nix-daemon[855]: reaped child process 1904, status = succeeded worker1 # [ 291.118354] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 291.133714] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 client # [ 291.157181] nix[1982]: copying 1 paths to 'lb:50051'... worker1 # [ 291.148106] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 client # [ 291.157419] nix[1982]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'... worker1 # [ 291.163955] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 291.311259] nix-daemon[855]: accepted connection from pid 1938, user nix-grpc-daemon (trusted) worker1 # [ 291.330519] nix-daemon[855]: reaped child process 1941, status = succeeded lb # [ 291.396078] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 291.343028] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 291.344750] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.978Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)" worker1 # [ 291.363275] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.998Z level=INFO msg="Uploading 1 narinfos" lb # [ 291.425666] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 291.438217] niks3-server[1211]: 2026/09/23 05:37:44 INFO Signed narinfos id=67 count=1 lb # [ 291.446115] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst lb # [ 291.454569] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls worker1 # [ 291.406140] nix-grpc-daemon[1880]: time=2026-09-23T05:37:44.041Z level=INFO msg="Upload complete. (227ms)" lb # [ 291.468157] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 291.472756] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo lb # [ 291.478467] niks3-server[1211]: 2026/09/23 05:37:44 INFO Completed upload id=67 worker1 # [ 291.416383] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 291.432067] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 291.441865] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 291.445113] nix-daemon[855]: reaped child process 1936, status = succeeded client # [ 291.463976] nix[1982]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051... worker1 # [ 291.457889] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker2 # [ 292.046408] nix-daemon[844]: reaped child process 1844, status = succeeded worker2 # [ 292.149117] nix-daemon[844]: accepted connection from pid 1869, user nix-grpc-daemon (trusted) worker2 # [ 292.165843] nix-daemon[844]: reaped child process 1872, status = succeeded lb # [ 292.226304] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 292.184358] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 292.185819] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.811Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)" lb # [ 292.270738] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 292.288391] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 292.298866] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls worker2 # [ 292.255494] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.883Z level=INFO msg="Uploading 1 narinfos" lb # [ 292.307110] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv lb # [ 292.315093] niks3-server[1211]: 2026/09/23 05:37:44 INFO Signed narinfos id=68 count=1 lb # [ 292.325108] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 292.336338] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo lb # [ 292.350098] niks3-server[1211]: 2026/09/23 05:37:44 INFO Completed upload id=68 worker2 # [ 292.298562] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.926Z level=INFO msg="Upload complete. (278ms)" worker2 # [ 292.309121] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=1 worker2 # [ 292.329130] nix-daemon[844]: reaped child process 1842, status = succeeded worker1 # [ 293.083178] nix-daemon[855]: reaped child process 1945, status = succeeded worker1 # [ 293.181141] nix-daemon[855]: accepted connection from pid 1970, user nix-grpc-daemon (trusted) worker1 # [ 293.199190] nix-daemon[855]: reaped child process 1973, status = succeeded lb # [ 293.268709] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 293.219003] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 293.220627] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.853Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)" lb # [ 293.310515] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker1 # [ 293.259255] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.894Z level=INFO msg="Uploading 1 narinfos" lb # [ 293.327436] niks3-server[1211]: 2026/09/23 05:37:45 INFO Signed narinfos id=69 count=1 lb # [ 293.338416] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls lb # [ 293.349112] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv lb # [ 293.362897] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst worker1 # [ 293.307372] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.942Z level=INFO msg="Upload complete. (236ms)" lb # [ 293.369384] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 293.374195] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo lb # [ 293.380332] niks3-server[1211]: 2026/09/23 05:37:46 INFO Completed upload id=69 worker1 # [ 293.319380] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=1 worker1 # [ 293.339233] nix-daemon[855]: reaped child process 1943, status = succeeded client # [ 293.377989] systemd[1]: burst.service: Deactivated successfully. client # [ 293.383109] systemd[1]: burst.service: Consumed 227ms CPU time over 6.537s wall clock time, 53.6M memory peak, 14.4K incoming IP traffic, 19.2K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 8.07 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 294.891543] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 294.899492] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 294.934333] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 294.934631] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 294.934833] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 294.934956] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 294.978710] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 294.993680] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 295.008517] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 295.023525] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 295.182549] nix-daemon[855]: accepted connection from pid 1987, user nix-grpc-daemon (trusted) worker1 # [ 295.201685] nix-daemon[855]: reaped child process 1990, status = succeeded lb # [ 295.269844] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 295.218286] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 295.219977] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.853Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 295.299785] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 295.249694] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.885Z level=INFO msg="Uploading 1 narinfos" lb # [ 295.313339] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 295.323528] niks3-server[1211]: 2026/09/23 05:37:47 INFO Signed narinfos id=70 count=1 lb # [ 295.330211] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 295.277118] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.912Z level=INFO msg="Upload complete. (236ms)" lb # [ 295.340524] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 295.347869] niks3-server[1211]: 2026/09/23 05:37:47 INFO Completed upload id=70 lb # [ 295.351939] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 295.288334] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 295.300459] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 295.310928] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 295.314203] nix-daemon[855]: reaped child process 1985, status = succeeded worker1 # [ 295.327716] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 296.421023] nix-daemon[855]: reaped child process 1994, status = succeeded worker1 # [ 296.505285] nix-daemon[855]: accepted connection from pid 2010, user nix-grpc-daemon (trusted) worker1 # [ 296.524375] nix-daemon[855]: reaped child process 2013, status = succeeded lb # [ 296.590078] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 296.539647] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.173Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 296.541564] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.173Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" worker1 # [ 296.576656] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.212Z level=INFO msg="Uploading 1 narinfos" lb # [ 296.631373] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 296.647408] niks3-server[1211]: 2026/09/23 05:37:49 INFO Signed narinfos id=71 count=1 lb # [ 296.653803] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 296.662009] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 296.671277] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 296.616084] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.251Z level=INFO msg="Upload complete. (277ms)" lb # [ 296.679037] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 296.683467] niks3-server[1211]: 2026/09/23 05:37:49 INFO Completed upload id=71 lb # [ 296.687744] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 296.629124] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=1 worker1 # [ 296.653097] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 296.661873] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 296.668313] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 296.674733] nix-daemon[855]: reaped child process 1992, status = succeeded worker1 # [ 296.678687] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 296.686343] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 296.696295] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 296.828832] nix-daemon[855]: accepted connection from pid 2019, user nix-grpc-daemon (trusted) worker1 # [ 296.847113] nix-daemon[855]: reaped child process 2022, status = succeeded lb # [ 296.912394] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 296.858873] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 296.860407] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.494Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 296.940579] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 296.884116] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.519Z level=INFO msg="Uploading 1 narinfos" lb # [ 296.953737] niks3-server[1211]: 2026/09/23 05:37:49 INFO Signed narinfos id=72 count=1 lb # [ 296.961281] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 296.973481] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 296.916411] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.552Z level=INFO msg="Upload complete. (209ms)" lb # [ 296.979249] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 296.984408] niks3-server[1211]: 2026/09/23 05:37:49 INFO Completed upload id=72 worker1 # [ 296.924257] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 296.987522] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 296.937170] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 296.943039] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 296.947815] nix-daemon[855]: reaped child process 2017, status = succeeded worker1 # [ 296.958516] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 297.799258] nix-daemon[855]: reaped child process 2026, status = succeeded worker1 # [ 297.923088] nix-daemon[855]: accepted connection from pid 2042, user nix-grpc-daemon (trusted) worker1 # [ 297.938815] nix-daemon[855]: reaped child process 2045, status = succeeded lb # [ 298.010271] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 297.958984] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.592Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 297.960678] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.592Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 298.041715] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls worker1 # [ 297.998606] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.634Z level=INFO msg="Uploading 1 narinfos" lb # [ 298.060498] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 298.072496] niks3-server[1211]: 2026/09/23 05:37:50 INFO Signed narinfos id=73 count=1 lb # [ 298.079826] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 298.095062] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 298.038911] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.674Z level=INFO msg="Upload complete. (268ms)" lb # [ 298.104203] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 298.111726] niks3-server[1211]: 2026/09/23 05:37:50 INFO Completed upload id=73 lb # [ 298.117464] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 298.057177] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=1 worker1 # [ 298.079802] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 298.085791] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 298.090475] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 298.096209] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 298.108348] nix-daemon[855]: reaped child process 2024, status = succeeded worker1 # [ 298.110548] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 298.119509] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 298.261802] nix-daemon[855]: accepted connection from pid 2051, user nix-grpc-daemon (trusted) worker1 # [ 298.279298] nix-daemon[855]: reaped child process 2054, status = succeeded lb # [ 298.345074] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 298.291649] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 298.293286] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.927Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 298.375175] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 298.323904] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.959Z level=INFO msg="Uploading 1 narinfos" lb # [ 298.389800] niks3-server[1211]: 2026/09/23 05:37:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 298.401685] niks3-server[1211]: 2026/09/23 05:37:51 INFO Signed narinfos id=74 count=1 lb # [ 298.407548] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst worker1 # [ 298.356371] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.992Z level=INFO msg="Upload complete. (232ms)" lb # [ 298.421067] niks3-server[1211]: 2026/09/23 05:37:51 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete worker1 # [ 298.364252] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 298.428583] niks3-server[1211]: 2026/09/23 05:37:51 INFO Completed upload id=74 lb # [ 298.432915] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 298.379495] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 298.388103] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 298.391025] nix-daemon[855]: reaped child process 2049, status = succeeded worker1 # [ 298.401634] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) lb # [ 299.390869] envoy[907]: conn peer=[2001:db8:1::1]:34334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=5203 ms=6314 worker1 # [ 299.664308] nix-daemon[855]: reaped child process 2058, status = succeeded worker1 # [ 299.749235] nix-daemon[855]: accepted connection from pid 2074, user nix-grpc-daemon (trusted) worker1 # [ 299.767842] nix-daemon[855]: reaped child process 2077, status = succeeded lb # [ 299.829639] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 299.776689] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 299.778162] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.412Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" worker1 # [ 299.820626] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.456Z level=INFO msg="Uploading 1 narinfos" lb # [ 299.873210] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 299.891768] niks3-server[1211]: 2026/09/23 05:37:52 INFO Signed narinfos id=75 count=1 lb # [ 299.899588] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 299.913399] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls worker1 # [ 299.859036] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.494Z level=INFO msg="Upload complete. (245ms)" lb # [ 299.923831] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 299.929395] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 299.935450] niks3-server[1211]: 2026/09/23 05:37:52 INFO Completed upload id=75 lb # [ 299.939307] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 299.876117] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=1 worker1 # [ 299.895620] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 299.901255] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 299.906838] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 299.912519] nix-daemon[855]: reaped child process 2056, status = succeeded worker1 # [ 299.916113] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 299.921280] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.555Z level=INFO msg="All 1 paths already cached" worker1 # [ 299.925200] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 299.938228] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 300.065121] nix-daemon[855]: accepted connection from pid 2083, user nix-grpc-daemon (trusted) worker1 # [ 300.082111] nix-daemon[855]: reaped child process 2086, status = succeeded lb # [ 300.150948] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 300.097882] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.733Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 300.099376] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.733Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 300.181217] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 300.124782] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.760Z level=INFO msg="Uploading 1 narinfos" lb # [ 300.192943] niks3-server[1211]: 2026/09/23 05:37:52 INFO Signed narinfos id=76 count=1 lb # [ 300.201920] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls worker1 # [ 300.151251] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.786Z level=INFO msg="Upload complete. (205ms)" lb # [ 300.213697] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 300.157539] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 300.221959] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 300.225904] niks3-server[1211]: 2026/09/23 05:37:52 INFO Completed upload id=76 lb # [ 300.228349] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 300.170988] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 300.178868] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 300.181594] nix-daemon[855]: reaped child process 2081, status = succeeded worker1 # [ 300.194666] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 301.654894] nix-daemon[855]: reaped child process 2090, status = succeeded worker1 # [ 301.788461] nix-daemon[855]: accepted connection from pid 2106, user nix-grpc-daemon (trusted) worker1 # [ 301.807416] nix-daemon[855]: reaped child process 2109, status = succeeded lb # [ 301.876174] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 301.824448] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.457Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 301.826240] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.457Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 301.920770] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker1 # [ 301.866946] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.502Z level=INFO msg="Uploading 1 narinfos" lb # [ 301.936921] niks3-server[1211]: 2026/09/23 05:37:54 INFO Signed narinfos id=77 count=1 lb # [ 301.939500] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 301.955304] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 301.959828] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete worker1 # [ 301.893490] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.529Z level=INFO msg="Upload complete. (288ms)" lb # [ 301.963472] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 301.967154] niks3-server[1211]: 2026/09/23 05:37:54 INFO Completed upload id=77 lb # [ 301.970512] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 301.913405] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=1 worker1 # [ 301.928311] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 301.935295] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 301.940662] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 301.945402] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 301.950256] nix-daemon[855]: reaped child process 2088, status = succeeded worker1 # [ 301.952185] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.584Z level=INFO msg="All 1 paths already cached" worker1 # [ 301.966665] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 301.972745] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 302.095235] nix-daemon[855]: accepted connection from pid 2115, user nix-grpc-daemon (trusted) worker1 # [ 302.113477] nix-daemon[855]: reaped child process 2118, status = succeeded lb # [ 302.180592] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 302.127986] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.763Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 302.129932] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.763Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 302.210455] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 302.155349] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.790Z level=INFO msg="Uploading 1 narinfos" lb # [ 302.226182] niks3-server[1211]: 2026/09/23 05:37:54 INFO Signed narinfos id=78 count=1 lb # [ 302.233565] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 302.240556] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 302.188700] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.824Z level=INFO msg="Upload complete. (213ms)" lb # [ 302.251956] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 302.258649] niks3-server[1211]: 2026/09/23 05:37:54 INFO Completed upload id=78 worker1 # [ 302.196307] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 302.262097] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 302.210224] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 302.219778] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 302.224649] nix-daemon[855]: reaped child process 2113, status = succeeded worker1 # [ 302.238806] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 303.699346] nix-daemon[855]: reaped child process 2122, status = succeeded worker1 # [ 303.826122] nix-daemon[855]: accepted connection from pid 2138, user nix-grpc-daemon (trusted) worker1 # [ 303.841870] nix-daemon[855]: reaped child process 2141, status = succeeded lb # [ 303.910951] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 303.864474] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.495Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 303.866318] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.495Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 303.964556] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 303.914300] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.549Z level=INFO msg="Uploading 1 narinfos" lb # [ 303.979146] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 303.987403] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 303.998450] niks3-server[1211]: 2026/09/23 05:37:56 INFO Signed narinfos id=79 count=1 worker1 # [ 303.935264] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.570Z level=INFO msg="Upload complete. (286ms)" lb # [ 304.004831] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 304.008939] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 304.013988] niks3-server[1211]: 2026/09/23 05:37:56 INFO Completed upload id=79 lb # [ 304.016000] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 303.950752] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=1 worker1 # [ 303.972603] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 303.980800] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 303.988946] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 303.996437] nix-daemon[855]: reaped child process 2120, status = succeeded worker1 # [ 304.000673] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 304.007098] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.640Z level=INFO msg="All 1 paths already cached" worker1 # [ 304.011314] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 304.024101] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 304.155386] nix-daemon[855]: accepted connection from pid 2147, user nix-grpc-daemon (trusted) worker1 # [ 304.170621] nix-daemon[855]: reaped child process 2150, status = succeeded lb # [ 304.239703] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 304.186361] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.821Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 304.187816] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.821Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 304.268490] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign worker1 # [ 304.215825] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.851Z level=INFO msg="Uploading 1 narinfos" lb # [ 304.283295] niks3-server[1211]: 2026/09/23 05:37:56 INFO Signed narinfos id=80 count=1 lb # [ 304.291546] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 304.240086] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.875Z level=INFO msg="Upload complete. (205ms)" lb # [ 304.302045] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 304.247979] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 304.310871] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 304.314700] niks3-server[1211]: 2026/09/23 05:37:56 INFO Completed upload id=80 lb # [ 304.317237] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 304.259448] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 304.267826] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 304.270633] nix-daemon[855]: reaped child process 2145, status = succeeded worker1 # [ 304.282722] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 305.549053] nix-daemon[855]: reaped child process 2154, status = succeeded worker1 # [ 305.657998] nix-daemon[855]: accepted connection from pid 2170, user nix-grpc-daemon (trusted) worker1 # [ 305.676384] nix-daemon[855]: reaped child process 2173, status = succeeded lb # [ 305.737994] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 305.688420] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.322Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 305.690039] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.322Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 305.783319] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker1 # [ 305.730516] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.365Z level=INFO msg="Uploading 1 narinfos" lb # [ 305.795311] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 305.803894] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 305.811924] niks3-server[1211]: 2026/09/23 05:37:58 INFO Signed narinfos id=81 count=1 worker1 # [ 305.749953] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.385Z level=INFO msg="Upload complete. (258ms)" lb # [ 305.816220] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 305.821913] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 305.825725] niks3-server[1211]: 2026/09/23 05:37:58 INFO Completed upload id=81 lb # [ 305.827230] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 305.762515] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=1 worker1 # [ 305.783261] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 305.789868] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 305.795339] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 305.800627] nix-daemon[855]: reaped child process 2152, status = succeeded worker1 # [ 305.802714] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 305.807826] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.442Z level=INFO msg="All 2 paths already cached" worker1 # [ 305.811653] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 305.825888] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 305.944239] nix-daemon[855]: accepted connection from pid 2179, user nix-grpc-daemon (trusted) worker1 # [ 305.961087] nix-daemon[855]: reaped child process 2182, status = succeeded lb # [ 306.028676] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 305.975833] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.611Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 305.977507] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.611Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 306.060352] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 306.008167] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.643Z level=INFO msg="Uploading 1 narinfos" lb # [ 306.070040] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 306.080842] niks3-server[1211]: 2026/09/23 05:37:58 INFO Signed narinfos id=82 count=1 lb # [ 306.086645] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 306.036031] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.671Z level=INFO msg="Upload complete. (205ms)" lb # [ 306.097996] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 306.107290] niks3-server[1211]: 2026/09/23 05:37:58 INFO Completed upload id=82 worker1 # [ 306.044227] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 306.109932] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 306.057257] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 306.065195] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 306.068566] nix-daemon[855]: reaped child process 2177, status = succeeded worker1 # [ 306.079538] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 307.583585] nix-daemon[855]: reaped child process 2186, status = succeeded worker1 # [ 307.655309] nix-daemon[855]: accepted connection from pid 2202, user nix-grpc-daemon (trusted) worker1 # [ 307.673342] nix-daemon[855]: reaped child process 2205, status = succeeded lb # [ 307.741935] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 307.688404] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.322Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 307.690110] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.323Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 307.785900] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst worker1 # [ 307.736503] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.371Z level=INFO msg="Uploading 1 narinfos" lb # [ 307.801738] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 307.811240] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 307.821246] niks3-server[1211]: 2026/09/23 05:38:00 INFO Signed narinfos id=83 count=1 worker1 # [ 307.758615] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.394Z level=INFO msg="Upload complete. (269ms)" lb # [ 307.823440] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 307.829957] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 307.835461] niks3-server[1211]: 2026/09/23 05:38:00 INFO Completed upload id=83 lb # [ 307.838660] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 307.774289] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=1 worker1 # [ 307.791326] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 307.797079] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 307.800570] nix-daemon[855]: reaped child process 2184, status = succeeded worker1 # [ 307.818130] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 307.831284] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.466Z level=INFO msg="All 2 paths already cached" worker1 # [ 307.841285] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 307.855399] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 308.004220] nix-daemon[855]: accepted connection from pid 2209, user nix-grpc-daemon (trusted) worker1 # [ 308.020197] nix-daemon[855]: reaped child process 2213, status = succeeded lb # [ 308.090005] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 308.039686] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.674Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 308.041862] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.675Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" worker1 # [ 308.065937] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.701Z level=INFO msg="Uploading 1 narinfos" lb # [ 308.122192] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 308.133625] niks3-server[1211]: 2026/09/23 05:38:00 INFO Signed narinfos id=84 count=1 lb # [ 308.144444] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 308.096155] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.731Z level=INFO msg="Upload complete. (227ms)" lb # [ 308.160610] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst lb # [ 308.167519] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 308.175987] niks3-server[1211]: 2026/09/23 05:38:00 INFO Completed upload id=84 worker1 # [ 308.104353] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 308.178761] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 308.121136] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 308.128531] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 308.132540] nix-daemon[855]: reaped child process 2207, status = succeeded worker1 # [ 308.144145] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 308.995336] nix-daemon[855]: reaped child process 2217, status = succeeded worker1 # [ 309.102092] nix-daemon[855]: accepted connection from pid 2233, user nix-grpc-daemon (trusted) worker1 # [ 309.120422] nix-daemon[855]: reaped child process 2236, status = succeeded lb # [ 309.187707] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 309.137414] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 309.138984] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.771Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 309.223977] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 309.235952] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 309.248174] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker1 # [ 309.191814] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.827Z level=INFO msg="Uploading 1 narinfos" lb # [ 309.258365] niks3-server[1211]: 2026/09/23 05:38:01 INFO Signed narinfos id=85 count=1 lb # [ 309.265492] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls worker1 # [ 309.215482] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.851Z level=INFO msg="Upload complete. (261ms)" lb # [ 309.278279] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 309.286314] niks3-server[1211]: 2026/09/23 05:38:01 INFO Completed upload id=85 lb # [ 309.289651] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 309.229313] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=1 worker1 # [ 309.248263] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 309.255342] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 309.259913] nix-daemon[855]: reaped child process 2215, status = succeeded worker1 # [ 309.274254] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 309.286907] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.922Z level=INFO msg="All 2 paths already cached" worker1 # [ 309.296928] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 309.313387] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 309.449266] nix-daemon[855]: accepted connection from pid 2240, user nix-grpc-daemon (trusted) worker1 # [ 309.466925] nix-daemon[855]: reaped child process 2243, status = succeeded lb # [ 309.534496] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 309.480952] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.116Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 309.482476] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.116Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 309.562498] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign worker1 # [ 309.508050] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.143Z level=INFO msg="Uploading 1 narinfos" lb # [ 309.576188] niks3-server[1211]: 2026/09/23 05:38:02 INFO Signed narinfos id=86 count=1 lb # [ 309.579373] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 309.597843] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete worker1 # [ 309.540220] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.175Z level=INFO msg="Upload complete. (216ms)" lb # [ 309.602170] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 309.605991] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo lb # [ 309.616831] niks3-server[1211]: 2026/09/23 05:38:02 INFO Completed upload id=86 worker1 # [ 309.548153] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 309.562464] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 worker1 # [ 309.571671] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 309.575037] nix-daemon[855]: reaped child process 2238, status = succeeded worker1 # [ 309.585074] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted) worker1 # [ 310.492957] nix-daemon[855]: reaped child process 2247, status = succeeded worker1 # [ 310.562918] nix-daemon[855]: accepted connection from pid 2263, user nix-grpc-daemon (trusted) worker1 # [ 310.577930] nix-daemon[855]: reaped child process 2266, status = succeeded lb # [ 310.649075] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 310.599439] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.232Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 310.601350] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.232Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 310.691071] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign worker1 # [ 310.637835] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.273Z level=INFO msg="Uploading 1 narinfos" lb # [ 310.706634] niks3-server[1211]: 2026/09/23 05:38:03 INFO Signed narinfos id=87 count=1 lb # [ 310.712992] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 310.721193] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 310.730778] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls worker1 # [ 310.668492] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.304Z level=INFO msg="Upload complete. (271ms)" lb # [ 310.734452] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 310.738313] niks3-server[1211]: 2026/09/23 05:38:03 INFO Completed upload id=87 lb # [ 310.740201] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 310.681108] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=1 worker1 # [ 310.697596] nix-daemon[855]: reaped child process 2245, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.13 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 310.924904] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 2a609e6589e44209a703c306f76fad59 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client # [ 311.137232] nix[2056]: don't know how to build these paths: client # [ 311.138348] nix[2056]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 311.138824] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 311.134083] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40798 client # [ 311.167270] nix[2056]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker worker1 # [ 311.158300] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 314.877359] postgres[786]: [786] LOG: checkpoint starting: time lb # [ 319.392580] envoy[907]: conn peer=[2001:db8:1::1]:48498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27140 tx=12300 ms=15862 client # [ 341.173525] nix[2056]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 341.186227] nix[2056]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 341.224841] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 341.231964] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 341.236355] systemd[1]: gpu.service: Consumed 204ms CPU time over 30.305s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.94 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 343.155685] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:35Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 344.094103] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 344.102749] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 344.115277] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:36Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 344.115697] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:36Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 344.171895] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 344.187471] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 344.202965] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 344.218010] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 344.383093] nix-daemon[844]: accepted connection from pid 1885, user nix-grpc-daemon (trusted) worker2 # [ 344.402906] nix-daemon[844]: reaped child process 1888, status = succeeded lb # [ 344.461276] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 344.422577] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.050Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 344.424288] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.050Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 344.496133] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign worker2 # [ 344.453402] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.081Z level=INFO msg="Uploading 1 narinfos" lb # [ 344.510426] niks3-server[1211]: 2026/09/23 05:38:37 INFO Signed narinfos id=88 count=1 lb # [ 344.523482] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 344.532103] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 344.551707] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 344.562491] niks3-server[1211]: 2026/09/23 05:38:37 INFO Completed upload id=88 worker2 # [ 344.509575] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.137Z level=INFO msg="Upload complete. (274ms)" lb # [ 344.566465] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 344.520353] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 344.537514] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 344.550513] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 344.555007] nix-daemon[844]: reaped child process 1883, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 344.569698] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 345.827984] nix-daemon[844]: reaped child process 1892, status = succeeded worker2 # [ 345.939661] nix-daemon[844]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 345.960852] nix-daemon[844]: reaped child process 1911, status = succeeded lb # [ 346.014617] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 345.973827] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.600Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 345.975491] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.600Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 346.052148] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls worker2 # [ 346.011100] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.639Z level=INFO msg="Uploading 1 narinfos" lb # [ 346.064117] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign lb # [ 346.067826] niks3-server[1211]: 2026/09/23 05:38:38 INFO Signed narinfos id=89 count=1 lb # [ 346.078680] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 346.088352] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 346.093745] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete worker2 # [ 346.041695] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.669Z level=INFO msg="Upload complete. (257ms)" lb # [ 346.096980] niks3-server[1211]: 2026/09/23 05:38:38 INFO Completed upload id=89 lb # [ 346.100980] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 346.053143] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=1 worker1 # [ 346.050170] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:38Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 346.056162] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:38Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 346.072875] nix-daemon[844]: reaped child process 1890, status = succeeded worker2 # [ 346.104582] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 346.119604] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 346.133697] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.761Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 346.144119] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 346.157144] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 346.313803] nix-daemon[844]: accepted connection from pid 1915, user nix-grpc-daemon (trusted) worker2 # [ 346.329346] nix-daemon[844]: reaped child process 1918, status = succeeded lb # [ 346.387781] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 346.345985] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.973Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 346.347689] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.974Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 346.418239] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign worker2 # [ 346.372322] nix-grpc-daemon[1638]: time=2026-09-23T05:38:39.000Z level=INFO msg="Uploading 1 narinfos" lb # [ 346.432614] niks3-server[1211]: 2026/09/23 05:38:39 INFO Signed narinfos id=90 count=1 lb # [ 346.439147] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst lb # [ 346.451593] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls worker2 # [ 346.405657] nix-grpc-daemon[1638]: time=2026-09-23T05:38:39.033Z level=INFO msg="Upload complete. (232ms)" lb # [ 346.458878] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 346.469085] niks3-server[1211]: 2026/09/23 05:38:39 INFO Completed upload id=90 worker2 # [ 346.414805] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=632 lb # [ 346.473318] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 346.430089] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 346.439274] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 346.442201] nix-daemon[844]: reaped child process 1913, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 346.453139] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 348.081109] nix-daemon[844]: reaped child process 1922, status = succeeded worker2 # [ 348.179882] nix-daemon[844]: accepted connection from pid 1938, user nix-grpc-daemon (trusted) worker2 # [ 348.199837] nix-daemon[844]: reaped child process 1941, status = succeeded lb # [ 348.251786] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 348.212363] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 348.213791] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.838Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 348.294591] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls worker2 # [ 348.255607] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.883Z level=INFO msg="Uploading 1 narinfos" lb # [ 348.308391] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 348.320355] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign lb # [ 348.327106] niks3-server[1211]: 2026/09/23 05:38:40 INFO Signed narinfos id=91 count=1 worker2 # [ 348.281914] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.910Z level=INFO msg="Upload complete. (216ms)" lb # [ 348.336915] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 348.342281] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete worker2 # [ 348.292853] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=1 lb # [ 348.350982] niks3-server[1211]: 2026/09/23 05:38:40 INFO Completed upload id=91 lb # [ 348.353993] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 348.305810] nix-daemon[844]: reaped child process 1920, status = succeeded worker1 # [ 348.300756] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:40Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 348.312906] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker1 # [ 348.306552] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:40Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 348.324957] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 348.336811] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.965Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 348.346836] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 348.358609] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 348.498030] nix-daemon[844]: accepted connection from pid 1945, user nix-grpc-daemon (trusted) worker2 # [ 348.516581] nix-daemon[844]: reaped child process 1948, status = succeeded lb # [ 348.574086] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 348.530570] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.158Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 348.532143] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.158Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 348.603180] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 348.559548] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.187Z level=INFO msg="Uploading 1 narinfos" lb # [ 348.617354] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign lb # [ 348.627222] niks3-server[1211]: 2026/09/23 05:38:41 INFO Signed narinfos id=92 count=1 lb # [ 348.634966] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 348.589557] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.217Z level=INFO msg="Upload complete. (218ms)" lb # [ 348.643714] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 348.648227] niks3-server[1211]: 2026/09/23 05:38:41 INFO Completed upload id=92 worker2 # [ 348.596255] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 348.651464] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 348.609760] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 worker2 # [ 348.618866] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) worker2 # [ 348.621272] nix-daemon[844]: reaped child process 1943, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 348.634439] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted) lb # [ 349.393247] envoy[907]: conn peer=[2001:db8:1::1]:38452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30067 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 349.689164] nix-daemon[844]: reaped child process 1952, status = succeeded worker2 # [ 349.805980] nix-daemon[844]: accepted connection from pid 1968, user nix-grpc-daemon (trusted) worker2 # [ 349.824755] nix-daemon[844]: reaped child process 1971, status = succeeded lb # [ 349.875720] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 349.833788] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 349.835218] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.460Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 349.915777] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 349.870073] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.498Z level=INFO msg="Uploading 1 narinfos" lb # [ 349.931723] niks3-server[1211]: 2026/09/23 05:38:42 INFO Signed narinfos id=93 count=1 lb # [ 349.938060] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 349.948525] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls worker2 # [ 349.903475] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.531Z level=INFO msg="Upload complete. (258ms)" lb # [ 349.957052] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 349.961508] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 349.966843] niks3-server[1211]: 2026/09/23 05:38:42 INFO Completed upload id=93 lb # [ 349.970892] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 349.917197] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=1 worker2 # [ 349.932938] nix-daemon[844]: reaped child process 1950, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.10 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.17 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 351.158787] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:43Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.87 seconds) (finished: run the VM test script, in 352.37 seconds) test script finished in 352.63s cleanup kill QemuMachine (pid 50) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 49) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 0.96 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh