tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker2 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... worker1 # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) worker2: QEMU running (pid 51) client # Disk image does not exist, creating the virtualisation disk image... worker2 # Formatting '/build/vm-state-worker2/tmp.uQSOPTwJsU', fmt=raw size=1073741824 worker1: QEMU running (pid 52) lb # Formatting '/build/vm-state-lb/tmp.TmizxiiMR8', fmt=raw size=1073741824 worker1 # Formatting '/build/vm-state-worker1/tmp.tdXdMITBdE', fmt=raw size=1073741824 lb: QEMU running (pid 50) lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) client # Formatting '/build/vm-state-client/tmp.bxac97Zvk2', fmt=raw size=1073741824 lb # Discarding device blocks: 0/262144 done worker1 # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Filesystem UUID: e86ec5e8-8bca-4b52-9b9c-8ca6e786976c worker1 # Filesystem UUID: eb5679b0-4df0-438c-9004-2a9d3fefaa74 worker2 # Filesystem UUID: 3184b9ef-249a-49ba-8cf1-b67ca6fecee2 lb # Superblock backups stored on blocks: worker1 # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: lb # 32768, 98304, 163840, 229376 worker1 # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 lb # worker1 # worker2 # lb # Allocating group tables: 0/8 done worker1 # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done lb # Writing inode tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done lb # Creating journal (8192 blocks): done worker1 # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done lb # worker1 # worker2 # lb # Virtualisation disk image created. worker1 # Virtualisation disk image created. worker2 # Virtualisation disk image created. lb # Starting virtiofs daemons... worker1 # Starting virtiofs daemons... worker2 # Starting virtiofs daemons... lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.48 seconds) worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # Discarding device blocks: 0/262144 done worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # Filesystem UUID: 90dd4bde-bcaa-4091-be50-17c845d35dc4 worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # Superblock backups stored on blocks: worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # 32768, 98304, 163840, 229376 worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # Allocating group tables: 0/8 done worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client: waiting for unit network-addresses-eth1.service worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client: waiting for the VM to finish booting worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Writing inode tables: 0/8 done worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # Creating journal (8192 blocks): done client # Writing superblocks and filesystem accounting information: 0/8 done worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # Virtualisation disk image created. worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # Starting virtiofs daemons... worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test/init regInfo=/nix/store/6a50qpfalbjwv7f1s4agz8m5sg4l2qn4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6a50qpfalbjwv7f1s4agz8m5sg4l2qn4-closure-info/registration", will be passed to user space. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] random: crng init done client # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] Normal empty client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] Device empty client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] Movable zone start for each node client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] Early memory node ranges client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7nr90ian7b661w5sws26k1lwc9mbmx4q-nixos-system-worker2-test/init regInfo=/nix/store/6r9fszg6kxsc3k2rsf87aynbkvl8fb2b-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6r9fszg6kxsc3k2rsf87aynbkvl8fb2b-closure-info/registration", will be passed to user space. client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000039] arm-pv: using stolen time PV worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000680] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000861] Console: colour dummy device 80x25 lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000869] printk: legacy console [tty0] enabled lb # [ 0.000000] KASLR enabled worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] random: crng init done worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.001061] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] efi: UEFI not found. client # [ 0.001068] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.001141] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.001315] landlock: Up and running. lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.001318] Yama: becoming mindful. worker2 # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.001948] LSM support for eBPF active lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Zone ranges: worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.002105] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] DMA32 empty lb # [ 0.000000] Normal empty client # [ 0.002138] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Device empty client # [ 0.004022] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Movable zone start for each node client # [ 0.004027] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.005293] fsl-mc MSI: its@8080000 domain created lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005384] EFI services will not be available. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.005450] smp: Bringing up secondary CPUs ... lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005458] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.005461] SMP: Total of 1 processors activated. lb # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.005466] CPU: All CPU(s) started at EL1 worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.005477] CPU features: detected: Branch Target Identification lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] random: crng init done lb # [ 0.000000] psci: Trusted OS migration not required client # [ 0.005482] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.005485] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.005488] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.005492] CPU features: detected: Common not Private translations worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.005495] CPU features: detected: CRC32 instructions worker1 # [ 0.000000] Zone ranges: worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005498] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Normal empty lb # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.005501] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.000000] Device empty worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] Movable zone start for each node client # [ 0.005505] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] Early memory node ranges client # [ 0.005508] CPU features: detected: E0PD worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.005511] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005514] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.005517] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.005520] CPU features: detected: Fine Grained Traps worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.005524] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.005529] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.005532] CPU features: detected: LSE atomic instructions worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.005535] CPU features: detected: Privileged Access Never client # [ 0.005538] CPU features: detected: PMUv3 worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000035] arm-pv: using stolen time PV client # [ 0.005541] CPU features: detected: RAS Extension Support lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space. client # [ 0.005543] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.005546] CPU features: detected: Random Number Generator worker2 # [ 0.000523] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.000722] Console: colour dummy device 80x25 lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000729] printk: legacy console [tty0] enabled client # [ 0.005549] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000918] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.005551] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000924] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] software IO TLB: area num 1. client # [ 0.005554] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.001012] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.005559] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.001208] landlock: Up and running. client # [ 0.005600] alternatives: applying system-wide alternatives worker2 # [ 0.001211] Yama: becoming mindful. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.001768] LSM support for eBPF active lb # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.008615] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.001920] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Policy zone: DMA worker2 # [ 0.001951] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.008857] Memory: 1405636K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.004063] rcu: Hierarchical SRCU implementation. client # [ 0.009290] devtmpfs: initialized worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wrqfs8d5q8wxr8pbh005xzrx73np2k9s-nixos-system-worker1-test/init regInfo=/nix/store/bngrdnfx7hv1pjgib4i5amm9vsmkd6h4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.004068] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.011143] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.005323] fsl-mc MSI: its@8080000 domain created lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.005414] EFI services will not be available. worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bngrdnfx7hv1pjgib4i5amm9vsmkd6h4-closure-info/registration", will be passed to user space. client # [ 0.011169] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.005495] smp: Bringing up secondary CPUs ... lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.011357] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.005505] smp: Brought up 1 node, 1 CPU lb # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.011361] 0 pages in range for non-PLT usage worker2 # [ 0.005508] SMP: Total of 1 processors activated. lb # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.011362] 508272 pages in range for PLT usage worker2 # [ 0.005514] CPU: All CPU(s) started at EL1 lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.011469] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.005525] CPU features: detected: Branch Target Identification worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.012259] DMI not present or invalid. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.005529] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.016000] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.005532] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.018283] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.005536] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.018457] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.005540] CPU features: detected: Common not Private translations worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.018654] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.005543] CPU features: detected: CRC32 instructions lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.018675] audit: initializing netlink subsys (disabled) worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.005546] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.019248] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.019250] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.005550] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.019254] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.005553] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.005556] CPU features: detected: E0PD client # [ 0.019257] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.005559] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.019262] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.005562] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.019289] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.019297] cpuidle: using governor ladder worker2 # [ 0.005565] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.019303] cpuidle: using governor menu worker2 # [ 0.005568] CPU features: detected: Fine Grained Traps worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.019496] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.005571] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.019512] ASID allocator initialised with 65536 entries worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.005576] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.020799] Serial: AMBA PL011 UART driver worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.005579] CPU features: detected: LSE atomic instructions client # [ 0.026055] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.005582] CPU features: detected: Privileged Access Never lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) client # [ 0.026218] printk: console [ttyAMA0] enabled worker2 # [ 0.005585] CPU features: detected: PMUv3 worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker2 # [ 0.005588] CPU features: detected: RAS Extension Support worker2 # [ 0.005591] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 worker2 # [ 0.005593] CPU features: detected: Random Number Generator lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.005596] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.005599] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.005602] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.005606] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.005647] alternatives: applying system-wide alternatives worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.008655] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.147021] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.000033] arm-pv: using stolen time PV client # [ 0.147039] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000456] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.008829] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) client # [ 0.147044] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.009212] devtmpfs: initialized lb # [ 0.000647] Console: colour dummy device 80x25 client # [ 0.147048] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000655] printk: legacy console [tty0] enabled worker2 # [ 0.011034] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.147053] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000867] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.011077] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.147057] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.000875] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.011283] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.147061] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.000971] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.001099] landlock: Up and running. client # [ 0.147065] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.011288] 0 pages in range for non-PLT usage lb # [ 0.001102] Yama: becoming mindful. worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.011289] 508272 pages in range for PLT usage lb # [ 0.001580] LSM support for eBPF active worker2 # [ 0.011403] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.001731] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.012188] DMI not present or invalid. client # [ 0.154410] fbcon: Taking over console worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.015577] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.001764] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.154426] ACPI: Interpreter disabled. lb # [ 0.003560] rcu: Hierarchical SRCU implementation. worker2 # [ 0.017823] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.003566] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.018020] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.004799] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.004897] EFI services will not be available. worker1 # [ 0.000029] arm-pv: using stolen time PV worker2 # [ 0.018227] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.005026] smp: Bringing up secondary CPUs ... worker2 # [ 0.018250] audit: initializing netlink subsys (disabled) lb # [ 0.005035] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000386] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.005038] SMP: Total of 1 processors activated. worker2 # [ 0.018816] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.000570] Console: colour dummy device 80x25 lb # [ 0.005043] CPU: All CPU(s) started at EL1 worker1 # [ 0.000577] printk: legacy console [tty0] enabled worker2 # [ 0.018818] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.005054] CPU features: detected: Branch Target Identification worker2 # [ 0.018821] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.005059] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.018824] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.000781] pid_max: default: 32768 minimum: 301 lb # [ 0.005062] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.018830] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.000853] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000972] landlock: Up and running. worker2 # [ 0.018856] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.005066] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.000975] Yama: becoming mindful. worker2 # [ 0.018865] cpuidle: using governor ladder worker1 # [ 0.001420] LSM support for eBPF active lb # [ 0.005070] CPU features: detected: Common not Private translations client # [ 0.163427] iommu: Default domain type: Translated worker2 # [ 0.018872] cpuidle: using governor menu lb # [ 0.005073] CPU features: detected: CRC32 instructions worker1 # [ 0.001567] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.163437] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.019129] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.163782] SCSI subsystem initialized lb # [ 0.005076] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.001599] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.019146] ASID allocator initialised with 65536 entries worker2 # [ 0.020434] Serial: AMBA PL011 UART driver worker1 # [ 0.003517] rcu: Hierarchical SRCU implementation. lb # [ 0.005079] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.165705] usbcore: registered new interface driver usbfs worker1 # [ 0.003521] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.025882] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.165737] usbcore: registered new interface driver hub lb # [ 0.005083] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.004757] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.026039] printk: console [ttyAMA0] enabled lb # [ 0.005086] CPU features: detected: E0PD client # [ 0.165752] usbcore: registered new device driver usb worker1 # [ 0.004849] EFI services will not be available. client # [ 0.166012] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.004913] smp: Bringing up secondary CPUs ... lb # [ 0.005088] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.004921] smp: Brought up 1 node, 1 CPU lb # [ 0.005091] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.166019] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.004924] SMP: Total of 1 processors activated. client # [ 0.166029] PTP clock support registered worker1 # [ 0.004930] CPU: All CPU(s) started at EL1 lb # [ 0.005094] CPU features: detected: Enhanced Virtualization Traps client # [ 0.166078] EDAC MC: Ver: 3.0.0 lb # [ 0.005097] CPU features: detected: Fine Grained Traps worker1 # [ 0.004940] CPU features: detected: Branch Target Identification client # [ 0.170567] scmi_core: SCMI protocol bus registered worker1 # [ 0.004944] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.005101] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.171525] FPGA manager framework lb # [ 0.005107] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.004948] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.172425] vgaarb: loaded lb # [ 0.005110] CPU features: detected: LSE atomic instructions worker1 # [ 0.004951] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.005112] CPU features: detected: Privileged Access Never client # [ 0.173047] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.005115] CPU features: detected: PMUv3 worker1 # [ 0.004955] CPU features: detected: Common not Private translations worker2 # [ 0.144329] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.005118] CPU features: detected: RAS Extension Support worker1 # [ 0.004959] CPU features: detected: CRC32 instructions worker2 # [ 0.144345] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.005121] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.004961] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.144351] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.005123] CPU features: detected: Random Number Generator worker1 # [ 0.004965] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.144355] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.005126] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.004968] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.144359] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.005128] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.004971] CPU features: detected: E0PD worker2 # [ 0.144364] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.005131] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.004973] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.144368] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.005136] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.004977] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.179211] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.144372] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005173] alternatives: applying system-wide alternatives worker1 # [ 0.004980] CPU features: detected: Enhanced Virtualization Traps client # [ 0.179252] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.004984] CPU features: detected: Fine Grained Traps lb # [ 0.008085] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.180887] netfs: FS-Cache loaded worker1 # [ 0.004988] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.180986] pnp: PnP ACPI: disabled worker2 # [ 0.151708] fbcon: Taking over console worker1 # [ 0.004993] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.008274] Memory: 1405692K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) worker2 # [ 0.151723] ACPI: Interpreter disabled. lb # [ 0.008660] devtmpfs: initialized worker1 # [ 0.004996] CPU features: detected: LSE atomic instructions worker1 # [ 0.004999] CPU features: detected: Privileged Access Never lb # [ 0.010532] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.005002] CPU features: detected: PMUv3 lb # [ 0.010554] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.005005] CPU features: detected: RAS Extension Support client # [ 0.184824] NET: Registered PF_INET protocol family worker1 # [ 0.005008] CPU features: detected: RASv1p1 Extension Support lb # [ 0.010746] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.005010] CPU features: detected: Random Number Generator worker1 # [ 0.005013] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.005015] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.005018] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.005024] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.005063] alternatives: applying system-wide alternatives client # [ 0.185722] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.007952] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.160699] iommu: Default domain type: Translated worker2 # [ 0.160709] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.161036] SCSI subsystem initialized worker1 # [ 0.008174] Memory: 1405612K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker1 # [ 0.008543] devtmpfs: initialized worker2 # [ 0.163087] usbcore: registered new interface driver usbfs worker1 # [ 0.010421] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.163120] usbcore: registered new interface driver hub lb # [ 0.010750] 0 pages in range for non-PLT usage worker1 # [ 0.010443] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.010751] 508272 pages in range for PLT usage worker2 # [ 0.163135] usbcore: registered new device driver usb worker2 # [ 0.163393] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.010864] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.010627] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.011630] DMI not present or invalid. worker1 # [ 0.010631] 0 pages in range for non-PLT usage worker2 # [ 0.163399] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.010633] 508272 pages in range for PLT usage lb # [ 0.014813] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.163408] PTP clock support registered worker1 # [ 0.010749] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.163459] EDAC MC: Ver: 3.0.0 lb # [ 0.017019] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.011517] DMI not present or invalid. worker2 # [ 0.167990] scmi_core: SCMI protocol bus registered worker1 # [ 0.014588] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.017195] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.016748] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.168973] FPGA manager framework lb # [ 0.017393] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.169887] vgaarb: loaded lb # [ 0.017415] audit: initializing netlink subsys (disabled) lb # [ 0.017947] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.170514] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.017949] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.017952] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.017955] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.017960] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.018005] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018013] cpuidle: using governor ladder lb # [ 0.018019] cpuidle: using governor menu lb # [ 0.018205] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.176731] VFS: Disk quotas dquot_6.6.0 lb # [ 0.018220] ASID allocator initialised with 65536 entries worker2 # [ 0.176777] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.019395] Serial: AMBA PL011 UART driver worker1 # [ 0.016915] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.178416] netfs: FS-Cache loaded lb # [ 0.024642] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.178961] pnp: PnP ACPI: disabled worker1 # [ 0.017108] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.024760] printk: console [ttyAMA0] enabled worker1 # [ 0.017128] audit: initializing netlink subsys (disabled) worker1 # [ 0.017643] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.146049] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.017645] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.146068] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.017649] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.146073] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.182396] NET: Registered PF_INET protocol family worker1 # [ 0.017652] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.146077] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.017657] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.146082] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.017681] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.146086] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.017689] cpuidle: using governor ladder worker1 # [ 0.017695] cpuidle: using governor menu lb # [ 0.146091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.017887] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.146095] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.017902] ASID allocator initialised with 65536 entries worker2 # [ 0.183297] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.019127] Serial: AMBA PL011 UART driver worker1 # [ 0.024325] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.153759] fbcon: Taking over console worker1 # [ 0.024493] printk: console [ttyAMA0] enabled lb # [ 0.153773] ACPI: Interpreter disabled. lb # [ 0.155677] iommu: Default domain type: Translated worker1 # [ 0.144509] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.155687] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.144527] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.157492] SCSI subsystem initialized worker1 # [ 0.144532] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.214239] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.144537] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.214286] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.144541] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.144545] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.214315] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.144549] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.214388] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.144553] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.214530] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.214612] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.214647] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.152096] fbcon: Taking over console client # [ 0.214682] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.152111] ACPI: Interpreter disabled. client # [ 0.214772] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.154056] iommu: Default domain type: Translated client # [ 0.214799] NET: Registered PF_XDP protocol family lb # [ 0.165197] usbcore: registered new interface driver usbfs worker1 # [ 0.154066] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.214817] PCI: CLS 0 bytes, default 64 lb # [ 0.165223] usbcore: registered new interface driver hub client # [ 0.215067] Trying to unpack rootfs image as initramfs... worker1 # [ 0.155821] SCSI subsystem initialized lb # [ 0.165266] usbcore: registered new device driver usb lb # [ 0.165553] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.165559] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.165569] PTP clock support registered lb # [ 0.165608] EDAC MC: Ver: 3.0.0 lb # [ 0.170227] scmi_core: SCMI protocol bus registered lb # [ 0.171234] FPGA manager framework lb # [ 0.172233] vgaarb: loaded lb # [ 0.172870] clocksource: Switched to clocksource arch_sys_counter client # [ 0.230840] kvm [1]: HYP mode not available lb # [ 0.174274] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.162963] usbcore: registered new interface driver usbfs lb # [ 0.174304] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.162998] usbcore: registered new interface driver hub worker1 # [ 0.163015] usbcore: registered new device driver usb lb # [ 0.175871] netfs: FS-Cache loaded lb # [ 0.175993] pnp: PnP ACPI: disabled worker1 # [ 0.163309] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.163316] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.163334] PTP clock support registered worker1 # [ 0.163393] EDAC MC: Ver: 3.0.0 worker1 # [ 0.168027] scmi_core: SCMI protocol bus registered worker1 # [ 0.169014] FPGA manager framework worker1 # [ 0.170200] vgaarb: loaded worker1 # [ 0.170824] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.172879] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.172907] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.212512] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.182054] NET: Registered PF_INET protocol family lb # [ 0.182315] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.212558] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.212585] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.212656] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.176674] netfs: FS-Cache loaded worker1 # [ 0.176796] pnp: PnP ACPI: disabled worker2 # [ 0.212799] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.212884] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.212920] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.212955] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.213053] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.213073] NET: Registered PF_XDP protocol family worker2 # [ 0.213090] PCI: CLS 0 bytes, default 64 worker2 # [ 0.213348] Trying to unpack rootfs image as initramfs... worker1 # [ 0.180733] NET: Registered PF_INET protocol family worker1 # [ 0.180996] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.228298] kvm [1]: HYP mode not available lb # [ 0.211452] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.211503] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.210794] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.211530] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.211606] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.211749] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.211832] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.211875] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.211911] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.212008] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.212029] NET: Registered PF_XDP protocol family lb # [ 0.212046] PCI: CLS 0 bytes, default 64 lb # [ 0.212300] Trying to unpack rootfs image as initramfs... worker1 # [ 0.212069] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.212099] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.212170] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.212314] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.212400] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.212437] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212473] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212565] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.212585] NET: Registered PF_XDP protocol family lb # [ 0.230667] kvm [1]: HYP mode not available worker1 # [ 0.212605] PCI: CLS 0 bytes, default 64 worker1 # [ 0.212834] Trying to unpack rootfs image as initramfs... worker1 # [ 0.234854] kvm [1]: HYP mode not available client # [ 0.321603] Initialise system trusted keyrings client # [ 0.322442] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.323687] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.324465] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.323244] Initialise system trusted keyrings client # [ 0.353246] Key type asymmetric registered client # [ 0.353271] Asymmetric key parser 'x509' registered worker2 # [ 0.323976] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.353338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.325646] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.355466] io scheduler mq-deadline registered worker2 # [ 0.326418] 9p: Installing v9fs 9p2000 file system support client # [ 0.355477] io scheduler kyber registered client # [ 0.365189] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.366597] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.366987] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.367004] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.367016] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.367025] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.367046] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.367071] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.367144] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.367154] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.367160] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.367165] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.367170] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.367256] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.317850] Initialise system trusted keyrings client # [ 0.367685] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.367866] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.367882] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.367911] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.367927] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.355179] Key type asymmetric registered worker2 # [ 0.355204] Asymmetric key parser 'x509' registered client # [ 0.368377] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.368581] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.355274] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.368596] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.357396] io scheduler mq-deadline registered worker2 # [ 0.357406] io scheduler kyber registered client # [ 0.368625] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.324945] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.388098] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.326368] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.388296] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.388313] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.388343] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.327177] 9p: Installing v9fs 9p2000 file system support client # [ 0.388898] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.366663] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.327386] Initialise system trusted keyrings worker1 # [ 0.328128] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.368162] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.329402] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.368585] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.330316] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.368603] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.368615] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.368624] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.368648] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.401170] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.401201] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.368674] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.401232] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.368747] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.347894] Key type asymmetric registered client # [ 0.401251] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.368756] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.347915] Asymmetric key parser 'x509' registered worker2 # [ 0.368762] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.401764] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.348000] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.368767] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.401953] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.368772] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.401983] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.368859] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.402441] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.402627] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.369288] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.402656] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.369468] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.369485] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.403044] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.369514] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.403225] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.369531] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.403515] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.369998] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.403701] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.357004] io scheduler mq-deadline registered worker2 # [ 0.370177] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.357022] io scheduler kyber registered client # [ 0.403731] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.370192] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.404179] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.370222] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.404366] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.370688] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.404421] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.370872] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.404874] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.370888] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.405077] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.370917] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.405107] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.362086] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.371379] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.405566] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.371560] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.405879] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.371575] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.405898] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.363377] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.371605] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.405927] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363755] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.371623] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.406394] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.372072] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.363772] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.406576] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.372259] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.363785] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.406592] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.372289] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.406622] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363794] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.372743] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.407212] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.363814] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.372932] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.359063] Key type asymmetric registered client # [ 0.407224] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.359090] Asymmetric key parser 'x509' registered worker2 # [ 0.372964] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363848] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.407229] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.359169] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.373374] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.363930] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.361200] io scheduler mq-deadline registered client # [ 0.407275] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.373558] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.363939] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.361211] io scheduler kyber registered lb # [ 0.363945] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.407321] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.373829] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.363950] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.374016] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.407368] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.363955] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.374046] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.407414] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.374490] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.364055] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.407460] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.364499] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.407506] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.364689] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.407552] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.364705] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.407598] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.364736] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364752] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.407645] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.407718] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.407763] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.407786] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.370977] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.407807] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.407828] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.407849] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned client # [ 0.407871] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.372397] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.407893] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.372809] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.407917] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.372827] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.407939] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.372839] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.407961] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.385324] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.385530] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.407984] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.372848] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.385548] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408005] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.372874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.385578] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408027] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.372899] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.408048] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.386050] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.372971] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.408069] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.386236] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.372981] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.408090] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.386252] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.372987] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.408111] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.386282] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408132] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.372992] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.386735] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.422782] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408159] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.372997] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.386919] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.422828] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408169] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.386935] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.373095] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.408174] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.423378] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.386965] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.373529] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.408986] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.423566] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.386984] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.373715] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.423596] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.373731] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.387444] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.424063] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.387632] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.373761] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.424371] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.373777] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.387662] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.424389] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.374251] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.388123] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.424419] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.374433] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.388312] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.424877] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.374449] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388343] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425060] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.465206] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 54906 usecs worker1 # [ 0.374479] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425076] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388736] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.425106] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425682] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.393413] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.425694] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.393601] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.393617] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.425699] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.393647] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425749] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.394112] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.425800] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.394294] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.425850] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.394310] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.394339] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425900] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.394357] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.425947] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.409000] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.394805] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.425995] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.409345] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.426043] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.409540] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.409571] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426089] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.410031] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.426135] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.410220] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426213] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.410249] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426260] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.410705] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.426282] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.410893] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426303] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.410924] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426324] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.411384] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.426346] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.411686] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.426367] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.411704] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426389] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.411734] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426415] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.412204] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.426437] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.412389] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.485943] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.407050] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426459] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.488146] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.412405] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.407091] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426481] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.412435] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426503] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.407574] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.426550] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.407762] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426572] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.407791] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426593] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.408178] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.426614] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.408360] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.426635] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.408660] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.426656] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.408852] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.426682] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.426692] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.408882] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.426696] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.409341] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.427519] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.409527] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.409557] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.410018] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.433116] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.410205] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.499343] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.433140] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.410234] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.433145] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.410695] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.433199] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.410994] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.411010] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.433248] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.411040] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.433297] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.411499] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.433346] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.411681] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.433394] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.411697] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.411726] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.433443] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.412296] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.433490] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.412308] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.433538] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.412313] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.433585] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.412358] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.433710] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.412405] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.433759] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.412452] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.433781] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.433804] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.412498] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.433825] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.412545] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.433847] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.412591] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.509330] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.433869] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.513303] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.412637] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.433891] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.515466] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.412682] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.433919] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.412729] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.433942] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.488469] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.412814] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.433968] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.433991] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.412875] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.412897] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.434014] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.434036] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.412918] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.434058] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.412939] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.434081] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.412960] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.434103] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.412981] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.434124] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.413002] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.434147] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.413026] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.434175] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.413048] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.434186] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.413070] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.525259] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.434191] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.413092] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.435030] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.527322] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.413114] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.413134] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.498737] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.413155] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.413176] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.501627] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.413197] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.413218] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.413239] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.413264] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.413273] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.413278] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.414093] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.511591] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.537216] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.514424] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.539095] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.541601] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.518125] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.485941] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.488116] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.554655] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.523716] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.526147] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.557138] msm_serial: driver initialized client # [ 0.557321] SuperH (H)SCI(F) driver initialized client # [ 0.557375] STM32 USART driver initialized lb # [ 0.499240] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.490709] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.535842] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.538135] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.500237] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.547920] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.509121] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.512276] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.514495] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.516402] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.561778] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.526423] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.591856] loop: module loaded lb # [ 0.528316] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.511441] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.592034] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.513608] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.592848] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.515667] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.517776] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.564328] msm_serial: driver initialized worker2 # [ 0.564512] SuperH (H)SCI(F) driver initialized worker2 # [ 0.564565] STM32 USART driver initialized client # [ 0.597675] megasas: 07.734.00.00-rc1 client # [ 0.598378] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.539566] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.528229] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.530141] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.546072] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.609761] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.609794] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.611416] Using buffer write method client # [ 0.611482] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.541478] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.544246] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.546489] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.621098] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.621128] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.624390] Using buffer write method lb # [ 0.562140] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.624428] Concatenating MTD devices: client # [ 0.624432] (0): "0.flash" client # [ 0.624441] (1): "0.flash" client # [ 0.624445] into device "0.flash" lb # [ 0.564783] msm_serial: driver initialized worker2 # [ 0.598760] loop: module loaded worker2 # [ 0.598947] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.599854] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.565524] SuperH (H)SCI(F) driver initialized lb # [ 0.565585] STM32 USART driver initialized worker1 # [ 0.562892] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.565448] msm_serial: driver initialized worker1 # [ 0.565628] SuperH (H)SCI(F) driver initialized worker1 # [ 0.565683] STM32 USART driver initialized worker2 # [ 0.607231] megasas: 07.734.00.00-rc1 worker2 # [ 0.607976] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.609873] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.609896] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.611705] Using buffer write method worker2 # [ 0.611762] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.613633] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.613656] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.595386] loop: module loaded lb # [ 0.595572] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.596328] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.633285] Using buffer write method worker2 # [ 0.633320] Concatenating MTD devices: worker2 # [ 0.633324] (0): "0.flash" worker2 # [ 0.633327] (1): "0.flash" worker2 # [ 0.633336] into device "0.flash" lb # [ 0.601509] megasas: 07.734.00.00-rc1 lb # [ 0.602208] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.604443] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.604494] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.599066] loop: module loaded worker1 # [ 0.599264] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.600055] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.613895] Using buffer write method lb # [ 0.613973] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.616857] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.607420] megasas: 07.734.00.00-rc1 worker1 # [ 0.608123] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.624933] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.626452] Using buffer write method lb # [ 0.626482] Concatenating MTD devices: worker1 # [ 0.610331] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.626486] (0): "0.flash" worker1 # [ 0.610357] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.626490] (1): "0.flash" worker1 # [ 0.611854] Using buffer write method lb # [ 0.626500] into device "0.flash" worker1 # [ 0.611927] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.613579] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.613601] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.632425] Using buffer write method worker1 # [ 0.632460] Concatenating MTD devices: worker1 # [ 0.632465] (0): "0.flash" worker1 # [ 0.632469] (1): "0.flash" worker1 # [ 0.632478] into device "0.flash" client # [ 0.876776] Freeing initrd memory: 26972K client # [ 0.882807] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.886763] thunder_xcv, ver 1.0 client # [ 0.886814] thunder_bgx, ver 1.0 client # [ 0.886836] nicpf, ver 1.0 client # [ 0.887407] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.887415] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.887440] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.887447] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.887476] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.887482] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.887505] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.887511] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.887645] sky2: driver version 1.30 client # [ 0.889799] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.889831] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.890108] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.899826] usbcore: registered new interface driver usb-storage client # [ 0.899917] usbcore: registered new interface driver usbserial_generic client # [ 0.899931] usbserial: USB Serial support registered for generic client # [ 0.900535] hv_vmbus: registering driver hyperv_keyboard client # [ 0.901094] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.901467] hub 1-0:1.0: USB hub found client # [ 0.901495] hub 1-0:1.0: 6 ports detected client # [ 0.906797] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.906828] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665) client # [ 0.907187] i2c_dev: i2c /dev entries driver client # [ 0.912554] sdhci: Secure Digital Host Controller Interface driver client # [ 0.912572] sdhci: Copyright(c) Pierre Ossman client # [ 0.912836] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.884051] Freeing initrd memory: 26944K client # [ 0.915219] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.916795] hid: raw HID events driver (C) Jiri Kosina client # [ 0.917037] usbcore: registered new interface driver usbhid client # [ 0.917044] usbhid: USB HID core driver worker2 # [ 0.890182] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.920674] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.893935] thunder_xcv, ver 1.0 client # [ 0.922222] drop_monitor: Initializing network drop monitor service worker2 # [ 0.893978] thunder_bgx, ver 1.0 client # [ 0.922377] NET: Registered PF_INET6 protocol family worker2 # [ 0.894001] nicpf, ver 1.0 client # [ 0.925588] Segment Routing with IPv6 client # [ 0.925610] In-situ OAM (IOAM) with IPv6 client # [ 0.925637] NET: Registered PF_PACKET protocol family worker2 # [ 0.895797] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.927232] 9pnet: Installing 9P2000 support client # [ 0.927283] Key type dns_resolver registered worker2 # [ 0.895809] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.895835] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.895845] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.895875] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.895881] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.895925] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.895931] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.896072] sky2: driver version 1.30 worker2 # [ 0.897668] usbcore: registered new interface driver usb-storage worker2 # [ 0.897717] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.897736] usbserial: USB Serial support registered for generic client # [ 0.934009] registered taskstats version 1 worker2 # [ 0.898334] hv_vmbus: registering driver hyperv_keyboard client # [ 0.934175] Loading compiled-in X.509 certificates worker2 # [ 0.899236] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.899263] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.899475] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.910842] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.911265] hub 1-0:1.0: USB hub found worker2 # [ 0.911285] hub 1-0:1.0: 6 ports detected client # [ 0.942841] Demotion targets for Node 0: null lb # [ 0.878692] Freeing initrd memory: 26948K client # [ 0.942956] Key type .fscrypt registered worker2 # [ 0.914053] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.942962] Key type fscrypt-provisioning registered worker1 # [ 0.870326] Freeing initrd memory: 26976K worker2 # [ 0.914082] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665) client # [ 0.943063] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.914389] i2c_dev: i2c /dev entries driver client # [ 0.943083] ima: Allocated hash algorithm: sha1 client # [ 0.943106] ima: No architecture policies found client # [ 0.947218] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.884729] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.876362] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.919727] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.888559] thunder_xcv, ver 1.0 worker2 # [ 0.919738] sdhci: Copyright(c) Pierre Ossman lb # [ 0.888605] thunder_bgx, ver 1.0 lb # [ 0.888627] nicpf, ver 1.0 worker2 # [ 0.920005] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.920376] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.921894] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.922175] usbcore: registered new interface driver usbhid worker2 # [ 0.922181] usbhid: USB HID core driver lb # [ 0.890465] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.890475] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.880241] thunder_xcv, ver 1.0 worker1 # [ 0.880284] thunder_bgx, ver 1.0 lb # [ 0.890500] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.880308] nicpf, ver 1.0 lb # [ 0.890508] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.880877] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.927854] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.890538] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.880884] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.929406] drop_monitor: Initializing network drop monitor service lb # [ 0.890544] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.880909] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.929576] NET: Registered PF_INET6 protocol family lb # [ 0.890568] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.880919] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.932561] Segment Routing with IPv6 worker1 # [ 0.880949] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.890573] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.932582] In-situ OAM (IOAM) with IPv6 lb # [ 0.890711] sky2: driver version 1.30 worker1 # [ 0.880955] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.932627] NET: Registered PF_PACKET protocol family lb # [ 0.892286] usbcore: registered new interface driver usb-storage worker1 # [ 0.880999] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.934229] 9pnet: Installing 9P2000 support lb # [ 0.892380] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.881005] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.934276] Key type dns_resolver registered worker1 # [ 0.881142] sky2: driver version 1.30 lb # [ 0.892394] usbserial: USB Serial support registered for generic client # [ 0.964846] clk: Disabling unused clocks worker1 # [ 0.882776] usbcore: registered new interface driver usb-storage lb # [ 0.893650] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.964877] PM: genpd: Disabling unused power domains worker1 # [ 0.883570] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.893676] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.883596] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.893842] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.883763] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.904717] hv_vmbus: registering driver hyperv_keyboard client # [ 0.968994] Freeing unused kernel memory: 4736K lb # [ 0.905600] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.969783] Run /init as init process lb # [ 0.906635] hub 1-0:1.0: USB hub found worker1 # [ 0.893897] usbcore: registered new interface driver usbserial_generic lb # [ 0.907114] hub 1-0:1.0: 6 ports detected worker1 # [ 0.893922] usbserial: USB Serial support registered for generic worker1 # [ 0.894580] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.941011] registered taskstats version 1 worker1 # [ 0.894877] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.941169] Loading compiled-in X.509 certificates lb # [ 0.908840] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.895185] hub 1-0:1.0: USB hub found worker1 # [ 0.895205] hub 1-0:1.0: 6 ports detected lb # [ 0.909539] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665) lb # [ 0.909855] i2c_dev: i2c /dev entries driver worker1 # [ 0.899944] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.899972] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665) worker1 # [ 0.900271] i2c_dev: i2c /dev entries driver worker2 # [ 0.949760] Demotion targets for Node 0: null worker2 # [ 0.949872] Key type .fscrypt registered lb # [ 0.914632] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.949878] Key type fscrypt-provisioning registered lb # [ 0.914642] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.949975] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.905652] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.914906] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.949994] ima: Allocated hash algorithm: sha1 worker1 # [ 0.905664] sdhci: Copyright(c) Pierre Ossman lb # [ 0.915279] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.950016] ima: No architecture policies found worker1 # [ 0.905947] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.916780] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.906318] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.983282] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.954267] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.920350] usbcore: registered new interface driver usbhid lb # [ 0.920362] usbhid: USB HID core driver worker1 # [ 0.910501] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.910768] usbcore: registered new interface driver usbhid worker1 # [ 0.910775] usbhid: USB HID core driver lb # [ 0.922709] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.924248] drop_monitor: Initializing network drop monitor service worker1 # [ 0.913669] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.924391] NET: Registered PF_INET6 protocol family lb # [ 0.927569] Segment Routing with IPv6 worker1 # [ 0.916386] drop_monitor: Initializing network drop monitor service lb # [ 0.927587] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.916535] NET: Registered PF_INET6 protocol family lb # [ 0.927616] NET: Registered PF_PACKET protocol family worker1 # [ 0.918447] Segment Routing with IPv6 worker1 # [ 0.918466] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.918492] NET: Registered PF_PACKET protocol family lb # [ 0.929516] 9pnet: Installing 9P2000 support lb # [ 0.929564] Key type dns_resolver registered worker1 # [ 0.920135] 9pnet: Installing 9P2000 support worker1 # [ 0.920180] Key type dns_resolver registered lb # [ 0.936045] registered taskstats version 1 lb # [ 0.936198] Loading compiled-in X.509 certificates worker2 # [ 0.972401] clk: Disabling unused clocks worker2 # [ 0.972426] PM: genpd: Disabling unused power domains worker2 # [ 0.976617] Freeing unused kernel memory: 4736K worker2 # [ 0.976858] Run /init as init process lb # [ 0.944807] Demotion targets for Node 0: null lb # [ 0.945660] Key type .fscrypt registered worker1 # [ 0.927286] registered taskstats version 1 worker1 # [ 0.927428] Loading compiled-in X.509 certificates lb # [ 0.945672] Key type fscrypt-provisioning registered lb # [ 0.945770] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.936033] Demotion targets for Node 0: null lb # [ 0.945789] ima: Allocated hash algorithm: sha1 worker1 # [ 0.936145] Key type .fscrypt registered lb # [ 0.945810] ima: No architecture policies found worker1 # [ 0.936151] Key type fscrypt-provisioning registered worker1 # [ 0.936245] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.949329] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.936265] ima: Allocated hash algorithm: sha1 worker1 # [ 0.936286] ima: No architecture policies found worker1 # [ 0.940557] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.992855] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.967757] clk: Disabling unused clocks lb # [ 0.967786] PM: genpd: Disabling unused power domains worker1 # [ 0.957886] clk: Disabling unused clocks worker1 # [ 0.957908] PM: genpd: Disabling unused power domains lb # [ 0.972005] Freeing unused kernel memory: 4736K lb # [ 0.972223] Run /init as init process worker1 # [ 0.962090] Freeing unused kernel memory: 4736K worker1 # [ 0.962277] Run /init as init process lb # [ 0.987097] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.977011] systemd[1]: Successfully made /usr/ read-only. client # [ 1.149120] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.158605] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.152961] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.146909] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.301614] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.317887] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.329994] systemd[1]: Detected virtualization qemu. client # [ 1.331975] systemd[1]: Detected architecture arm64. client # [ 1.333965] systemd[1]: Running in initrd. client # [ 1.336582] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.308597] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.339540] systemd[1]: Hostname set to . lb # [ 1.302716] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.327779] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.340020] systemd[1]: Detected virtualization qemu. worker2 # [ 1.342005] systemd[1]: Detected architecture arm64. worker1 # [ 1.296688] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.344035] systemd[1]: Running in initrd. worker2 # [ 1.346759] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.349719] systemd[1]: Hostname set to . client # [ 1.389340] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.321973] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.311935] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.324171] systemd[1]: Detected virtualization qemu. lb # [ 1.334272] systemd[1]: Detected virtualization qemu. worker1 # [ 1.326182] systemd[1]: Detected architecture arm64. lb # [ 1.334379] systemd[1]: Detected architecture arm64. worker1 # [ 1.328188] systemd[1]: Running in initrd. lb # [ 1.334404] systemd[1]: Running in initrd. lb # [ 1.335394] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.330951] systemd[1]: Initializing machine ID from random generator. lb # [ 1.335842] systemd[1]: Hostname set to . worker1 # [ 1.334051] systemd[1]: Hostname set to . worker2 # [ 1.394833] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.389186] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.379125] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.509099] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.518602] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.508921] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.502917] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.648007] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.669769] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.674972] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.655283] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.618522] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.673848] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.679748] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.658303] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.658724] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.661392] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.678613] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.678968] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.759593] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.770956] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.771896] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.772901] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.773777] systemd[1]: Reached target Path Units. client # [ 1.774387] systemd[1]: Reached target Slice Units. client # [ 1.775123] systemd[1]: Reached target Swaps. worker1 # [ 1.700170] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.775700] systemd[1]: Reached target Timer Units. client # [ 1.776490] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.777462] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.778321] systemd[1]: Listening on Journal Sockets. client # [ 1.779070] systemd[1]: Listening on udev Control Socket. client # [ 1.779852] systemd[1]: Listening on udev Kernel Socket. client # [ 1.780553] systemd[1]: Reached target Socket Units. client # [ 1.782538] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.709304] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.782605] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.710483] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.711879] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.712883] systemd[1]: Reached target Path Units. worker1 # [ 1.713658] systemd[1]: Reached target Slice Units. worker1 # [ 1.714454] systemd[1]: Reached target Swaps. worker1 # [ 1.715209] systemd[1]: Reached target Timer Units. worker1 # [ 1.716167] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.717469] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.792571] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.718563] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.719561] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.719692] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.719716] systemd[1]: Reached target Socket Units. worker2 # [ 1.769526] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.723613] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.723691] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.734779] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.780956] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.782177] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.783628] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.813366] systemd[1]: Starting Journal Service... worker2 # [ 1.784723] systemd[1]: Reached target Path Units. worker2 # [ 1.785553] systemd[1]: Reached target Slice Units. worker2 # [ 1.786404] systemd[1]: Reached target Swaps. client # [ 1.815725] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.787287] systemd[1]: Reached target Timer Units. client # [ 1.816658] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.788334] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.789599] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.790774] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.790918] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.791057] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.791080] systemd[1]: Reached target Socket Units. worker2 # [ 1.796318] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.797485] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.828031] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.756006] systemd[1]: Starting Journal Service... lb # [ 1.768998] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.758582] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.758713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.806712] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.767066] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.779107] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.842860] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.780161] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.814821] systemd[1]: Starting Journal Service... client # [ 1.843545] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.781390] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.782325] systemd[1]: Reached target Path Units. lb # [ 1.783095] systemd[1]: Reached target Slice Units. lb # [ 1.783814] systemd[1]: Reached target Swaps. lb # [ 1.784461] systemd[1]: Reached target Timer Units. lb # [ 1.785391] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.786452] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.787451] systemd[1]: Listening on Journal Sockets. lb # [ 1.788317] systemd[1]: Listening on udev Control Socket. lb # [ 1.789247] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.789272] systemd[1]: Reached target Socket Units. lb # [ 1.792143] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.781403] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.782256] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.793229] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.857399] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.801042] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.795535] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.809291] systemd[1]: Starting Journal Service... worker2 # [ 1.846767] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.846906] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.864639] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.865810] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.900345] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.841555] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.830820] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.841682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.906560] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.880110] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.880762] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.849320] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.913459] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.841662] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.921482] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.894900] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.867490] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.869947] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.937218] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.863411] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.866351] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.882176] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.871292] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.946540] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.946789] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.946799] [drm] features: -context_init client # [ 1.947490] [drm] number of scanouts: 1 client # [ 1.947508] [drm] number of cap sets: 0 worker1 # [ 1.884120] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.884364] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.884374] [drm] features: -context_init worker1 # [ 1.885105] [drm] number of scanouts: 1 worker1 # [ 1.885123] [drm] number of cap sets: 0 worker2 # [ 1.931170] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.932795] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.900790] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.902752] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.965166] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.965378] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.965406] systemd[1]: Reached target Local File Systems. client # [ 1.969151] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.943428] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.913100] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.973533] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.973553] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.903141] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.903161] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.926169] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.926404] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.926414] [drm] features: -context_init lb # [ 1.927126] [drm] number of scanouts: 1 lb # [ 1.927144] [drm] number of cap sets: 0 worker2 # [ 1.958691] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.964806] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.965046] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.965056] [drm] features: -context_init worker2 # [ 1.965778] [drm] number of scanouts: 1 lb # [ 1.933624] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.965797] [drm] number of cap sets: 0 lb # [ 1.937260] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.931172] Console: switching to colour frame buffer device 160x50 client # [ 2.005333] Console: switching to colour frame buffer device 160x50 client # [ 2.012065] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 1.987194] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.987549] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.987577] systemd[1]: Reached target Local File Systems. worker1 # [ 1.947559] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.947770] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.947798] systemd[1]: Reached target Local File Systems. lb # [ 1.957235] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.957254] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.994906] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.955209] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.999999] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 2.000017] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.960773] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.977623] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 2.041716] systemd[1]: Finished Load Kernel Modules. lb # [ 1.977815] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.977841] systemd[1]: Reached target Local File Systems. worker1 # [ 1.972124] systemd[1]: Finished Load Kernel Modules. lb # [ 1.985243] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.019015] Console: switching to colour frame buffer device 160x50 lb # [ 1.990259] Console: switching to colour frame buffer device 160x50 client # [ 2.057490] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.039147] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.006709] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.995092] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.054831] systemd[1]: Started Journal Service. lb # [ 2.021523] systemd[1]: Finished Load Kernel Modules. lb # [ 2.025069] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.015075] systemd[1]: Started Journal Service. worker2 # [ 2.048335] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.090208] systemd[1]: Started Journal Service. worker2 # [ 2.049603] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.076449] systemd-modules-load[73]: Inserted module 'dm_mod' worker2 # [ 2.050750] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.051802] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 2.080469] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.081582] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.082628] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 2.083572] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.007454] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.091092] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.064408] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.065495] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.066448] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.016929] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.018061] systemd-modules-load[73]: Module 'virtio_console' is built in worker1 # [ 2.019130] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 2.098323] systemd[1]: Starting Create System Files and Directories... client # [ 2.099406] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.024211] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.044415] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 2.032372] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.081904] systemd[1]: Starting Create System Files and Directories... lb # [ 2.045527] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.034713] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.063471] systemd[1]: Started Journal Service. worker2 # [ 2.092450] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 2.056431] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.063970] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.051604] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker1 # [ 2.055016] systemd[1]: Finished Create System Files and Directories. client # [ 2.133073] systemd[1]: Finished Create System Files and Directories. client # [ 2.135480] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.072373] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.075902] systemd[1]: Starting Create System Files and Directories... lb # [ 2.081428] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.125409] systemd[1]: Finished Create System Files and Directories. lb # [ 2.092411] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.132233] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.086219] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.099583] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.144355] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.125167] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.196511] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.142453] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.200767] systemd[1]: Starting Virtual Console Setup... lb # [ 2.184144] systemd[1]: Starting Virtual Console Setup... client # [ 2.248515] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.251978] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.196517] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.200131] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.256483] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.260000] systemd[1]: Finished Virtual Console Setup. lb # [ 2.232482] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.235913] systemd[1]: Finished Virtual Console Setup. client # [ 2.827761] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.828783] systemd[1]: Reached target System Initialization. client # [ 2.829585] systemd[1]: Reached target Basic System. worker1 # [ 2.797384] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.798357] systemd[1]: Reached target System Initialization. worker1 # [ 2.799187] systemd[1]: Reached target Basic System. lb # [ 2.840182] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.841773] systemd[1]: Reached target System Initialization. lb # [ 2.844120] systemd[1]: Reached target Basic System. worker2 # [ 2.899993] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.901000] systemd[1]: Reached target System Initialization. worker2 # [ 2.901807] systemd[1]: Reached target Basic System. client # [ 3.000452] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.004725] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.008640] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.946141] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.982759] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.978438] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.983091] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.069798] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.072727] systemd[1]: Reached target Initrd Root Device. client # [ 3.076466] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.052404] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.026675] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.032381] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.092482] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.049663] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.099315] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.130727] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.056273] systemd[1]: Reached target Initrd Root Device. client # [ 3.137811] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.061959] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.142476] systemd[1]: Mounting /sysroot... lb # [ 3.096410] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.101455] systemd[1]: Reached target Initrd Root Device. lb # [ 3.105532] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.195567] EXT4-fs (vda): mounted filesystem 90dd4bde-bcaa-4091-be50-17c845d35dc4 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.156137] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.188096] systemd[1]: Mounted /sysroot. worker2 # [ 3.162377] systemd[1]: Reached target Initrd Root Device. client # [ 3.189591] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.114851] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.167237] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.196108] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.122383] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.125310] systemd[1]: Mounting /sysroot... client # [ 3.219186] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.157985] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.223910] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.165912] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.230922] systemd[1]: Reloading... lb # [ 3.168769] systemd[1]: Mounting /sysroot... worker1 # [ 3.181989] EXT4-fs (vda): mounted filesystem eb5679b0-4df0-438c-9004-2a9d3fefaa74 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.168965] systemd[1]: Mounted /sysroot. worker1 # [ 3.172092] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.227684] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.180136] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.237765] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.240131] systemd[1]: Mounting /sysroot... lb # [ 3.230090] EXT4-fs (vda): mounted filesystem e86ec5e8-8bca-4b52-9b9c-8ca6e786976c r/w with ordered data mode. Quota mode: none. worker1 # [ 3.203967] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.218991] systemd[1]: Mounted /sysroot. lb # [ 3.221099] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.211152] systemd[1]: Reload requested from client PID 117 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.228244] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.216537] systemd[1]: Reloading... worker2 # [ 3.292539] EXT4-fs (vda): mounted filesystem 3184b9ef-249a-49ba-8cf1-b67ca6fecee2 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.284977] systemd[1]: Mounted /sysroot. worker2 # [ 3.287104] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.292126] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.259880] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.266227] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.270432] systemd[1]: Reloading... worker2 # [ 3.320438] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.330262] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.331902] systemd[1]: Reloading... client # [ 3.442709] systemd[1]: Reloading finished in 215 ms. client # [ 3.475940] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.480198] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.487170] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.490159] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.491805] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.428991] systemd[1]: Reloading finished in 215 ms. worker1 # [ 3.459073] systemd-sysroot-fstab-check[117]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.461301] systemd-sysroot-fstab-check[117]: Requesting swap.target/start/replace... lb # [ 3.478878] systemd[1]: Reloading finished in 213 ms. worker1 # [ 3.468567] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.470943] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.473145] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.538322] systemd[1]: Reloading finished in 208 ms. lb # [ 3.515108] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.520248] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.523734] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.526556] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.528717] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.571676] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.575978] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.583222] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.587530] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.590148] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.821266] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.833736] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.838070] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.769719] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.858781] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.782910] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.795772] systemd[1]: Mounting /sysroot/run... client # [ 3.885757] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.814312] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.873770] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.838949] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.886782] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.890899] systemd[1]: Mounting /sysroot/run... lb # [ 3.855433] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.859402] systemd[1]: Mounting /sysroot/run... client # [ 3.952945] fuse: init (API version 7.45) worker2 # [ 3.911151] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.875029] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.883466] fuse: init (API version 7.45) client # [ 3.943968] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.869398] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.872076] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.949870] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.894511] virtiofs virtio6: discovered new tag: nix-store client # [ 3.971283] virtiofs virtio6: discovered new tag: nix-store client # [ 3.972123] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.903570] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 3.901347] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.986780] virtiofs virtio7: discovered new tag: shared client # [ 3.987531] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.947008] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.920491] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.921254] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.996562] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.908325] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.909278] systemd[1]: Mounted /sysroot/run. client # [ 4.003492] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 3.926381] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.992999] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.935266] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.936053] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.001067] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 4.007580] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.932632] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.009912] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.943708] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.001362] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 4.032209] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.033619] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.007782] systemd[1]: Mounted /sysroot/run. lb # [ 3.972153] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.027292] fuse: init (API version 7.45) worker1 # [ 3.965869] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.998881] fuse: init (API version 7.45) worker2 # [ 4.033635] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.985614] systemd[1]: Mounted /sysroot/run. worker2 # [ 4.034480] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.973784] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.975488] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.006167] virtiofs virtio6: discovered new tag: nix-store lb # [ 4.006980] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 4.049740] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.055628] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.026715] virtiofs virtio7: discovered new tag: shared lb # [ 4.027513] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 4.049948] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.066432] virtiofs virtio8: discovered new tag: xchg lb # [ 4.032482] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.068805] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.038552] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.030954] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.069178] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.033642] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.073033] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.038461] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.075490] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.078181] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.042989] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.043824] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.081791] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.378022] (udev-worker)[100]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.385060] (udev-worker)[100]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.419087] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.422000] systemd[1]: Stopped Virtual Console Setup. client # [ 4.424126] systemd[1]: Stopping Virtual Console Setup... client # [ 4.426924] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.367245] (udev-worker)[99]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.453148] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.456569] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.378896] (udev-worker)[99]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.408853] (udev-worker)[92]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.414970] (udev-worker)[92]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.404351] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.405451] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.408119] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.412626] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.418869] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.420468] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.426357] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.479375] (udev-worker)[94]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.486529] (udev-worker)[94]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.452496] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.455092] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.458348] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.459569] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.453201] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.456651] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.516201] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.517502] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.522122] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.522987] systemd[1]: Starting Virtual Console Setup... lb # [ 4.489059] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.492762] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.540748] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.542217] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.544484] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.565383] systemd-vconsole-setup[155]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.568633] systemd[1]: Finished Virtual Console Setup. client # [ 4.824438] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.771659] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.894693] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.897680] systemd[1]: Reached target Initrd File Systems. lb # [ 4.838225] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.874684] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.902476] systemd[1]: Starting Find NixOS closure... client # [ 4.912359] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.842666] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.845604] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.851930] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.860300] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.877420] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.878461] systemd[1]: Reached target Initrd File Systems. lb # [ 4.879875] systemd[1]: Starting Find NixOS closure... lb # [ 4.888237] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.951465] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.953378] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.967502] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.942835] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.970714] systemd[1]: Reached target Initrd Default Target. client # [ 4.972733] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.946765] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.949305] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.957299] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.921853] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.910468] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.912527] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.930093] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.920122] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.922081] systemd[1]: Reached target Initrd Default Target. lb # [ 4.937108] systemd[1]: Finished Find NixOS closure. client # [ 5.000904] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.925588] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.939390] systemd[1]: Reached target Initrd Default Target. lb # [ 4.940962] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 5.004470] systemd[1]: Stopped target Basic System. client # [ 5.005526] systemd[1]: Stopped target Initrd Root Device. client # [ 5.008232] systemd[1]: Stopped target Path Units. client # [ 5.009993] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.012442] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.016279] systemd[1]: Stopped target Slice Units. client # [ 5.017396] systemd[1]: Stopped target Socket Units. client # [ 5.020971] systemd[1]: Stopped target System Initialization. client # [ 5.024413] systemd[1]: Stopped target Swaps. client # [ 5.026051] systemd[1]: Stopped target Timer Units. lb # [ 4.963378] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.965367] systemd[1]: Stopped target Basic System. client # [ 5.028971] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.966399] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.967524] systemd[1]: Stopped target Path Units. client # [ 5.032666] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 5.006765] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.971708] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.959068] systemd[1]: Stopped target Initrd Default Target. client # [ 5.036159] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.972978] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.961252] systemd[1]: Stopped target Basic System. lb # [ 4.974757] systemd[1]: Stopped target Slice Units. lb # [ 4.976544] systemd[1]: Stopped target Socket Units. worker2 # [ 5.011193] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.964357] systemd[1]: Stopped target Initrd Root Device. client # [ 5.041526] systemd[1]: Stopped Find NixOS closure. lb # [ 4.978513] systemd[1]: Stopped target System Initialization. worker1 # [ 4.966062] systemd[1]: Stopped target Path Units. client # [ 5.043068] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.043962] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.969765] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.983963] systemd[1]: Stopped target Swaps. lb # [ 4.984785] systemd[1]: Stopped target Timer Units. worker1 # [ 4.971842] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.022582] systemd[1]: Finished Find NixOS closure. lb # [ 4.988367] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.976250] systemd[1]: Stopped target Slice Units. worker2 # [ 5.026278] systemd[1]: Reached target Initrd Default Target. lb # [ 4.990345] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.977466] systemd[1]: Stopped target Socket Units. client # [ 5.054707] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.028185] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 5.055592] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 4.980597] systemd[1]: Stopped target System Initialization. lb # [ 4.996589] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.060548] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 4.985095] systemd[1]: Stopped target Swaps. client # [ 5.061346] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.000270] systemd[1]: Stopped Find NixOS closure. client # [ 5.063238] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 4.987229] systemd[1]: Stopped target Timer Units. lb # [ 5.001041] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 5.001861] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.990497] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.003738] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.068566] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.069684] systemd[1]: Stopped Create System Files and Directories. client # [ 5.070546] systemd[1]: Stopped target Local File Systems. client # [ 5.071332] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 4.996632] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.074157] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.011622] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.075251] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.076193] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.000306] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.077189] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.078167] systemd[1]: Stopped Virtual Console Setup. client # [ 5.078873] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.016412] systemd[1]: Stopped Load Kernel Modules. client # [ 5.079776] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.003520] systemd[1]: Stopped Find NixOS closure. lb # [ 5.018315] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.005610] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.084273] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.057503] systemd[1]: Stopped target Initrd Default Target. lb # [ 5.022523] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.009594] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.059670] systemd[1]: Stopped target Basic System. lb # [ 5.023662] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.088159] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.089454] systemd[1]: systemd-udevd.service: Consumed 1.397s CPU time over 3.105s wall clock time, 22.4M memory peak. worker2 # [ 5.064109] systemd[1]: Stopped target Initrd Root Device. lb # [ 5.028506] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.065572] systemd[1]: Stopped target Path Units. client # [ 5.092245] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.029423] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.016292] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.093248] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.030172] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.066515] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.017885] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.031093] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.096139] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.069096] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.097156] systemd[1]: Closed udev Control Socket. lb # [ 5.034163] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.035051] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.072231] systemd[1]: Stopped target Slice Units. worker1 # [ 5.023052] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.036232] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.100135] systemd[1]: Starting Cleanup udev Database... lb # [ 5.037242] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.074100] systemd[1]: Stopped target Socket Units. client # [ 5.100912] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.037971] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.101938] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.038885] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.025651] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.039776] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.076142] systemd[1]: Stopped target System Initialization. client # [ 5.104270] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.028381] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.080324] systemd[1]: Stopped target Swaps. client # [ 5.108280] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.032266] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.109289] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.033377] systemd[1]: Stopped Create System Files and Directories. client # [ 5.110237] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.034740] systemd[1]: Stopped target Local File Systems. lb # [ 5.048395] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.085133] systemd[1]: Stopped target Timer Units. worker1 # [ 5.036249] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.049447] systemd[1]: systemd-udevd.service: Consumed 1.415s CPU time over 3.049s wall clock time, 25.1M memory peak. worker2 # [ 5.086943] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.050977] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.038956] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.052295] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.040423] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.089955] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.041237] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.042239] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.043249] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.056265] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.057299] systemd[1]: Closed udev Control Socket. worker1 # [ 5.043982] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.058006] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.045082] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.094235] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.045969] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.046860] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.060112] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.047859] systemd[1]: systemd-udevd.service: Consumed 1.427s CPU time over 3.081s wall clock time, 22.4M memory peak. lb # [ 5.064398] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.102009] systemd[1]: Stopped Find NixOS closure. lb # [ 5.065303] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.052316] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.066409] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.103590] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.054897] systemd[1]: Closed udev Control Socket. lb # [ 5.068240] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.056109] systemd[1]: Starting Cleanup udev Database... client # [ 5.133240] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.108363] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.072381] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.135931] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.109579] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.060166] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.137219] systemd[1]: Reached target Switch Root. worker2 # [ 5.110370] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.061272] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.111396] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.062129] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.140127] systemd[1]: Starting NixOS Activation... worker1 # [ 5.064188] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.068125] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.069119] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.070005] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.072161] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.122014] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.124326] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.127098] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.128665] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.093786] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.132525] systemd[1]: Stopped target Local File Systems. lb # [ 5.096514] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.133429] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.136194] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.100632] systemd[1]: Reached target Switch Root. worker2 # [ 5.137282] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.101419] systemd[1]: Starting NixOS Activation... worker2 # [ 5.138467] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.139604] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.140816] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.141542] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 5.142660] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.143564] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.097368] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.098906] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.148294] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.101021] systemd[1]: Reached target Switch Root. worker1 # [ 5.103280] systemd[1]: Starting NixOS Activation... worker2 # [ 5.152225] systemd[1]: systemd-udevd.service: Consumed 1.446s CPU time over 3.141s wall clock time, 22.5M memory peak. worker2 # [ 5.154115] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.156230] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.160178] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.161244] systemd[1]: Closed udev Control Socket. worker2 # [ 5.161934] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.164124] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.165176] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.168295] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.169433] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.172214] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.173202] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.220803] initrd-nixos-activation-start[177]: booting system configuration /nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test worker2 # [ 5.196506] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.199378] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.200684] systemd[1]: Reached target Switch Root. worker2 # [ 5.201798] systemd[1]: Starting NixOS Activation... lb # [ 5.175663] initrd-nixos-activation-start[178]: booting system configuration /nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test client # [ 5.253675] initrd-nixos-activation-start[177]: running activation script... worker1 # [ 5.187454] initrd-nixos-activation-start[176]: booting system configuration /nix/store/wrqfs8d5q8wxr8pbh005xzrx73np2k9s-nixos-system-worker1-test lb # [ 5.207475] initrd-nixos-activation-start[178]: running activation script... worker1 # [ 5.219752] initrd-nixos-activation-start[176]: running activation script... worker2 # [ 5.283626] initrd-nixos-activation-start[178]: booting system configuration /nix/store/7nr90ian7b661w5sws26k1lwc9mbmx4q-nixos-system-worker2-test worker2 # [ 5.316978] initrd-nixos-activation-start[178]: running activation script... client # [ 5.485913] initrd-nixos-activation-start[200]: setting up /etc... lb # [ 5.445008] initrd-nixos-activation-start[201]: setting up /etc... worker1 # [ 5.462028] initrd-nixos-activation-start[199]: setting up /etc... worker2 # [ 5.575600] initrd-nixos-activation-start[201]: setting up /etc... client # [ 5.605151] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.607896] systemd[1]: Finished NixOS Activation. client # [ 5.609240] systemd[1]: Starting Switch Root... lb # [ 5.562429] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.565211] systemd[1]: Finished NixOS Activation. lb # [ 5.566392] systemd[1]: Starting Switch Root... client # [ 5.631049] systemd[1]: Switching root. lb # [ 5.586936] systemd[1]: Switching root. worker1 # [ 5.581443] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.584331] systemd[1]: Finished NixOS Activation. worker1 # [ 5.588317] systemd[1]: Starting Switch Root... worker1 # [ 5.607390] systemd[1]: Switching root. worker2 # [ 5.700701] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.703538] systemd[1]: Finished NixOS Activation. worker2 # [ 5.704856] systemd[1]: Starting Switch Root... worker2 # [ 5.727374] systemd[1]: Switching root. client # [ 5.815555] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.774408] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.800684] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.908489] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 6.305304] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.317855] systemd[1]: Detected virtualization qemu. lb # [ 6.321272] systemd[1]: Detected architecture arm64. client # [ 6.377541] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.324292] systemd[1]: Detected first boot. client # [ 6.390469] systemd[1]: Detected virtualization qemu. lb # [ 6.330667] systemd[1]: Initializing machine ID from random generator. client # [ 6.393550] systemd[1]: Detected architecture arm64. client # [ 6.397404] systemd[1]: Detected first boot. client # [ 6.403196] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.332552] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.343667] systemd[1]: Detected virtualization qemu. worker1 # [ 6.345661] systemd[1]: Detected architecture arm64. worker1 # [ 6.348147] systemd[1]: Detected first boot. worker1 # [ 6.351311] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.442735] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.455349] systemd[1]: Detected virtualization qemu. worker2 # [ 6.459337] systemd[1]: Detected architecture arm64. worker2 # [ 6.461668] systemd[1]: Detected first boot. worker2 # [ 6.466739] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.645205] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.642641] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.730443] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.688089] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.813801] systemd[1]: Applying preset policy. lb # [ 6.826051] systemd[1]: Applying preset policy. client # [ 6.928816] systemd[1]: Applying preset policy. worker1 # [ 6.891475] systemd[1]: Applying preset policy. worker2 # [ 7.051066] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.077210] systemd[1]: Populated /etc with preset unit settings. client # [ 7.171285] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.176672] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.282290] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.283740] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.287182] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.289973] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.292199] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.293462] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.295858] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.297492] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.299093] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.301446] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.302636] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.304124] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.305709] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.308388] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.309595] systemd[1]: Reached target Path Units. worker2 # [ 7.311157] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.313524] systemd[1]: Reached target Slice Units. worker2 # [ 7.314482] systemd[1]: Reached target Swaps. worker2 # [ 7.317428] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.320586] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.322872] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.325337] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.327353] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.331896] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.334346] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.337521] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.339969] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.341926] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.310736] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.344522] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.312033] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.349574] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.315398] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.351159] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.352434] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.318708] systemd[1]: Created slice Slice /system/getty. lb # [ 7.320598] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.356272] systemd[1]: Mounting Huge Pages File System... lb # [ 7.322134] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.323923] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.325961] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.328240] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.363022] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.329347] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.394802] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.331798] systemd[1]: Stopped target initrd-fs.target. client # [ 7.396124] systemd[1]: Stopped initrd-switch-root.service. lb # [ 7.332827] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.334463] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.399469] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 7.336956] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.338153] systemd[1]: Reached target Path Units. client # [ 7.402694] systemd[1]: Created slice Slice /system/getty. lb # [ 7.339628] systemd[1]: Reached target Remote File Systems. client # [ 7.404634] systemd[1]: Created slice User and Session Slice. lb # [ 7.341302] systemd[1]: Reached target Slice Units. client # [ 7.406189] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.343632] systemd[1]: Reached target Swaps. worker2 # [ 7.374953] systemd[1]: Mounting Kernel Debug File System... client # [ 7.407952] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.346115] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.409831] systemd[1]: Expecting device /dev/hvc0... client # [ 7.412085] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 7.349130] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.413176] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.351345] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.414763] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.386158] systemd[1]: Mounting Kernel Trace File System... client # [ 7.416277] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.353733] systemd[1]: Listening on Factory Reset Management. lb # [ 7.354977] systemd[1]: Listening on Hostname Service Socket. client # [ 7.417977] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.420364] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.421625] systemd[1]: Reached target Path Units. lb # [ 7.359271] systemd[1]: Starting Journal Log Access Socket... client # [ 7.423834] systemd[1]: Reached target Remote File Systems. lb # [ 7.361561] systemd[1]: Listening on Journal Audit Socket. client # [ 7.424888] systemd[1]: Reached target Slice Units. client # [ 7.426381] systemd[1]: Reached target Swaps. lb # [ 7.364311] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.429314] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.365960] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.402279] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.432245] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.367614] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.434550] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.370750] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.406864] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.436944] systemd[1]: Listening on Factory Reset Management. client # [ 7.438185] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.377683] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.379050] systemd[1]: Listening on udev Control Socket. client # [ 7.442397] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.380723] systemd[1]: Listening on udev Varlink Socket. client # [ 7.445540] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.385239] systemd[1]: Mounting Huge Pages File System... client # [ 7.449543] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.451759] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.423053] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.453786] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.424560] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.455378] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.393923] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.460569] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.430844] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.461988] systemd[1]: Listening on udev Control Socket. client # [ 7.463609] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.436239] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.467233] systemd[1]: Mounting Huge Pages File System... lb # [ 7.408106] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.398575] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.474167] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.400321] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.404947] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.409502] systemd[1]: Created slice Slice /system/getty. lb # [ 7.420359] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.411704] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.412949] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.490459] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.415974] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.463145] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.418420] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.420414] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 7.432840] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.495236] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.422398] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.467459] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.424546] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.435134] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.425568] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.427280] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.429763] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.430997] systemd[1]: Reached target Path Units. worker1 # [ 7.433292] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.434340] systemd[1]: Reached target Slice Units. worker1 # [ 7.435963] systemd[1]: Reached target Swaps. worker1 # [ 7.439763] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.513795] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.442714] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.516365] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.453887] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.444979] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.456205] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.447336] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.448522] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.452749] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.463981] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.454856] systemd[1]: Listening on Journal Audit Socket. client # [ 7.530216] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.467893] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.458484] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.504047] systemd[1]: Starting Journal Service... worker1 # [ 7.460147] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.535030] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.462293] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.464009] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.470213] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.471653] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.473317] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.519078] systemd[1]: Starting Load Kernel Modules... client # [ 7.548383] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.550527] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.477144] systemd[1]: Mounting Huge Pages File System... lb # [ 7.493649] systemd[1]: Mounting FUSE Control File System... lb # [ 7.494993] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.484713] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.531714] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.491069] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.540316] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.571030] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.542461] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.575702] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.504148] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.520033] systemd[1]: Starting Journal Service... worker1 # [ 7.513594] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.561009] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.519070] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.565573] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.570413] systemd[1]: Mounted Huge Pages File System. lb # [ 7.536316] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.574045] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.603491] systemd[1]: Starting Journal Service... worker1 # [ 7.532629] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.577941] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.581055] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.583839] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.587032] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.538957] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.539445] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.539738] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.558952] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.623739] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.595044] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.556405] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.603373] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.558183] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.577342] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.642209] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.581137] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.592559] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.659979] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.661480] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.588578] systemd[1]: Starting Journal Service... lb # [ 7.604374] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.607959] systemd[1]: Mounted Huge Pages File System. client # [ 7.673518] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.614902] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.680004] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.618590] systemd[1]: Mounted Kernel Debug File System. lb # [ 7.621563] systemd-journald[271]: Collecting audit messages is enabled. client # [ 7.693980] systemd[1]: Mounted Huge Pages File System. lb # [ 7.633181] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.624833] systemd[1]: Starting Load Kernel Modules... client # [ 7.701784] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.675809] systemd[1]: Finished Load Kernel Modules. client # [ 7.707965] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.633363] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.646298] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.647736] systemd[1]: Started Journal Service. lb # [ 7.629149] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.681811] systemd[1]: Starting Firewall... client # [ 7.714257] systemd-journald[270]: Collecting audit messages is enabled. worker1 # [ 7.640200] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.643025] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.691730] systemd[1]: Starting Apply Kernel Variables... client # [ 7.721500] systemd[1]: Mounted Kernel Trace File System. client # [ 7.726414] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.654196] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.727020] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.699102] systemd-journald[271]: Collecting audit messages is enabled. worker1 # [ 7.660445] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.736981] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.663308] systemd[1]: Mounted Huge Pages File System. client # [ 7.742798] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.713380] EXT4-fs (vda): re-mounted 3184b9ef-249a-49ba-8cf1-b67ca6fecee2. worker1 # [ 7.666175] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.666649] systemd[1]: Mounted Kernel Debug File System. client # [ 7.730228] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.673269] systemd[1]: Mounted Kernel Trace File System. client # [ 7.749812] systemd[1]: Started Journal Service. worker1 # [ 7.675634] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.677887] systemd[1]: Mounted FUSE Control File System. client # [ 7.740918] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.727756] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.742151] systemd-modules-load[271]: Module 'atkbd' is built in client # [ 7.743120] systemd-modules-load[271]: Module 'loop' is built in worker2 # [ 7.730443] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.732762] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.755031] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.743871] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.745364] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.712324] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.775202] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.719659] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.733185] EXT4-fs (vda): re-mounted e86ec5e8-8bca-4b52-9b9c-8ca6e786976c. worker2 # [ 7.771806] systemd[1]: Started Journal Service. worker2 # [ 7.764623] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.765836] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.774073] systemd-modules-load[272]: Module 'atkbd' is built in worker2 # [ 7.793366] systemd-modules-load[272]: Module 'loop' is built in client # [ 7.822128] systemd-oomd[272]: No swap; memory pressure usage will be degraded worker2 # [ 7.799570] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.854443] EXT4-fs (vda): re-mounted 90dd4bde-bcaa-4091-be50-17c845d35dc4. client # [ 7.842354] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.788793] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.793200] EXT4-fs (vda): re-mounted eb5679b0-4df0-438c-9004-2a9d3fefaa74. client # [ 7.854658] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.855719] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.803880] systemd[1]: Starting Firewall... worker1 # [ 7.806548] systemd-journald[269]: Collecting audit messages is enabled. client # [ 7.866287] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.867312] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.815615] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 7.819414] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.880264] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 7.822204] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.881107] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.825715] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.865361] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.835947] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 7.838535] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.880500] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.885431] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.860796] systemd[1]: Started Journal Service. worker1 # [ 7.849656] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.891761] systemd-oomd[273]: No swap; memory pressure usage will be degraded worker1 # [ 7.850853] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.903554] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.908966] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.952530] systemd-journald[270]: Received client request to flush runtime journal. worker2 # [ 7.909849] systemd[1]: Reached target First Boot Complete. worker1 # [ 7.866083] systemd-modules-load[270]: Module 'atkbd' is built in worker1 # [ 7.876309] systemd-modules-load[270]: Module 'loop' is built in worker1 # [ 7.886087] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 7.955261] systemd-journald[271]: Received client request to flush runtime journal. client # [ 7.993203] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.994162] systemd[1]: Reached target First Boot Complete. client # [ 7.994952] systemd[1]: Finished Apply Kernel Variables. client # [ 8.005012] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.953719] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker2 # [ 8.009133] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.965260] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.972340] systemd[1]: Reached target First Boot Complete. worker1 # [ 7.980457] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.004379] systemd-journald[269]: Received client request to flush runtime journal. worker2 # [ 8.066329] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.072580] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.076610] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.049198] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 8.056315] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.057440] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.060476] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 8.154242] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.156288] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.183969] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker1 # [ 8.211223] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.220617] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.271284] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.221913] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.633807] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.274389] systemd-journald[271]: Received client request to flush runtime journal. worker2 # [ 8.295519] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.334508] systemd-modules-load[272]: Module 'atkbd' is built in worker1 # [ 8.329698] systemd-udevd[310]: Using default interface naming scheme 'v261'. lb # [ 8.343204] systemd-modules-load[272]: Module 'loop' is built in lb # [ 8.348566] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 8.349883] systemd[1]: Finished Load Kernel Modules. lb # [ 8.350740] systemd[1]: Starting Firewall... lb # [ 8.351507] systemd[1]: Starting Apply Kernel Variables... lb # [ 8.365604] systemd[1]: Mounted Kernel Configuration File System. lb # [ 8.366643] systemd[1]: Mounted FUSE Control File System. lb # [ 8.367833] systemd-oomd[273]: No swap; memory pressure usage will be degraded lb # [ 8.380809] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 8.384214] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 8.424381] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.425738] systemd[1]: Reached target Local File Systems. lb # [ 8.392700] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 8.428609] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 8.395630] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.383322] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.437063] systemd[1]: Starting register-nix-paths.service... lb # [ 8.406042] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.409115] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.452367] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.416116] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 8.417253] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.418111] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.419142] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.460944] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.467538] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.432394] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.433229] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.469919] systemd[1]: Starting Create System Files and Directories... lb # [ 8.434021] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.434908] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.435870] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 8.448228] systemd[1]: Mounting /run/wrappers... lb # [ 8.448948] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.449867] systemd[1]: Mounted /run/wrappers. lb # [ 8.450517] systemd[1]: Reached target Local File Systems. lb # [ 8.451291] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.528544] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 8.457522] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.458322] systemd[1]: Reached target Local File Systems. client # [ 8.535675] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.537890] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.476567] systemd[1]: Starting register-nix-paths.service... lb # [ 8.477472] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.541523] systemd[1]: Mounting /run/wrappers... lb # [ 8.478430] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.479778] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.468279] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.544720] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.469314] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.476371] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.482647] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.497433] systemd[1]: Starting Create System Files and Directories... lb # [ 8.498710] systemd-udevd[295]: Using default interface naming scheme 'v261'. worker1 # [ 8.500111] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.501167] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.502066] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.587009] systemd[1]: Mounted /run/wrappers. client # [ 8.588382] systemd[1]: Reached target Local File Systems. client # [ 8.592180] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.594702] systemd[1]: Starting register-nix-paths.service... client # [ 8.600110] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.601004] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.605784] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.618566] systemd[1]: Starting Create System Files and Directories... client # [ 8.631706] systemd-udevd[301]: Using default interface naming scheme 'v261'. worker2 # [ 8.606646] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.619077] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.589816] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.602075] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.616838] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.623441] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.667752] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.681042] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.716473] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.693046] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.726042] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.689041] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.760153] systemd[1]: Finished Create System Files and Directories. lb # [ 8.703632] systemd[1]: Finished Create System Files and Directories. client # [ 8.768232] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.774881] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 8.714483] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.786027] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.729711] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 8.722850] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.732420] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.740364] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.811594] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.865011] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.894041] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.868193] systemd[1]: Starting Update is Completed... client # [ 8.911972] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.918220] systemd[1]: Starting Update is Completed... lb # [ 8.865601] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.856130] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.946077] systemd[1]: Finished Update is Completed. client # [ 8.983505] systemd[1]: Finished Update is Completed. lb # [ 8.936336] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.931248] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 8.948462] systemd[1]: Starting Update is Completed... worker1 # [ 8.940996] systemd[1]: Starting Update is Completed... worker1 # [ 9.024637] systemd[1]: Finished Update is Completed. lb # [ 9.057757] systemd[1]: Finished Update is Completed. client # [ 9.190604] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.230432] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.268548] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 9.384332] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.388303] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.434643] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 9.409585] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.476160] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.479244] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.510011] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.455103] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 9.446777] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.456456] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 9.675133] systemd[1]: Finished Firewall. client # [ 9.710340] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.717735] (udev-worker)[350]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 9.723274] (udev-worker)[350]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.670623] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 9.706234] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.711089] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.714527] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 9.778537] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 9.746278] systemd[1]: Finished Firewall. lb # [ 9.767553] systemd[1]: Finished register-nix-paths.service. lb # [ 9.771262] systemd[1]: Reached target System Initialization. lb # [ 9.776929] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.777966] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.778823] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 9.779766] systemd[1]: Reached target Timer Units. lb # [ 9.785506] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 9.791083] systemd[1]: Listening on niks3 server socket. lb # [ 9.792991] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.793852] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.795036] systemd[1]: Reached target Socket Units. lb # [ 9.795764] systemd[1]: Reached target Basic System. lb # [ 9.805339] systemd[1]: Started backdoor.service. lb # [ 9.806064] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 9.846996] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.813145] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.820145] systemd[1]: Starting Post-Boot Actions... worker2 # [ 9.878541] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.867659] systemd[1]: Started Reset console on configuration changes. worker1 # [ 9.883008] systemd[1]: Finished Coldplug All udev Devices. client # [ 9.963678] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 9.966509] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 9.906257] systemd[1]: Starting resolvconf update... client # [ 9.971500] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 9.973620] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.974871] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 9.920362] systemd[1]: Started rustfs.service. client # [ 9.984746] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 9.986362] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 9.987650] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 9.998218] systemd[1]: Finished register-nix-paths.service. lb # [ 9.940166] systemd[1]: Starting rustfs-bucket.service... client # [ 10.028969] mousedev: PS/2 mouse device common for all mice worker1 # [ 9.954463] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 9.976693] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # connecting to host... worker2 # [ 10.047289] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 10.017843] systemd[1]: Finished Post-Boot Actions. lb # [ 10.023233] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.031013] nsncd[450]: Sep 22 14:51:14.632 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.092439] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.099731] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.101575] systemd[1]: Reached target System Initialization. worker2 # [ 10.103669] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.069026] (udev-worker)[401]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.106634] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.112529] systemd[1]: Reached target Timer Units. lb # [ 10.078514] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.079544] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.120231] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.127203] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.094380] (udev-worker)[404]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 10.140629] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 10.109090] (udev-worker)[404]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.110286] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.149381] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.160370] systemd[1]: Reached target Socket Units. lb # [ 10.124206] systemd[1]: Starting User Login Management... worker2 # [ 10.168649] systemd[1]: Reached target Basic System. worker2 # [ 10.177468] systemd[1]: Started backdoor.service. worker2 # [ 10.186559] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.153677] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.145961] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.201086] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.159115] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.163084] systemd[1]: Reached target System Initialization. worker2 # [ 10.212827] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.165803] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.167840] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.221411] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.173262] systemd[1]: Reached target Timer Units. worker1 # [ 10.177880] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.233376] systemd[1]: Starting resolvconf update... worker1 # [ 10.186870] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.187793] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 10.242231] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.198311] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.215101] systemd[1]: Reached target Socket Units. worker1 # [ 10.219003] systemd[1]: Reached target Basic System. worker1 # [ 10.224072] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.279997] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.234155] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.244129] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.253475] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.308570] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.261328] systemd[1]: Starting resolvconf update... worker2 # [ 10.315967] nsncd[490]: Sep 22 14:51:14.902 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.271492] systemd[1]: Starting D-Bus System Message Bus... worker2 # connecting to host... worker2 # [ 10.334555] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.345482] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.346450] systemd[1]: Starting User Login Management... worker1 # [ 10.313489] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.374869] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.394662] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.350113] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.404328] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.356354] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.411976] dbus-broker-launch[494]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.363451] nsncd[488]: Sep 22 14:51:14.971 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.425622] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.381616] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.388635] systemd[1]: Starting User Login Management... client # [ 10.469401] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 10.472202] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 10.398884] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 10.476765] systemd[1]: Reached target System Initialization. lb # [ 10.413799] dbus-broker-launch[481]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.478812] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.404058] systemd[1]: Started backdoor.service. client # [ 10.482707] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.485363] systemd[1]: Reached target Timer Units. worker1 # [ 10.409713] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 10.424626] dbus-broker-launch[481]: NSS returned no entry for 'systemd-timesync' client # [ 10.488171] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.425776] dbus-broker-launch[481]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 10.492711] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.494137] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.422343] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.499778] systemd[1]: Reached target Socket Units. client # [ 10.500757] systemd[1]: Reached target Basic System. worker2 # [ 10.474214] dbus-broker-launch[494]: Ready client # [ 10.502608] systemd[1]: Started backdoor.service. client # [ 10.505233] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.508866] systemd[1]: Starting nftables firewall... client # [ 10.518149] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.439848] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.450116] dbus-broker-launch[492]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 10.470768] systemd-logind[483]: New seat seat0. lb # [ 10.479640] systemd[1]: Started User Login Management. worker1 # [ 10.468534] systemd[1]: Started D-Bus System Message Bus. client # [ 10.550686] systemd[1]: Starting Post-Boot Actions... lb # [ 10.488099] systemd[1]: Starting linger-users.service... worker2 # [ 10.525601] systemd-logind[513]: New seat seat0. lb # [ 10.496642] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.532985] systemd[1]: Started User Login Management. worker2 # [ 10.536765] systemd[1]: Starting linger-users.service... client # [ 10.564317] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.554702] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.560822] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 10.561747] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.562590] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.563405] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.519828] dbus-broker-launch[492]: Ready client # [ 10.599525] systemd[1]: Starting resolvconf update... lb # [ 10.537890] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.538865] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.539726] systemd[1]: Stopped target User and Group Name Lookups. worker1 # connecting to host... worker2 # [ 10.585683] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.586576] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.554715] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.596377] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.561459] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 10.566827] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.571114] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # connecting to host... lb # [ 10.580679] dbus-broker-launch[481]: Ready worker2 # [ 10.618152] (udev-worker)[447]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.626263] (udev-worker)[340]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.589587] systemd-logind[500]: New seat seat0. worker2 # [ 10.639732] (udev-worker)[340]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.668481] nsncd[453]: Sep 22 14:51:15.277 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.608668] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.645188] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.646151] systemd[1]: Finished linger-users.service. worker1 # [ 10.598601] systemd[1]: Started User Login Management. worker1 # [ 10.599443] systemd[1]: Starting linger-users.service... client # [ 10.678832] systemd[1]: Started Name Service Cache Daemon (nsncd). client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 10.672343] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.673261] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.629989] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.716919] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.634453] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.657931] systemd[1]: linger-users.service: Deactivated successfully. client # [ 10.722887] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.674069] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.639427] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.665073] systemd[1]: Finished linger-users.service. client # [ 10.740097] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.685026] nsncd[562]: Sep 22 14:51:15.275 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.645687] systemd[1]: Stopping User and Group Name Lookups... lb # [ 10.723118] systemd[1]: Started Name Service Cache Daemon (nsncd). client: connected to guest root shell client # [ 10.773636] systemd[1]: Starting User Login Management... worker2 # [ 10.698009] systemd[1]: Finished resolvconf update. worker1 # [ 10.646476] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 10.730059] systemd[1]: Reached target Host and Network Name Lookups. client: (connecting took 11.18 seconds) worker1 # [ 10.647300] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.735415] nsncd[562]: Sep 22 14:51:15.332 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client: (finished: waiting for the VM to finish booting, in 11.18 seconds) worker1 # [ 10.661274] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.743009] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.777820] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.698738] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.662134] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.767875] systemd[1]: Finished resolvconf update. client # [ 10.782068] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.707489] systemd[1]: Starting DHCP Client... worker1 # [ 10.693217] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.712319] systemd[1]: Starting Extra networking commands.... worker1 # [ 10.694166] systemd[1]: Finished linger-users.service. lb # [ 10.777989] systemd[1]: Starting DHCP Client... worker1 # [ 10.700601] (udev-worker)[341]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.710425] (udev-worker)[454]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.719026] (udev-worker)[454]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.746032] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.749993] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.751251] nsncd[562]: Sep 22 14:51:15.363 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.760168] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.768384] systemd[1]: Finished resolvconf update. worker1 # [ 10.769091] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.776993] systemd[1]: Starting DHCP Client... worker1 # [ 10.780529] systemd[1]: Starting Extra networking commands.... lb # [ 10.796403] systemd-logind[483]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 10.889015] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.941238] dhcpcd[597]: dhcpcd-10.3.2 starting worker2 # [ 10.951253] dhcpcd[639]: dev: loaded udev client # [ 10.998102] dbus-broker-launch[479]: Looking up NSS user entry for 'systemd-timesync'... client # [ 11.010110] systemd-logind[481]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.000774] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.013744] systemd-logind[481]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.014825] systemd-logind[481]: New seat seat0. client # [ 11.027521] systemd[1]: Started User Login Management. client # [ 11.034919] dbus-broker-launch[479]: NSS returned no entry for 'systemd-timesync' client # [ 11.035936] dbus-broker-launch[479]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 11.056484] systemd[1]: Starting linger-users.service... worker2 # [ 11.038910] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.039834] systemd[1]: Reached target Network. client # [ 11.079754] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 11.003872] dhcpcd[596]: dhcpcd-10.3.2 starting client # [ 11.084782] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.085655] systemd[1]: Stopped target User and Group Name Lookups. client # [ 11.086480] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 11.060159] systemd[1]: Starting Permit User Sessions... client # [ 11.087281] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 11.012261] dhcpcd[635]: dev: loaded udev client # [ 11.106582] systemd[1]: Started D-Bus System Message Bus. client # [ 11.107407] systemd[1]: nscd.service: Deactivated successfully. lb # [ 11.051452] dhcpcd[613]: dhcpcd-10.3.2 starting worker2 # [ 11.091049] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.108283] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.066034] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.123890] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.101282] systemd[1]: Starting Address configuration of eth1... lb # [ 11.071056] dhcpcd[627]: dev: loaded udev client # [ 11.149218] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 11.157214] dbus-broker-launch[479]: Ready worker2 # [ 11.144967] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.145495] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.148278] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.148613] cfg80211: failed to load regulatory.db client # [ 11.168099] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.177192] systemd[1]: Finished linger-users.service. worker1 # [ 11.104468] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.157076] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.112331] systemd[1]: Reached target Network. worker1 # [ 11.113115] systemd[1]: Starting Permit User Sessions... worker2 # [ 11.163973] systemd[1]: Started Getty on tty1. worker2 # [ 11.167144] systemd[1]: Reached target Login Prompts. lb # [ 11.160528] mousedev: PS/2 mouse device common for all mice lb # [ 11.151069] systemd[1]: Finished Firewall. lb # [ 11.151791] systemd[1]: Reached target Preparation for Network. lb # [ 11.172474] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.159348] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.166615] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.199220] systemd-logind[513]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.167534] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.163218] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 11.242843] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.169048] systemd[1]: Starting Address configuration of eth1... client # [ 11.247464] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 11.239318] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.251014] nsncd[542]: Sep 22 14:51:15.855 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.258405] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.207177] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.207712] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.254297] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.243199] dhcpcd[639]: eth0: waiting for carrier worker1 # [ 11.211308] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.211663] cfg80211: failed to load regulatory.db worker2 # [ 11.249078] dhcpcd[639]: eth0: waiting for carrier client # [ 11.276160] systemd[1]: Finished resolvconf update. worker2 # [ 11.249866] dhcpcd[639]: eth0: carrier acquired worker2 # [ 11.254354] network-addresses-eth1-start[652]: adding address 192.168.1.4/24... done worker2 # [ 11.271144] mousedev: PS/2 mouse device common for all mice client # [ 11.286872] systemd[1]: Starting DHCP Client... worker2 # [ 11.269155] dhcpcd[639]: DUID 00:01:00:01:32:45:52:63:52:54:00:12:34:56 worker2 # [ 11.270148] dhcpcd[639]: eth0: IAID 00:12:34:56 worker2 # [ 11.270822] dhcpcd[639]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.219566] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.225021] systemd[1]: Started Getty on tty1. worker1 # [ 11.225710] systemd[1]: Reached target Login Prompts. worker2 # [ 11.276713] network-addresses-eth1-start[652]: adding address 2001:db8:1::4/64... done worker2 # [ 11.289536] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.259756] systemd-logind[500]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 11.299613] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.318533] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.302922] dhcpcd[635]: eth0: waiting for carrier worker1 # [ 11.307320] dhcpcd[635]: eth0: waiting for carrier worker1 # [ 11.310791] network-addresses-eth1-start[652]: adding address 192.168.1.3/24... done worker1 # [ 11.311908] dhcpcd[635]: eth0: carrier acquired worker1 # [ 11.323428] network-addresses-eth1-start[652]: adding address 2001:db8:1::3/64... done worker1 # [ 11.327854] dhcpcd[635]: DUID 00:01:00:01:32:45:52:63:52:54:00:12:34:56 worker1 # [ 11.331515] dhcpcd[635]: eth0: IAID 00:12:34:56 worker1 # [ 11.333267] dhcpcd[635]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.384294] dhcpcd[639]: eth0: soliciting a DHCP lease worker2 # [ 11.388509] dhcpcd[639]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 11.392330] dhcpcd[639]: eth0: probing address 10.0.2.15/24 worker1 # [ 11.346412] systemd[1]: Finished Address configuration of eth1. lb # [ 11.393997] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.411444] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.469825] dhcpcd[580]: dhcpcd-10.3.2 starting worker2 # [ 11.452549] systemd-logind[513]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 11.480359] dhcpcd[587]: dev: loaded udev lb # [ 11.422741] systemd-logind[483]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.441144] network-addresses-eth1-start[637]: adding address 192.168.1.2/24... done worker1 # [ 11.456407] mousedev: PS/2 mouse device common for all mice client # [ 11.533279] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.473338] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.473865] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.481936] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.482262] cfg80211: failed to load regulatory.db lb # [ 11.478181] network-addresses-eth1-start[637]: adding address 2001:db8:1::2/64... done worker1 # [ 11.477216] systemd-logind[500]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.515743] systemd[1]: Finished Address configuration of eth1. client # [ 11.647486] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.574462] dhcpcd[677]: /nix/store/1ki7bgdr0niilkg9jbkpcdaggbv0kb8a-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (641) - Operation not permitted lb # [ 11.580282] dhcpcd[677]: .resolvconf-wrapped: clearing stale lock pid 641 client # [ 11.696997] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.699736] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.702510] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.702834] cfg80211: failed to load regulatory.db lb # [ 11.666991] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.653488] dhcpcd[627]: libudev: received NULL device lb # [ 11.656845] dhcpcd[627]: eth0: waiting for carrier lb # [ 11.659823] dhcpcd[627]: libudev: received NULL device lb # [ 11.664453] dhcpcd[627]: eth0: carrier acquired client # [ 11.728358] systemd[1]: Finished nftables firewall. client # [ 11.732879] systemd[1]: Reached target Preparation for Network. lb # [ 11.674970] systemd[1]: Finished Extra networking commands.. client # [ 11.739540] systemd[1]: Starting Address configuration of eth1... lb # [ 11.679797] systemd[1]: Reached target Network. client # [ 11.748180] systemd[1]: Starting Extra networking commands.... lb # [ 11.687608] systemd[1]: Starting PostgreSQL Server... lb # [ 11.690663] dhcpcd[627]: DUID 00:01:00:01:32:45:52:64:52:54:00:12:34:56 lb # [ 11.691616] dhcpcd[627]: eth0: IAID 00:12:34:56 lb # [ 11.695382] dhcpcd[627]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.700863] systemd[1]: Starting Permit User Sessions... client # [ 11.813387] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.799475] dhcpcd[587]: eth0: waiting for carrier client # [ 11.802653] dhcpcd[587]: libudev: received NULL device client # [ 11.803455] dhcpcd[587]: libudev: received NULL device lb # [ 11.741108] dhcpcd[627]: eth0: soliciting a DHCP lease lb # [ 11.744707] dhcpcd[627]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 11.808811] dhcpcd[587]: eth0: carrier acquired lb # [ 11.752227] dhcpcd[627]: eth0: probing address 10.0.2.15/24 client # [ 11.822384] dhcpcd[587]: DUID 00:01:00:01:32:45:52:64:52:54:00:12:34:56 client # [ 11.823349] dhcpcd[587]: eth0: IAID 00:12:34:56 client # [ 11.828357] dhcpcd[587]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.873639] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.798287] systemd[1]: Finished Permit User Sessions. lb # [ 11.809473] systemd[1]: Started Getty on tty1. lb # [ 11.810213] systemd[1]: Reached target Login Prompts. client # [ 11.878639] network-addresses-eth1-start[607]: adding address 192.168.1.1/24... done client # [ 11.903564] network-addresses-eth1-start[607]: adding address 2001:db8:1::1/64... done worker1 # [ 11.853040] dhcpcd[635]: eth0: soliciting a DHCP lease client # [ 11.931464] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.860892] dhcpcd[635]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 11.868251] dhcpcd[635]: eth0: probing address 10.0.2.15/24 client # [ 12.012758] systemd[1]: Finished Extra networking commands.. client # [ 12.016986] systemd[1]: Reached target Network. client # [ 12.020499] systemd[1]: Starting Permit User Sessions... lb # [ 12.005314] postgresql-pre-start[731]: The files belonging to this database system will be owned by user "postgres". lb # [ 12.006794] postgresql-pre-start[731]: This user must also own the server process. lb # [ 12.022565] postgresql-pre-start[731]: The database cluster will be initialized with locale "en_US.UTF-8". client # [ 12.103504] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.025949] postgresql-pre-start[731]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.027230] postgresql-pre-start[731]: The default text search configuration will be set to "english". lb # [ 12.031366] postgresql-pre-start[731]: Data page checksums are enabled. client # [ 12.097469] systemd[1]: Finished Permit User Sessions. lb # [ 12.033646] postgresql-pre-start[731]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.034976] postgresql-pre-start[731]: creating subdirectories ... ok lb # [ 12.035823] postgresql-pre-start[731]: selecting dynamic shared memory implementation ... posix client # [ 12.104459] systemd[1]: Started Getty on tty1. client # [ 12.106150] systemd[1]: Reached target Login Prompts. worker2 # [ 12.093555] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.161281] postgresql-pre-start[731]: selecting default "max_connections" ... 100 worker1 # [ 12.223191] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: waiting for unit network-addresses-eth1.service, in 12.68 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 12.291001] postgresql-pre-start[731]: selecting default "shared_buffers" ... 128MB client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2 # [ 12.403180] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 12.419729] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.440627] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker2 # [ 12.447677] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.448982] systemd[1]: Starting Virtual Console Setup... client # [ 12.491130] systemd[1]: Starting Virtual Console Setup... client # [ 12.506456] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.510182] systemd[1]: Stopped Virtual Console Setup. client # [ 12.520107] systemd[1]: Starting Virtual Console Setup... lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2 # [ 12.514709] systemd-logind[513]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 12.544961] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.07 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } client # [ 12.595840] systemd-logind[481]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 12.602481] dhcpcd[587]: eth0: soliciting a DHCP lease client # [ 12.608489] dhcpcd[587]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.612155] dhcpcd[587]: eth0: probing address 10.0.2.15/24 worker2 # [ 12.592461] systemd-vconsole-setup[693]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 12.596276] systemd[1]: Finished Virtual Console Setup. worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.03 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker1 # [ 12.589809] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 12.604971] systemd[1]: Starting Virtual Console Setup... client # [ 12.683594] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service client # [ 12.687569] systemd[1]: Finished Virtual Console Setup. worker1 # [ 12.621851] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 12.627831] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.632940] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.686333] systemd-logind[500]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.769349] systemd-vconsole-setup[701]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 12.773686] systemd[1]: Finished Virtual Console Setup. lb # [ 12.834012] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 13.171888] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.181577] systemd[1]: Starting Virtual Console Setup... lb # [ 13.266651] systemd-logind[483]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.256783] dhcpcd[635]: eth0: soliciting an IPv6 router worker1 # [ 13.260436] dhcpcd[635]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.263240] dhcpcd[635]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.266208] dhcpcd[635]: eth0: adding route to fec0::/64 worker1 # [ 13.268514] dhcpcd[635]: eth0: adding default route via fe80::2 lb # [ 13.419656] systemd-vconsole-setup[760]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 13.429107] systemd[1]: Finished Virtual Console Setup. lb # [ 13.817081] postgresql-pre-start[731]: selecting default time zone ... UTC lb # [ 13.821093] postgresql-pre-start[731]: creating configuration files ... ok lb # [ 13.825338] dhcpcd[627]: eth0: soliciting an IPv6 router lb # [ 13.826165] dhcpcd[627]: eth0: Router Advertisement from fe80::2 lb # [ 13.826985] dhcpcd[627]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 13.827887] dhcpcd[627]: eth0: adding route to fec0::/64 lb # [ 13.834083] dhcpcd[627]: eth0: adding default route via fe80::2 worker2 # [ 13.883973] dhcpcd[639]: eth0: soliciting an IPv6 router worker2 # [ 13.884515] dhcpcd[639]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.884603] dhcpcd[639]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.884679] dhcpcd[639]: eth0: adding route to fec0::/64 worker2 # [ 13.884803] dhcpcd[639]: eth0: adding default route via fe80::2 lb # [ 14.058275] postgresql-pre-start[731]: running bootstrap script ... ok client # [ 14.186830] dhcpcd[587]: eth0: soliciting an IPv6 router client # [ 14.190451] dhcpcd[587]: eth0: Router Advertisement from fe80::2 client # [ 14.193387] dhcpcd[587]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.196385] dhcpcd[587]: eth0: adding route to fec0::/64 client # [ 14.198617] dhcpcd[587]: eth0: adding default route via fe80::2 lb # [ 14.547903] postgresql-pre-start[731]: performing post-bootstrap initialization ... ok lb # [ 14.697744] postgresql-pre-start[731]: syncing data to disk ... ok lb # [ 14.699907] postgresql-pre-start[731]: initdb: warning: enabling "trust" authentication for local connections lb # [ 14.701881] postgresql-pre-start[731]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 14.704237] postgresql-pre-start[731]: Success. You can now start the database server using: lb # [ 14.705325] postgresql-pre-start[731]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 14.799259] postgres[779]: [779] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 14.802366] postgres[779]: [779] LOG: listening on IPv6 address "::1", port 5432 lb # [ 14.803566] postgres[779]: [779] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 14.806753] postgres[779]: [779] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 14.819632] postgres[788]: [788] LOG: database system was shut down at 2026-09-22 14:51:19 GMT lb # [ 14.828108] postgres[779]: [779] LOG: database system is ready to accept connections lb # [ 14.832431] systemd[1]: Started PostgreSQL Server. lb # [ 14.841892] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.100984] postgresql-setup-start[811]: CREATE DATABASE lb # [ 15.141698] postgresql-setup-start[819]: CREATE ROLE lb # [ 15.158284] postgresql-setup-start[821]: ALTER DATABASE lb # [ 15.164063] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.166102] systemd[1]: Reached target PostgreSQL. worker2 # [ 15.934139] dhcpcd[639]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 15.936468] dhcpcd[639]: eth0: adding route to 10.0.2.0/24 worker2 # [ 15.937736] dhcpcd[639]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.041815] systemd[1]: Started DHCP Client. worker2 # [ 16.043746] systemd[1]: Reached target Multi-User System. worker2 # [ 16.048313] systemd[1]: Startup finished in 966ms (kernel) + 5.036s (initrd) + 10.040s (userspace) = 16.043s. worker1 # [ 16.267005] dhcpcd[635]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 16.270583] dhcpcd[635]: eth0: adding route to 10.0.2.0/24 worker1 # [ 16.278988] dhcpcd[635]: eth0: adding default route via 10.0.2.2 worker1 # [ 16.442359] systemd[1]: Started DHCP Client. worker1 # [ 16.444254] systemd[1]: Reached target Multi-User System. worker1 # [ 16.445159] systemd[1]: Startup finished in 948ms (kernel) + 4.940s (initrd) + 10.554s (userspace) = 16.443s. lb # [ 17.312722] dhcpcd[627]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.316753] dhcpcd[627]: eth0: adding route to 10.0.2.0/24 lb # [ 17.319120] dhcpcd[627]: eth0: adding default route via 10.0.2.2 lb # [ 17.459276] systemd[1]: Started DHCP Client. lb # [ 17.463898] systemd[1]: Reached target Network is Online. lb # [ 17.466552] systemd[1]: Started Envoy reverse proxy. lb # [ 17.476719] systemd[1]: Started mock-oidc.service. client # [ 17.587301] dhcpcd[587]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.590637] dhcpcd[587]: eth0: adding route to 10.0.2.0/24 client # [ 17.594970] dhcpcd[587]: eth0: adding default route via 10.0.2.2 client # [ 17.728152] systemd[1]: Started DHCP Client. client # [ 17.730685] systemd[1]: Reached target Multi-User System. client # [ 17.732423] systemd[1]: Startup finished in 956ms (kernel) + 4.980s (initrd) + 11.794s (userspace) = 17.731s. lb # [ 17.760698] mock-oidc-server[898]: Mock OIDC Server running lb # [ 17.763340] mock-oidc-server[898]: OIDC Address: 192.168.1.2:8080 lb # [ 17.764632] mock-oidc-server[898]: Issue Address: 0.0.0.0:8081 lb # [ 17.766008] mock-oidc-server[898]: Issuer: http://192.168.1.2:8080/oidc lb # [ 17.767239] mock-oidc-server[898]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 17.768835] mock-oidc-server[898]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 17.770425] mock-oidc-server[898]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 17.874072] envoy[897]: [2026-09-22 14:51:22.481][897][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 17.876914] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 17.878409] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 17.880502] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 17.887451] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 17.891273] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 17.898952] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 17.901386] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 17.907088] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 17.910335] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 17.963452] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 17.966540] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 17.969314] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 17.971315] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 17.975151] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 17.982673] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 17.986838] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 17.989035] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 17.990745] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 17.993088] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 17.994751] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 17.997222] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 17.999276] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.005738] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.007498] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.011311] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.013878] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 18.017259] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 18.019110] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.022047] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.024163] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.025929] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.030906] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 18.033990] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 18.039700] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.042831] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.044822] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.046749] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 18.049238] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.052440] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.055525] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.057683] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 18.060115] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.061898] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.063990] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 18.066094] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 18.070372] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.072326] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 18.077301] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 18.080776] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.083246] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.085154] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.087781] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 18.104294] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 18.112249] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule worker2 # [ 18.148990] systemd[1]: Reached target Network is Online. lb # [ 18.115033] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic worker1 # [ 18.101081] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.116960] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.119167] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain worker2 # [ 18.155744] systemd[1]: Starting scheduler-token.service... lb # [ 18.122270] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.126172] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.133025] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.134528] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.141805] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.154905] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.162824] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.170925] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 18.176353] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.191925] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 18.197288] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.207453] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 18.210616] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based worker2 # [ 18.251523] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 18.215248] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 18.218857] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default worker2 # [ 18.260137] systemd[1]: Finished scheduler-token.service. worker2 # [ 18.261864] systemd[1]: scheduler-token.service: Consumed 64ms CPU time over 99ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 18.220721] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.232613] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 18.234463] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua worker2 # [ 18.275577] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.238810] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 18.243118] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.247190] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 18.252902] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.254899] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.258938] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 18.261680] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 18.263516] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.266421] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 18.270374] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 18.279428] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.283699] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.288307] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.291507] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.293939] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default worker1 # [ 18.283271] nix-grpc-daemon[766]: ts=2026-09-22T14:51:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.295678] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher worker1 # [ 18.291984] nix-grpc-daemon[766]: ts=2026-09-22T14:51:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 18.305805] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor worker1 # [ 18.294152] nix-grpc-daemon[766]: ts=2026-09-22T14:51:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.308900] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 18.311115] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.314144] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats worker1 # [ 18.304519] nix-grpc-daemon[766]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.317230] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.319625] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 worker1 # [ 18.307455] nix-grpc-daemon[766]: E0000 00:00:1790088682.922837 766 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.321799] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation worker1 # [ 18.312844] nix-grpc-daemon[766]: E0000 00:00:1790088682.931354 766 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.323877] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.329159] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 18.331763] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 18.333869] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 18.335426] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher worker1 # [ 18.332557] nix-grpc-daemon[766]: ts=2026-09-22T14:51:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 18.335766] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.342454] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.353301] envoy[897]: [2026-09-22 14:51:22.484][897][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated worker1 # [ 18.341083] nix-grpc-daemon[766]: E0000 00:00:1790088682.959311 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.355189] envoy[897]: [2026-09-22 14:51:22.506][897][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 18.356781] envoy[897]: [2026-09-22 14:51:22.506][897][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 18.374378] envoy[897]: [2026-09-22 14:51:22.506][897][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 18.360871] nix-grpc-daemon[766]: E0000 00:00:1790088682.979383 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.414245] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 18.368051] nix-grpc-daemon[766]: E0000 00:00:1790088682.982371 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.375959] envoy[897]: [2026-09-22 14:51:22.506][897][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id worker2 # [ 18.422779] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 18.424735] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.389344] envoy[897]: [2026-09-22 14:51:22.506][897][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.391126] envoy[897]: [2026-09-22 14:51:22.565][897][info][main] [source/server/server.cc:968] runtime: {} lb # [ 18.392540] envoy[897]: [2026-09-22 14:51:22.566][897][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 18.394183] envoy[897]: [2026-09-22 14:51:22.566][897][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.395748] envoy[897]: [2026-09-22 14:51:22.567][897][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 worker1 # [ 18.383130] nix-grpc-daemon[766]: E0000 00:00:1790088683.000829 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.397805] envoy[897]: [2026-09-22 14:51:22.567][897][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.399423] envoy[897]: [2026-09-22 14:51:22.567][897][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) worker1 # [ 18.388704] nix-grpc-daemon[766]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.438696] nix-grpc-daemon[770]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.404748] envoy[897]: [2026-09-22 14:51:22.567][897][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 18.406307] envoy[897]: [2026-09-22 14:51:22.711][897][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker2 # [ 18.441884] nix-grpc-daemon[770]: E0000 00:00:1790088683.038615 770 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.407817] envoy[897]: [2026-09-22 14:51:22.733][897][info][config] [source/server/configuration_impl.cc:164] loading stats configuration worker2 # [ 18.446712] nix-grpc-daemon[770]: E0000 00:00:1790088683.047083 770 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.411089] envoy[897]: [2026-09-22 14:51:22.733][897][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 18.414471] envoy[897]: [2026-09-22 14:51:22.733][897][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 18.415966] envoy[897]: [2026-09-22 14:51:22.982][897][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.418432] envoy[897]: [2026-09-22 14:51:22.982][897][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 18.420281] envoy[897]: [2026-09-22 14:51:22.982][897][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 18.425201] envoy[897]: [2026-09-22 14:51:23.018][897][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 18.466258] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 18.471237] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 18.475440] nix-grpc-daemon[770]: E0000 00:00:1790088683.074410 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.493748] nix-grpc-daemon[770]: E0000 00:00:1790088683.093648 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.508065] nix-grpc-daemon[770]: E0000 00:00:1790088683.096585 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.515655] nix-grpc-daemon[770]: E0000 00:00:1790088683.115166 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.527119] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.529055] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.589502] nix-grpc-daemon[766]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.730740] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.736368] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.937828] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.943798] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.990262] nix-grpc-daemon[766]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.145444] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.151260] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.352649] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.358189] nix-grpc-daemon[770]: ts=2026-09-22T14:51:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.561310] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.565788] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.766911] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.772484] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.793312] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.798228] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.973632] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.978854] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.999490] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.005152] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.180215] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.185465] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.206208] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.211771] nix-grpc-daemon[766]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.386945] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.392606] nix-grpc-daemon[770]: ts=2026-09-22T14:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.412151] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.417446] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.595706] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.600111] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.618548] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.624167] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.796992] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.802521] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.827123] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.831632] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.003684] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.012161] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.027735] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.033379] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.210644] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.215971] nix-grpc-daemon[770]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.235488] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.246180] nix-grpc-daemon[766]: ts=2026-09-22T14:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.417264] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.422636] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.436721] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.442445] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.625742] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.629958] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.643402] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.648956] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.827005] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.833037] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.852339] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.856681] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.034382] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.039933] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.053224] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.058935] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.241378] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.247224] nix-grpc-daemon[770]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.260102] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.265796] nix-grpc-daemon[766]: ts=2026-09-22T14:51:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.448797] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.455279] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.466566] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.472395] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.657996] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.662418] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.673369] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.678941] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.859568] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.865498] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.881816] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.886151] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.066678] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.072168] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.082680] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.088399] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.273448] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.278991] nix-grpc-daemon[770]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.289258] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.294648] nix-grpc-daemon[766]: ts=2026-09-22T14:51:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.480501] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.487579] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.495867] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.501669] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.686798] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.691321] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.702818] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.709394] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.888481] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.894349] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.911897] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.917796] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.095757] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.101814] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.118565] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.123048] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.305663] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.314788] nix-grpc-daemon[770]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.324209] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.330057] nix-grpc-daemon[766]: ts=2026-09-22T14:51:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.512489] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.517870] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.530737] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.534169] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.720828] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.725716] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.735313] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.741576] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.927177] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.932735] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.944655] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.949181] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.133988] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.139469] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.145726] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.151428] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.340915] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.346530] nix-grpc-daemon[770]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.352391] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.358397] nix-grpc-daemon[766]: ts=2026-09-22T14:51:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.547507] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.553499] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.559436] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.565312] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.756727] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.761002] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.766054] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.771715] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.957826] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.963479] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.975172] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.979746] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.164632] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.170037] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.175409] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.178662] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.371304] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.377046] nix-grpc-daemon[770]: ts=2026-09-22T14:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.380203] nix-grpc-daemon[766]: ts=2026-09-22T14:51:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.387169] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.578577] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.584954] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.587692] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.593354] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.788686] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.792854] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.794227] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.799666] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.989823] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.995598] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.002619] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.006989] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.196857] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.202240] nix-grpc-daemon[770]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.203455] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.209519] nix-grpc-daemon[766]: ts=2026-09-22T14:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.403632] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.409269] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.410169] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.415654] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.610326] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.615744] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.616785] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.622335] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 27.653162] rustfs-bucket-start[998]: mb s3://farm lb # [ 27.673208] systemd[1]: Finished rustfs-bucket.service. lb # [ 27.686168] systemd[1]: Starting niks3 server... worker2 # [ 27.819083] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.823374] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 27.829838] postgres[1013]: [1013] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 27.831111] postgres[1013]: [1013] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC worker1 # [ 27.823241] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.829213] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 27.858433] niks3-server[1007]: 2026/09/22 14:51:32 OK 20241026095416_initial_model.sql (15.97ms) lb # [ 27.861982] niks3-server[1007]: 2026/09/22 14:51:32 OK 20251210153512_drop_unused_gin_index.sql (3.88ms) lb # [ 27.867699] niks3-server[1007]: 2026/09/22 14:51:32 OK 20251218171726_add_pins.sql (5.65ms) lb # [ 27.872273] niks3-server[1007]: 2026/09/22 14:51:32 OK 20260628120000_add_object_size_and_stats.sql (4.53ms) lb # [ 27.877301] niks3-server[1007]: 2026/09/22 14:51:32 OK 20260905000000_add_claims.sql (4.99ms) lb # [ 27.881154] niks3-server[1007]: 2026/09/22 14:51:32 OK 20260920000000_drop_claims.sql (3.74ms) lb # [ 27.883108] niks3-server[1007]: 2026/09/22 14:51:32 goose: successfully migrated database to version: 20260920000000 lb # [ 27.886641] niks3-server[1007]: 2026/09/22 14:51:32 OK 1_commit_pending_closure.sql (5.45ms) lb # [ 27.889259] niks3-server[1007]: 2026/09/22 14:51:32 OK 2_object_stats_trigger.sql (2.46ms) lb # [ 27.891199] niks3-server[1007]: 2026/09/22 14:51:32 goose: up to current file version: 2 lb # [ 27.894476] niks3-server[1007]: 2026/09/22 14:51:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 27.916304] niks3-server[1007]: 2026/09/22 14:51:32 INFO Read proxy enabled — serving cache objects from S3 lb # [ 27.920932] niks3-server[1007]: 2026/09/22 14:51:32 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 27.925583] niks3-server[1007]: 2026/09/22 14:51:32 INFO systemd watchdog enabled interval=15s lb # [ 27.929525] niks3-server[1007]: 2026/09/22 14:51:32 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 27.930899] systemd[1]: Started niks3 server. lb # [ 27.931926] systemd[1]: Reached target Multi-User System. lb # [ 27.933967] systemd[1]: Startup finished in 959ms (kernel) + 4.915s (initrd) + 22.053s (userspace) = 27.928s. worker2 # [ 28.019887] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.025266] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.032534] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.037217] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.226427] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.231802] nix-grpc-daemon[770]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.232661] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.238095] nix-grpc-daemon[766]: ts=2026-09-22T14:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.433049] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.438265] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 15.81 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 28.438717] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.444731] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 28.583493] envoy[897]: conn peer=[::1]:53222 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=3 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker2 # [ 28.639525] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.645229] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.645941] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.654992] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.791107] systemd[1]: Starting scheduler-token.service... worker2 # [ 28.846701] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.852424] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.874526] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 28.878804] systemd[1]: Finished scheduler-token.service. worker2 # [ 28.882784] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 28.854318] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.859855] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 29.051622] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.057220] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.062921] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.067453] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.258362] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.263749] nix-grpc-daemon[770]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.263635] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.269610] nix-grpc-daemon[766]: ts=2026-09-22T14:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.464748] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.467914] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.470504] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.476332] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.669228] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.674996] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.677308] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.683036] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.878155] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.882372] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.883978] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.889562] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 30.079349] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.085272] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 30.092665] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.097162] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.286500] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.291928] nix-grpc-daemon[770]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.293859] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.300721] nix-grpc-daemon[766]: ts=2026-09-22T14:51:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.493348] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.502111] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.502430] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.508847] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.701098] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.707628] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.710151] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.716634] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.913330] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.917803] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.917802] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.923986] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 31.114486] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.120676] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 31.128567] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.133101] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.322497] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.328926] nix-grpc-daemon[770]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.328275] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.332603] nix-grpc-daemon[766]: ts=2026-09-22T14:51:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.530302] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.536652] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.533791] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.539476] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.738285] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.744612] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.740735] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.746229] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.949896] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.954625] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.947340] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.953651] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.151595] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.157548] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 32.156417] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.161869] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker2 # [ 32.359352] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.365929] nix-grpc-daemon[770]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.363293] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.369212] nix-grpc-daemon[766]: ts=2026-09-22T14:51:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.566850] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.573301] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.570424] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.576217] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.774399] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.780305] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.777673] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.783666] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.986505] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.991103] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.985776] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.992173] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.187132] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.192780] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.196330] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.200760] nix-grpc-daemon[766]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2 # [ 33.395374] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.401355] nix-grpc-daemon[770]: ts=2026-09-22T14:51:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.396769] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.406944] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.599044] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.605911] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.607660] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.613710] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.807687] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.813495] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.815268] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.822161] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.020836] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.025356] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.023860] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.030457] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.219756] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.225658] nix-grpc-daemon[770]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.234383] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.239809] nix-grpc-daemon[766]: ts=2026-09-22T14:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 34.427492] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.433074] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1 # [ 34.440541] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.446514] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.634365] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.639998] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.647304] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.651624] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.841208] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.847807] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.852921] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.858895] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.052761] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.057289] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.060820] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.066727] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.254391] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.260463] nix-grpc-daemon[770]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.270455] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.275234] nix-grpc-daemon[766]: ts=2026-09-22T14:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 35.461755] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.469221] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.471375] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.477151] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 35.670938] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.676609] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.678248] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.684784] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.878109] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.883751] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.885952] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.891486] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.088708] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.093239] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.092680] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.098260] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.289636] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.295302] nix-grpc-daemon[770]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.301410] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.306656] nix-grpc-daemon[766]: ts=2026-09-22T14:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.496815] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.503020] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 36.507423] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.512865] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 36.704120] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.709559] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.713813] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.719155] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.910750] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.916276] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.919997] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.925548] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.119274] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.123586] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.126047] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.128963] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.320677] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.326118] nix-grpc-daemon[770]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.331911] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.336096] nix-grpc-daemon[766]: ts=2026-09-22T14:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.527114] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.532801] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.532506] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.537784] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 37.734084] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.739546] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.738575] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.743781] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.940803] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.946107] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.923469] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_take_over lb # [ 37.937774] niks3-server[1007]: 2026/09/22 14:51:42 INFO lead: acquired remote=192.168.1.3:48398 worker1 # [ 37.944285] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.947293] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.149135] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.153560] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.147866] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.153264] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.350187] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.355517] nix-grpc-daemon[770]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.355940] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.360127] nix-grpc-daemon[766]: ts=2026-09-22T14:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.556713] nix-grpc-daemon[770]: ts=2026-09-22T14:51:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.537992] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 38.542676] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:54534 worker1 # [ 38.548140] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 38.556288] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.561949] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:54534 worker1 # [ 38.566905] nix-grpc-daemon[766]: ts=2026-09-22T14:51:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.48 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 39.723262] nix-grpc-daemon[770]: ts=2026-09-22T14:51:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 39.745522] systemd[1]: Started Nix Daemon. worker2 # [ 39.921886] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 39.939350] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5) worker2 # [ 40.018271] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 2/5) worker2 # [ 40.053922] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 278 ms (attempt 3/5) worker2 # [ 40.334563] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 591 ms (attempt 4/5) worker2 # [ 40.929654] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 40.944314] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 40.917291] nix-grpc-daemon[766]: ts=2026-09-22T14:51:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 40.938245] systemd[1]: Started Nix Daemon. worker1 # [ 41.117287] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 41.137785] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5) worker1 # [ 41.165596] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5) worker1 # [ 41.307204] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5) worker1 # [ 41.449339] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 217 ms (attempt 4/5) worker1 # [ 41.670106] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 41.675406] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 41.687065] nix-grpc-daemon[766]: ts=2026-09-22T14:51:46Z level=debug event=path_info_miss path=9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test error="error: path '/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 41.702946] nix-grpc-daemon[766]: ts=2026-09-22T14:51:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 41.716444] nix-grpc-daemon[766]: ts=2026-09-22T14:51:46Z level=debug event=path_info_miss path=9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test error="error: path '/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.03 seconds) worker2 # [ 42.041358] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 42.047525] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:50030 worker2 # [ 42.060647] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:50030 worker2 # [ 42.075476] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=debug event=path_info_miss path=9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test error="error: path '/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 42.089575] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:50030 worker2 # [ 42.101857] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=debug event=path_info_miss path=9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test error="error: path '/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 42.339539] nix-grpc-daemon[770]: ts=2026-09-22T14:51:46Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 42.517020] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.09 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 42.651406] nix-grpc-daemon[766]: ts=2026-09-22T14:51:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 42.714920] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 42.709877] nix-grpc-daemon[766]: ts=2026-09-22T14:51:47Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 42.715886] nix-grpc-daemon[766]: ts=2026-09-22T14:51:47Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 42.806710] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 42.832524] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying 1 paths to 'lb:50051'... worker2 # [ 42.850475] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker2 # [ 42.877557] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 42.915135] nix-grpc-daemon[770]: ts=2026-09-22T14:51:47Z level=info event=niks3_push_started pid=852 worker2 # [ 43.101274] nix-grpc-daemon[852]: time=2026-09-22T14:51:47.701Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 43.221357] nix-daemon[839]: accepted connection from pid 865, user nix-grpc-daemon (trusted) worker2 # [ 43.247869] nix-daemon[839]: reaped child process 868, status = succeeded worker2 # [ 43.349573] nix-daemon[839]: accepted connection from pid 870, user nix-grpc-daemon (trusted) worker2 # [ 43.369095] nix-daemon[839]: reaped child process 873, status = succeeded lb # [ 43.338375] niks3-server[1007]: 2026/09/22 14:51:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 43.397188] nix-grpc-daemon[852]: time=2026-09-22T14:51:47.996Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 43.398701] nix-grpc-daemon[852]: time=2026-09-22T14:51:47.996Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 43.411157] niks3-server[1007]: 2026/09/22 14:51:48 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 43.423663] niks3-server[1007]: 2026/09/22 14:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 43.436468] niks3-server[1007]: 2026/09/22 14:51:48 INFO Signed narinfos id=1 count=1 worker2 # [ 43.477219] nix-grpc-daemon[852]: time=2026-09-22T14:51:48.076Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.445532] niks3-server[1007]: 2026/09/22 14:51:48 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 43.460845] niks3-server[1007]: 2026/09/22 14:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 43.478908] niks3-server[1007]: 2026/09/22 14:51:48 INFO Completed upload id=1 worker2 # [ 43.515993] nix-grpc-daemon[852]: time=2026-09-22T14:51:48.116Z level=INFO msg="Upload complete. (410ms)" worker2 # [ 43.528718] nix-grpc-daemon[770]: ts=2026-09-22T14:51:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 43.550967] nix-grpc-daemon[770]: ts=2026-09-22T14:51:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 43.563462] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 43.566529] nix-daemon[839]: reaped child process 850, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker2 # [ 43.584774] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 44.215859] nix-daemon[839]: reaped child process 877, status = succeeded worker2 # [ 44.333221] nix-daemon[839]: accepted connection from pid 893, user nix-grpc-daemon (trusted) lb # [ 44.328712] niks3-server[1007]: 2026/09/22 14:51:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 44.371007] nix-daemon[839]: reaped child process 896, status = succeeded worker2 # [ 44.384895] nix-grpc-daemon[852]: time=2026-09-22T14:51:48.981Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 44.390985] nix-grpc-daemon[852]: time=2026-09-22T14:51:48.981Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 44.383630] niks3-server[1007]: 2026/09/22 14:51:48 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst worker2 # [ 44.438567] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.038Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.402153] niks3-server[1007]: 2026/09/22 14:51:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 44.412356] niks3-server[1007]: 2026/09/22 14:51:49 INFO Signed narinfos id=2 count=1 lb # [ 44.417658] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 44.428397] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 44.431311] niks3-server[1007]: 2026/09/22 14:51:49 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 44.443488] niks3-server[1007]: 2026/09/22 14:51:49 INFO Completed upload id=2 worker2 # [ 44.480516] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.080Z level=INFO msg="Upload complete. (284ms)" lb # [ 44.446330] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 44.450596] nix-grpc-daemon[766]: ts=2026-09-22T14:51:49Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 44.492339] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 44.508743] nix-daemon[839]: reaped child process 875, status = succeeded worker1 # [ 44.456621] nix-grpc-daemon[766]: ts=2026-09-22T14:51:49Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 44.546338] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 44.564424] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 44.579089] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.179Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 44.590273] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker2 # [ 44.612622] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 44.776381] nix-daemon[839]: accepted connection from pid 901, user nix-grpc-daemon (trusted) worker2 # [ 44.795268] nix-daemon[839]: reaped child process 904, status = succeeded worker2 # [ 44.901811] nix-daemon[839]: accepted connection from pid 906, user nix-grpc-daemon (trusted) worker2 # [ 44.919247] nix-daemon[839]: reaped child process 909, status = succeeded lb # [ 44.888663] niks3-server[1007]: 2026/09/22 14:51:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 44.935227] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.535Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 44.936839] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.535Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 44.924542] niks3-server[1007]: 2026/09/22 14:51:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 44.936774] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 44.947769] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 44.955971] niks3-server[1007]: 2026/09/22 14:51:49 INFO Signed narinfos id=3 count=1 worker2 # [ 44.993071] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.593Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.966490] niks3-server[1007]: 2026/09/22 14:51:49 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 44.980303] niks3-server[1007]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo lb # [ 44.991125] niks3-server[1007]: 2026/09/22 14:51:49 INFO Completed upload id=3 worker2 # [ 45.028123] nix-grpc-daemon[852]: time=2026-09-22T14:51:49.628Z level=INFO msg="Upload complete. (398ms)" worker2 # [ 45.038764] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 45.055758] nix-grpc-daemon[770]: ts=2026-09-22T14:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 45.064534] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 45.071859] nix-daemon[839]: reaped child process 899, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker2 # [ 45.086109] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 45.939684] nix-daemon[839]: reaped child process 913, status = succeeded worker2 # [ 46.082241] nix-daemon[839]: accepted connection from pid 929, user nix-grpc-daemon (trusted) worker2 # [ 46.104518] nix-daemon[839]: reaped child process 932, status = succeeded lb # [ 46.073298] niks3-server[1007]: 2026/09/22 14:51:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.121761] nix-grpc-daemon[852]: time=2026-09-22T14:51:50.721Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 46.123550] nix-grpc-daemon[852]: time=2026-09-22T14:51:50.721Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 46.121996] niks3-server[1007]: 2026/09/22 14:51:50 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker2 # [ 46.175924] nix-grpc-daemon[852]: time=2026-09-22T14:51:50.775Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.131200] niks3-server[1007]: 2026/09/22 14:51:50 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 46.143212] niks3-server[1007]: 2026/09/22 14:51:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 46.151675] niks3-server[1007]: 2026/09/22 14:51:50 INFO Signed narinfos id=4 count=1 lb # [ 46.157193] niks3-server[1007]: 2026/09/22 14:51:50 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 46.168056] niks3-server[1007]: 2026/09/22 14:51:50 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete worker2 # [ 46.204122] nix-grpc-daemon[852]: time=2026-09-22T14:51:50.804Z level=INFO msg="Upload complete. (304ms)" lb # [ 46.171499] niks3-server[1007]: 2026/09/22 14:51:50 INFO Completed upload id=4 lb # [ 46.176257] niks3-server[1007]: 2026/09/22 14:51:50 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker2 # [ 46.217115] nix-grpc-daemon[770]: ts=2026-09-22T14:51:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 46.176757] nix-grpc-daemon[766]: ts=2026-09-22T14:51:50Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 46.177073] nix-grpc-daemon[766]: ts=2026-09-22T14:51:50Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 46.238636] nix-daemon[839]: reaped child process 911, status = succeeded worker2 # [ 46.272985] nix-grpc-daemon[770]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 46.283563] nix-grpc-daemon[770]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 46.297572] nix-grpc-daemon[852]: time=2026-09-22T14:51:50.897Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 46.308761] nix-grpc-daemon[770]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker2 # [ 46.329177] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 46.462985] nix-daemon[839]: accepted connection from pid 936, user nix-grpc-daemon (trusted) worker2 # [ 46.483168] nix-daemon[839]: reaped child process 939, status = succeeded worker2 # [ 46.596164] nix-daemon[839]: accepted connection from pid 941, user nix-grpc-daemon (trusted) worker2 # [ 46.619896] nix-daemon[839]: reaped child process 944, status = succeeded lb # [ 46.581317] niks3-server[1007]: 2026/09/22 14:51:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.631037] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.231Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 46.632634] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.231Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" worker2 # [ 46.668739] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.268Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.618988] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 46.637818] niks3-server[1007]: 2026/09/22 14:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 46.649943] niks3-server[1007]: 2026/09/22 14:51:51 INFO Signed narinfos id=5 count=1 lb # [ 46.656682] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 46.671140] niks3-server[1007]: 2026/09/22 14:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete worker2 # [ 46.714874] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.315Z level=INFO msg="Upload complete. (375ms)" lb # [ 46.678166] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo lb # [ 46.682253] niks3-server[1007]: 2026/09/22 14:51:51 INFO Completed upload id=5 worker2 # [ 46.725331] nix-grpc-daemon[770]: ts=2026-09-22T14:51:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 46.740732] nix-grpc-daemon[770]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 46.747293] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 46.756419] nix-daemon[839]: reaped child process 934, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker2 # [ 46.766173] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 47.998060] nix-daemon[839]: reaped child process 948, status = succeeded worker2 # [ 48.097693] nix-daemon[839]: accepted connection from pid 964, user nix-grpc-daemon (trusted) worker2 # [ 48.118041] nix-daemon[839]: reaped child process 967, status = succeeded lb # [ 48.089184] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 48.140321] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.738Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 48.141840] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.738Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 48.103713] envoy[897]: conn peer=[2001:db8:1::1]:47150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2073 lb # [ 48.114977] envoy[897]: conn peer=[2001:db8:1::1]:49806 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=84 lb # [ 48.124475] envoy[897]: conn peer=[2001:db8:1::1]:49822 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=11 lb # [ 48.133738] envoy[897]: conn peer=[2001:db8:1::1]:49828 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=13 lb # [ 48.153566] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 48.164511] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 48.211820] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.175584] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 48.183746] niks3-server[1007]: 2026/09/22 14:51:52 INFO Signed narinfos id=6 count=1 lb # [ 48.189234] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 48.200769] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 48.210352] niks3-server[1007]: 2026/09/22 14:51:52 INFO Completed upload id=6 worker2 # [ 48.243933] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.844Z level=INFO msg="Upload complete. (268ms)" lb # [ 48.212803] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker2 # [ 48.257328] nix-grpc-daemon[770]: ts=2026-09-22T14:51:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker2 # [ 48.277153] nix-daemon[839]: reaped child process 946, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.77 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath worker1 # [ 48.349269] nix-grpc-daemon[766]: E0000 00:00:1790088712.963320 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.402328] nix-grpc-daemon[766]: E0000 00:00:1790088713.019372 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 48.433121] nix-grpc-daemon[766]: E0000 00:00:1790088713.026196 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.436516] nix-grpc-daemon[766]: E0000 00:00:1790088713.050365 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.31 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 49.476715] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 49.485484] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 49.511691] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.28 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.14 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.16 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 50.402905] nix-grpc-daemon[770]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 50.360357] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 50.377956] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 50.384322] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 50.473380] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 50.491868] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 50.508715] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.107Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker2 # [ 50.521415] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 50.537529] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 50.696961] nix-daemon[839]: accepted connection from pid 984, user nix-grpc-daemon (trusted) worker2 # [ 50.722472] nix-daemon[839]: reaped child process 987, status = succeeded worker2 # [ 50.828283] nix-daemon[839]: accepted connection from pid 989, user nix-grpc-daemon (trusted) worker2 # [ 50.850652] nix-daemon[839]: reaped child process 992, status = succeeded lb # [ 50.822256] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 50.933311] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.532Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 50.936993] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.532Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 50.922851] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker2 # [ 50.974541] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.574Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.938784] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 50.949357] niks3-server[1007]: 2026/09/22 14:51:55 INFO Signed narinfos id=7 count=1 lb # [ 50.956377] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 50.967475] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker2 # [ 51.010133] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.610Z level=INFO msg="Upload complete. (454ms)" lb # [ 50.976284] niks3-server[1007]: 2026/09/22 14:51:55 INFO Completed upload id=7 lb # [ 50.980999] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker2 # [ 51.020501] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 51.041835] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 51.053095] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 51.061041] nix-daemon[839]: reaped child process 982, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker2 # [ 51.075192] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 51.914250] nix-daemon[839]: reaped child process 996, status = succeeded worker2 # [ 52.021733] nix-daemon[839]: accepted connection from pid 1012, user nix-grpc-daemon (trusted) worker2 # [ 52.046079] nix-daemon[839]: reaped child process 1015, status = succeeded lb # [ 52.014638] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.064493] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.662Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.066019] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.662Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 52.057132] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 52.118676] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.718Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.076826] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 52.088350] niks3-server[1007]: 2026/09/22 14:51:56 INFO Signed narinfos id=8 count=1 lb # [ 52.095412] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 52.102267] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 52.112132] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete worker2 # [ 52.154701] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.754Z level=INFO msg="Upload complete. (263ms)" lb # [ 52.118538] niks3-server[1007]: 2026/09/22 14:51:56 INFO Completed upload id=8 lb # [ 52.123908] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker2 # [ 52.169302] nix-grpc-daemon[770]: ts=2026-09-22T14:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker2 # [ 52.187873] nix-daemon[839]: reaped child process 994, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.03 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.56 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker1 # [ 52.735591] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 52.743794] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 52.766285] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker1 # [ 52.774233] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 52.783878] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 52.795941] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker1 # [ 52.812960] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 52.828843] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 52.903702] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=info event=niks3_push_started pid=901 worker1 # [ 53.093711] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.710Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 53.208677] nix-daemon[849]: accepted connection from pid 914, user nix-grpc-daemon (trusted) worker1 # [ 53.229050] nix-daemon[849]: reaped child process 917, status = succeeded worker1 # [ 53.338313] nix-daemon[849]: accepted connection from pid 919, user nix-grpc-daemon (trusted) worker1 # [ 53.357425] nix-daemon[849]: reaped child process 922, status = succeeded lb # [ 53.376760] niks3-server[1007]: 2026/09/22 14:51:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.375441] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.992Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.377439] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.992Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 53.416917] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst worker1 # [ 53.417964] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.036Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.430381] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 53.442635] niks3-server[1007]: 2026/09/22 14:51:58 INFO Signed narinfos id=9 count=1 lb # [ 53.449419] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 53.456771] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker1 # [ 53.452732] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.071Z level=INFO msg="Upload complete. (359ms)" lb # [ 53.467229] niks3-server[1007]: 2026/09/22 14:51:58 INFO Completed upload id=9 lb # [ 53.473408] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker1 # [ 53.464303] nix-grpc-daemon[766]: ts=2026-09-22T14:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 53.475453] nix-grpc-daemon[766]: ts=2026-09-22T14:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 53.482304] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 53.487306] nix-daemon[849]: reaped child process 899, status = succeeded worker1 # [ 53.497965] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker1 # [ 54.136858] nix-daemon[849]: reaped child process 926, status = succeeded worker1 # [ 54.254386] nix-daemon[849]: accepted connection from pid 942, user nix-grpc-daemon (trusted) worker1 # [ 54.275779] nix-daemon[849]: reaped child process 945, status = succeeded lb # [ 54.289754] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.288394] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 54.289851] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.905Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 54.328733] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 54.346238] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 54.358787] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker1 # [ 54.346881] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.964Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.365652] niks3-server[1007]: 2026/09/22 14:51:58 INFO Signed narinfos id=10 count=1 lb # [ 54.372162] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 54.385211] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker1 # [ 54.377659] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.996Z level=INFO msg="Upload complete. (268ms)" lb # [ 54.392319] niks3-server[1007]: 2026/09/22 14:51:58 INFO Completed upload id=10 lb # [ 54.395133] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker1 # [ 54.397097] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker1 # [ 54.413501] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 54.417348] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 54.421343] nix-daemon[849]: reaped child process 924, status = succeeded worker1 # [ 54.444241] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 54.459816] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 54.474438] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.093Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker1 # [ 54.485655] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 54.499480] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 54.658373] nix-daemon[849]: accepted connection from pid 951, user nix-grpc-daemon (trusted) worker1 # [ 54.678547] nix-daemon[849]: reaped child process 954, status = succeeded worker1 # [ 54.792969] nix-daemon[849]: accepted connection from pid 956, user nix-grpc-daemon (trusted) worker1 # [ 54.810532] nix-daemon[849]: reaped child process 959, status = succeeded lb # [ 54.830343] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.827006] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.445Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 54.828710] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.445Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 54.863138] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker1 # [ 54.857407] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.475Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.877818] niks3-server[1007]: 2026/09/22 14:51:59 INFO Signed narinfos id=11 count=1 lb # [ 54.887120] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 54.901837] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 54.906766] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete worker1 # [ 54.899263] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.517Z level=INFO msg="Upload complete. (384ms)" lb # [ 54.914305] niks3-server[1007]: 2026/09/22 14:51:59 INFO Completed upload id=11 worker1 # [ 54.908272] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 54.921850] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker1 # [ 54.919369] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 54.925637] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 54.930619] nix-daemon[849]: reaped child process 949, status = succeeded worker1 # [ 54.940776] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker1 # [ 55.820352] nix-daemon[849]: reaped child process 963, status = succeeded worker1 # [ 55.920164] nix-daemon[849]: accepted connection from pid 979, user nix-grpc-daemon (trusted) worker1 # [ 55.941189] nix-daemon[849]: reaped child process 982, status = succeeded lb # [ 55.958206] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 55.954107] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.571Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 55.955612] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.571Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 56.000797] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker1 # [ 55.999188] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.617Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.017088] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst lb # [ 56.025605] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 56.036325] niks3-server[1007]: 2026/09/22 14:52:00 INFO Signed narinfos id=12 count=1 worker1 # [ 56.023094] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.641Z level=INFO msg="Upload complete. (270ms)" lb # [ 56.038381] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 56.041693] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 56.046039] niks3-server[1007]: 2026/09/22 14:52:00 INFO Completed upload id=12 lb # [ 56.049524] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker1 # [ 56.039914] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker1 # [ 56.060344] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker1 # [ 56.065173] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 56.069738] nix-daemon[849]: reaped child process 961, status = succeeded worker1 # [ 56.100135] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 56.116045] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 56.130256] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.748Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker1 # [ 56.141878] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 56.156067] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 56.291733] nix-daemon[849]: accepted connection from pid 988, user nix-grpc-daemon (trusted) worker1 # [ 56.311870] nix-daemon[849]: reaped child process 991, status = succeeded worker1 # [ 56.420031] nix-daemon[849]: accepted connection from pid 993, user nix-grpc-daemon (trusted) worker1 # [ 56.438867] nix-daemon[849]: reaped child process 996, status = succeeded lb # [ 56.458800] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 56.455846] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.074Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 56.457616] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.074Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 56.492707] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst lb # [ 56.507170] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker1 # [ 56.498758] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.117Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.514651] niks3-server[1007]: 2026/09/22 14:52:01 INFO Signed narinfos id=13 count=1 lb # [ 56.520772] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 56.531901] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete worker1 # [ 56.524795] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.143Z level=INFO msg="Upload complete. (352ms)" lb # [ 56.539918] niks3-server[1007]: 2026/09/22 14:52:01 INFO Completed upload id=13 lb # [ 56.544990] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker1 # [ 56.536122] nix-grpc-daemon[766]: ts=2026-09-22T14:52:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 56.547560] nix-grpc-daemon[766]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 56.555005] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 56.560754] nix-daemon[849]: reaped child process 986, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker1 # [ 56.575422] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker1 # [ 58.056317] nix-daemon[849]: reaped child process 1000, status = succeeded lb # [ 58.102696] envoy[897]: conn peer=[2001:db8:1::1]:49842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9506 tx=4403 ms=5577 lb # [ 58.107748] envoy[897]: conn peer=[2001:db8:1::1]:49856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=57 lb # [ 58.112635] envoy[897]: conn peer=[2001:db8:1::1]:49872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1920 ms=1783 worker1 # [ 58.155191] nix-daemon[849]: accepted connection from pid 1016, user nix-grpc-daemon (trusted) worker1 # [ 58.177067] nix-daemon[849]: reaped child process 1019, status = succeeded lb # [ 58.194879] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 58.192496] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.810Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 58.194699] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.810Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 58.231422] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 58.251145] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker1 # [ 58.244285] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.862Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.263627] niks3-server[1007]: 2026/09/22 14:52:02 INFO Signed narinfos id=14 count=1 lb # [ 58.269667] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst lb # [ 58.279645] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls worker1 # [ 58.271544] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.890Z level=INFO msg="Upload complete. (284ms)" lb # [ 58.286857] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 58.292919] niks3-server[1007]: 2026/09/22 14:52:02 INFO Completed upload id=14 lb # [ 58.295675] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker1 # [ 58.285114] nix-grpc-daemon[766]: ts=2026-09-22T14:52:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker1 # [ 58.306223] nix-daemon[849]: reaped child process 998, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.84 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 58.663788] nix-grpc-daemon[766]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:34082 worker1 # [ 58.706079] nix-grpc-daemon[766]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:34082 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.24 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 58.994728] nix-grpc-daemon[770]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:49046 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 59.028988] nix-grpc-daemon[770]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:49046 worker2 # [ 59.038067] nix-grpc-daemon[770]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker2 # [ 59.056198] nix-grpc-daemon[770]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker2 # [ 59.071607] nix-grpc-daemon[770]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/926" client # removing stale temporary roots file "/nix/var/nix/temproots/849" client # removing stale temporary roots file "/nix/var/nix/temproots/941" client # removing stale temporary roots file "/nix/var/nix/temproots/897" client # removing stale temporary roots file "/nix/var/nix/temproots/918" client # removing stale temporary roots file "/nix/var/nix/temproots/905" client # removing stale temporary roots file "/nix/var/nix/temproots/865" client # removing stale temporary roots file "/nix/var/nix/temproots/886" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 59.459985] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 59.468681] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 59.476457] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.074Z level=INFO msg="All 1 paths already cached" worker1 # [ 59.430767] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 59.489403] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 59.448954] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 client # copying 3 paths... worker2 # [ 59.512674] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 59.539025] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 59.547014] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 59.503835] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 59.511733] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 59.569344] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 59.576166] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.56 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.11 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 59.757748] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 4908e2eca0a44b08b68866ba6a2bdb39 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker2 # [ 59.924442] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50030 client # [ 59.958425] nix[994]: don't know how to build these paths: client # [ 59.960264] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 59.892090] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 59.897556] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.988713] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 60.002017] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 client # [ 60.040221] nix[994]: copying 1 paths to 'lb:50051'... client # [ 60.043727] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker2 # [ 60.014406] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 60.030525] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 60.199071] nix-daemon[839]: accepted connection from pid 1022, user nix-grpc-daemon (trusted) worker2 # [ 60.230533] nix-daemon[839]: reaped child process 1025, status = succeeded worker2 # [ 60.327393] nix-daemon[839]: accepted connection from pid 1027, user nix-grpc-daemon (trusted) worker2 # [ 60.346765] nix-daemon[839]: reaped child process 1030, status = succeeded lb # [ 60.315505] niks3-server[1007]: 2026/09/22 14:52:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 60.365766] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 60.367228] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.965Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 60.348813] niks3-server[1007]: 2026/09/22 14:52:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker2 # [ 60.402080] nix-grpc-daemon[852]: time=2026-09-22T14:52:05.002Z level=INFO msg="Uploading 1 narinfos" lb # [ 60.365513] niks3-server[1007]: 2026/09/22 14:52:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 60.377365] niks3-server[1007]: 2026/09/22 14:52:04 INFO Signed narinfos id=15 count=1 lb # [ 60.391854] niks3-server[1007]: 2026/09/22 14:52:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 60.397870] niks3-server[1007]: 2026/09/22 14:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 60.409896] niks3-server[1007]: 2026/09/22 14:52:05 INFO Completed upload id=15 worker2 # [ 60.446785] nix-grpc-daemon[852]: time=2026-09-22T14:52:05.047Z level=INFO msg="Upload complete. (399ms)" lb # [ 60.413047] niks3-server[1007]: 2026/09/22 14:52:05 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker2 # [ 60.456383] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 60.468948] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 60.482400] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 60.488163] nix-daemon[839]: reaped child process 1020, status = succeeded client # [ 60.525887] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 60.502206] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # [ 60.850305] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 4186601b977047d99081da3bbbef5b9f client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) worker1 # [ 60.999170] nix-grpc-daemon[766]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 client # [ 61.082997] nix[1010]: don't know how to build these paths: client # [ 61.084983] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 61.017283] nix-grpc-daemon[766]: ts=2026-09-22T14:52:05Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 61.073982] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 client # [ 61.108172] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.06 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 64.534543] nix-daemon[839]: reaped child process 1034, status = succeeded worker2 # [ 64.656511] nix-daemon[839]: accepted connection from pid 1064, user nix-grpc-daemon (trusted) worker2 # [ 64.679263] nix-daemon[839]: reaped child process 1067, status = succeeded lb # [ 64.642287] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 64.691089] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 64.694627] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.289Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 64.684272] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 64.696986] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 64.710178] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker2 # [ 64.751243] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.351Z level=INFO msg="Uploading 1 narinfos" lb # [ 64.718431] niks3-server[1007]: 2026/09/22 14:52:09 INFO Signed narinfos id=16 count=1 lb # [ 64.723662] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 64.734070] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 64.740410] niks3-server[1007]: 2026/09/22 14:52:09 INFO Completed upload id=16 worker2 # [ 64.776135] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.376Z level=INFO msg="Upload complete. (255ms)" lb # [ 64.743587] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker2 # [ 64.789273] nix-grpc-daemon[770]: ts=2026-09-22T14:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 64.808378] nix-grpc-daemon[770]: ts=2026-09-22T14:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 64.815358] nix-daemon[839]: reaped child process 1032, status = succeeded client # [ 64.851898] systemd[1]: dedup1.service: Deactivated successfully. client # [ 64.856553] systemd[1]: dedup1.service: Consumed 179ms CPU time over 5.098s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic. client # [ 64.895946] systemd[1]: dedup2.service: Deactivated successfully. client # [ 64.899798] systemd[1]: dedup2.service: Consumed 203ms CPU time over 4.049s wall clock time, 35.3M memory peak, 6.1K incoming IP traffic, 5.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.10 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 65.961351] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 65.979710] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 65.987223] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 65.996939] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 66.008266] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 66.022721] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 66.038073] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 66.196772] nix-daemon[849]: accepted connection from pid 1050, user nix-grpc-daemon (trusted) worker1 # [ 66.220603] nix-daemon[849]: reaped child process 1053, status = succeeded worker1 # [ 66.328619] nix-daemon[849]: accepted connection from pid 1055, user nix-grpc-daemon (trusted) worker1 # [ 66.349896] nix-daemon[849]: reaped child process 1058, status = succeeded lb # [ 66.362261] niks3-server[1007]: 2026/09/22 14:52:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.359964] nix-grpc-daemon[901]: time=2026-09-22T14:52:10.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.361590] nix-grpc-daemon[901]: time=2026-09-22T14:52:10.978Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 66.395622] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker1 # [ 66.405513] nix-grpc-daemon[901]: time=2026-09-22T14:52:11.024Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.413125] niks3-server[1007]: 2026/09/22 14:52:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 66.421164] niks3-server[1007]: 2026/09/22 14:52:11 INFO Signed narinfos id=17 count=1 lb # [ 66.427590] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 66.437072] niks3-server[1007]: 2026/09/22 14:52:11 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker1 # [ 66.432346] nix-grpc-daemon[901]: time=2026-09-22T14:52:11.051Z level=INFO msg="Upload complete. (378ms)" lb # [ 66.447213] niks3-server[1007]: 2026/09/22 14:52:11 INFO Completed upload id=17 lb # [ 66.453738] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker1 # [ 66.440263] nix-grpc-daemon[766]: ts=2026-09-22T14:52:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 66.452338] nix-grpc-daemon[766]: ts=2026-09-22T14:52:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 66.464515] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 66.466996] nix-daemon[849]: reaped child process 1048, status = succeeded worker1 # [ 66.477353] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) lb # [ 68.104987] envoy[897]: conn peer=[2001:db8:1::1]:53044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9502 tx=4369 ms=5577 lb # [ 68.109878] envoy[897]: conn peer=[2001:db8:1::1]:53058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2930 ms=191 lb # [ 68.114765] envoy[897]: conn peer=[2001:db8:1::1]:53064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=112 lb # [ 68.119639] envoy[897]: conn peer=[2001:db8:1::1]:53096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=84 lb # [ 68.124457] envoy[897]: conn peer=[2001:db8:1::1]:53080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=99 lb # [ 68.129094] envoy[897]: conn peer=[2001:db8:1::1]:53102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1538 ms=4878 lb # [ 68.134163] envoy[897]: conn peer=[2001:db8:1::1]:46654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=3799 worker1 # [ 72.296551] nix-grpc-daemon[766]: ts=2026-09-22T14:52:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker1 # [ 72.342099] nix-daemon[849]: reaped child process 1060, status = succeeded worker1 # [ 72.346121] nix-daemon[849]: reaped child process 1062, status = succeeded worker2 # [ 72.625591] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 72.594927] nix-grpc-daemon[766]: ts=2026-09-22T14:52:17Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 72.601375] nix-grpc-daemon[766]: ts=2026-09-22T14:52:17Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 72.655204] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 72.667468] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 72.679572] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 72.695523] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 72.861006] nix-daemon[839]: accepted connection from pid 1080, user nix-grpc-daemon (trusted) worker2 # [ 72.881999] nix-daemon[839]: reaped child process 1083, status = succeeded worker2 # [ 72.981490] nix-daemon[839]: accepted connection from pid 1085, user nix-grpc-daemon (trusted) lb # [ 72.963128] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 73.001562] nix-daemon[839]: reaped child process 1088, status = succeeded worker2 # [ 73.006527] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.605Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 73.008342] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.605Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" worker2 # [ 73.028864] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.629Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.986370] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign lb # [ 72.994937] niks3-server[1007]: 2026/09/22 14:52:17 INFO Signed narinfos id=18 count=1 lb # [ 72.999544] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 73.005938] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 73.012892] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete worker2 # [ 73.055065] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.655Z level=INFO msg="Upload complete. (340ms)" lb # [ 73.019821] niks3-server[1007]: 2026/09/22 14:52:17 INFO Completed upload id=18 lb # [ 73.023260] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker2 # [ 73.063370] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 73.067077] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 73.076321] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 73.079884] nix-daemon[839]: reaped child process 1078, status = succeeded worker2 # [ 73.090872] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) lb # [ 78.102319] envoy[897]: conn peer=[2001:db8:1::1]:46664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=952 ms=6397 worker2 # [ 79.509825] nix-grpc-daemon[770]: ts=2026-09-22T14:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker2 # [ 79.545455] nix-daemon[839]: reaped child process 1090, status = succeeded worker2 # [ 79.547750] nix-daemon[839]: reaped child process 1092, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.92 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 79.789106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 2ac284a750024a9db40d8dbb7f5827c1 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 79.921544] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 client # [ 80.005775] nix[1084]: don't know how to build these paths: client # [ 80.006724] nix[1084]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 79.941642] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 79.951818] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 79.960961] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 79.968650] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 client # [ 80.053438] nix[1084]: copying 1 paths to 'lb:50051'... client # [ 80.057064] nix[1084]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker1 # [ 79.981350] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 79.996899] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 80.154582] nix-daemon[849]: accepted connection from pid 1089, user nix-grpc-daemon (trusted) worker1 # [ 80.181032] nix-daemon[849]: reaped child process 1092, status = succeeded worker1 # [ 80.275739] nix-daemon[849]: accepted connection from pid 1094, user nix-grpc-daemon (trusted) worker1 # [ 80.296252] nix-daemon[849]: reaped child process 1097, status = succeeded lb # [ 80.308818] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.306824] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 80.308378] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.924Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 80.344211] niks3-server[1007]: 2026/09/22 14:52:24 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 80.363282] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker1 # [ 80.357110] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.975Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.373724] niks3-server[1007]: 2026/09/22 14:52:24 INFO Signed narinfos id=19 count=1 lb # [ 80.380481] niks3-server[1007]: 2026/09/22 14:52:24 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 80.392595] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 80.402016] niks3-server[1007]: 2026/09/22 14:52:25 INFO Completed upload id=19 worker1 # [ 80.386654] nix-grpc-daemon[901]: time=2026-09-22T14:52:25.005Z level=INFO msg="Upload complete. (373ms)" lb # [ 80.406664] niks3-server[1007]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker1 # [ 80.396395] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 80.412141] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 80.420499] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 80.423288] nix-daemon[849]: reaped child process 1087, status = succeeded worker1 # [ 80.433551] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # [ 80.509164] nix[1084]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 80.894147] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 1ddf70d8171b4ed7a48f55da11b84ca8 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) worker1 # [ 81.034523] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 client # [ 81.125009] nix[1100]: this derivation will be built: worker2 # [ 81.103259] nix-grpc-daemon[770]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50030 client # [ 81.156854] nix[1100]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 81.093238] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 client # [ 81.177946] nix[1100]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker1 # [ 81.100805] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 81.111253] nix-grpc-daemon[770]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 84.278533] systemd[1]: share1.service: Sent signal SIGINT to main process 1084 (nix) on client request. client # [ 84.284946] nix[1084]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 84.291741] nix[1084]: [mutex.cc : 451] RAW: Unlock 0xf55357dfc788 @ 0xf5575c4359a4 0xf5575d50d40c 0xf5577c9a57bc 0xf5577c9a7414 0xf5577c99e1d4 0xf5577b9e4f7c 0xf5577b772d80 client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 84.324884] systemd-coredump[1115]: Process 1084 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing... client # [ 84.349358] systemd[1]: Created slice Slice /system/systemd-coredump. client # [ 84.358135] systemd[1]: Started Process Core Dump (PID 1115/UID 0). client # [ 86.764889] systemd-coredump[1120]: Cannot store coredump of 1084 (nix): No space left on device client # [ 86.889519] systemd-coredump[1120]: Process 1084 (nix) of user 0 terminated abnormally without generating a coredump. client # [ 86.906537] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Deactivated successfully. client # [ 86.908007] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Consumed 2.031s CPU time over 2.542s wall clock time, 968.8M memory peak, 926.3M written to disk. client # [ 86.916415] systemd[1]: share1.service: Main process exited, code=dumped, status=11/SEGV client # [ 86.919216] systemd[1]: share1.service: Failed with result 'core-dump'. client # [ 86.920707] systemd[1]: share1.service: Consumed 368ms CPU time over 7.130s wall clock time, 41.5M memory peak, 4K written to disk, 6.1K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 3.26 seconds) client: waiting for success: ! systemctl is-active share2 worker1 # [ 87.906791] nix-daemon[849]: reaped child process 1101, status = succeeded worker1 # [ 88.033450] nix-daemon[849]: accepted connection from pid 1137, user nix-grpc-daemon (trusted) worker1 # [ 88.053829] nix-daemon[849]: reaped child process 1140, status = succeeded lb # [ 88.072759] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 88.072424] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 88.074091] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.688Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 88.106497] envoy[897]: conn peer=[2001:db8:1::1]:45306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=975 ms=6903 lb # [ 88.120549] envoy[897]: conn peer=[2001:db8:1::1]:45314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=787 ms=6910 lb # [ 88.135281] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 88.147009] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 88.141962] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.759Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.156429] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 88.163364] niks3-server[1007]: 2026/09/22 14:52:32 INFO Signed narinfos id=20 count=1 lb # [ 88.167198] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls worker1 # [ 88.164448] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.783Z level=INFO msg="Upload complete. (309ms)" lb # [ 88.177535] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 88.183019] niks3-server[1007]: 2026/09/22 14:52:32 INFO Completed upload id=20 lb # [ 88.187206] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker1 # [ 88.177161] nix-grpc-daemon[766]: ts=2026-09-22T14:52:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 88.198683] nix-grpc-daemon[766]: ts=2026-09-22T14:52:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 88.206202] nix-daemon[849]: reaped child process 1099, status = succeeded client # [ 88.302871] systemd[1]: share2.service: Deactivated successfully. client # [ 88.308003] systemd[1]: share2.service: Consumed 200ms CPU time over 7.410s wall clock time, 35.3M memory peak, 4K written to disk, 6.5K incoming IP traffic, 5.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 9.26 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 89.121464] systemd[1]: Started Nix Daemon. client # [ 89.236768] nix-daemon[1166]: accepted connection from pid 1163, user root (trusted) worker2 # [ 89.359691] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 89.315463] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 89.329041] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 89.381715] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 89.335332] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 89.397265] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 89.412851] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 89.427220] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 89.590654] nix-daemon[839]: accepted connection from pid 1132, user nix-grpc-daemon (trusted) worker2 # [ 89.617442] nix-daemon[839]: reaped child process 1135, status = succeeded worker2 # [ 89.712118] nix-daemon[839]: accepted connection from pid 1137, user nix-grpc-daemon (trusted) worker2 # [ 89.729429] nix-daemon[839]: reaped child process 1140, status = succeeded lb # [ 89.699785] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 89.747689] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.347Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 89.749389] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.347Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 89.735264] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst worker2 # [ 89.785019] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.385Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.749095] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 89.759620] niks3-server[1007]: 2026/09/22 14:52:34 INFO Signed narinfos id=21 count=1 lb # [ 89.768140] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls worker2 # [ 89.809836] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.410Z level=INFO msg="Upload complete. (364ms)" lb # [ 89.774384] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete worker2 # [ 89.816237] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 89.779880] niks3-server[1007]: 2026/09/22 14:52:34 INFO Completed upload id=21 lb # [ 89.786520] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker2 # [ 89.830979] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 89.837836] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 89.842816] nix-daemon[839]: reaped child process 1130, status = succeeded worker2 # [ 89.853808] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 91.119713] nix-daemon[839]: reaped child process 1144, status = succeeded worker2 # [ 91.231070] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker2 # [ 91.247892] nix-daemon[839]: reaped child process 1163, status = succeeded lb # [ 91.219061] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 91.265409] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 91.270288] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.868Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 91.248631] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 91.254386] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 91.261820] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker2 # [ 91.302340] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.902Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.267024] niks3-server[1007]: 2026/09/22 14:52:35 INFO Signed narinfos id=22 count=1 lb # [ 91.270499] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 91.277104] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete worker2 # [ 91.317933] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.918Z level=INFO msg="Upload complete. (251ms)" lb # [ 91.283015] niks3-server[1007]: 2026/09/22 14:52:35 INFO Completed upload id=22 lb # [ 91.285149] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker2 # [ 91.328883] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker1 # [ 91.285534] nix-grpc-daemon[766]: ts=2026-09-22T14:52:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 91.342483] nix-daemon[839]: reaped child process 1142, status = succeeded worker1 # [ 91.297324] nix-grpc-daemon[766]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 91.383973] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 91.391634] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 91.579982] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 91.634949] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 91.645417] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.245Z level=INFO msg="All 1 paths already cached" worker1 # [ 91.614815] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 91.621946] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 91.674670] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 91.690718] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 91.705353] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.305Z level=INFO msg="All 1 paths already cached" worker2 # [ 91.717196] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 91.729140] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 91.882509] nix-daemon[839]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 91.905177] nix-daemon[839]: reaped child process 1170, status = succeeded worker2 # [ 92.009761] nix-daemon[839]: accepted connection from pid 1172, user nix-grpc-daemon (trusted) worker2 # [ 92.030602] nix-daemon[839]: reaped child process 1175, status = succeeded lb # [ 91.992429] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 92.038379] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.638Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 92.039952] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.638Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 92.018980] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst lb # [ 92.035568] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker2 # [ 92.078368] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.678Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.049140] niks3-server[1007]: 2026/09/22 14:52:36 INFO Signed narinfos id=23 count=1 lb # [ 92.057124] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 92.069002] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete worker2 # [ 92.109779] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.710Z level=INFO msg="Upload complete. (366ms)" lb # [ 92.075837] niks3-server[1007]: 2026/09/22 14:52:36 INFO Completed upload id=23 lb # [ 92.079229] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker2 # [ 92.120249] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 92.129986] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 92.137530] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 92.142868] nix-daemon[839]: reaped child process 1165, status = succeeded worker2 # [ 92.156759] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 93.636345] nix-daemon[839]: reaped child process 1179, status = succeeded worker2 # [ 93.739055] nix-daemon[839]: accepted connection from pid 1195, user nix-grpc-daemon (trusted) worker2 # [ 93.758712] nix-daemon[839]: reaped child process 1198, status = succeeded lb # [ 93.731058] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 93.776438] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.375Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 93.778171] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.375Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 93.773578] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 93.787270] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst worker2 # [ 93.837065] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.437Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.801922] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 93.810783] niks3-server[1007]: 2026/09/22 14:52:38 INFO Signed narinfos id=24 count=1 lb # [ 93.817087] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls worker2 # [ 93.863631] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.463Z level=INFO msg="Upload complete. (296ms)" lb # [ 93.827468] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 93.834449] niks3-server[1007]: 2026/09/22 14:52:38 INFO Completed upload id=24 lb # [ 93.838286] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker2 # [ 93.877223] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 93.836710] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 93.896246] nix-daemon[839]: reaped child process 1177, status = succeeded worker1 # [ 93.849556] nix-grpc-daemon[766]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker1 # [ 93.890272] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 93.897500] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 94.238772] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 94.194405] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 94.205938] nix-grpc-daemon[901]: time=2026-09-22T14:52:38.823Z level=INFO msg="All 2 paths already cached" worker1 # [ 94.226178] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 94.232674] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 94.322005] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 94.339692] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 94.354400] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.954Z level=INFO msg="All 2 paths already cached" worker2 # [ 94.364902] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 94.379955] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 94.545991] nix-daemon[839]: accepted connection from pid 1202, user nix-grpc-daemon (trusted) worker2 # [ 94.570881] nix-daemon[839]: reaped child process 1205, status = succeeded worker2 # [ 94.669446] nix-daemon[839]: accepted connection from pid 1207, user nix-grpc-daemon (trusted) worker2 # [ 94.687054] nix-daemon[839]: reaped child process 1210, status = succeeded lb # [ 94.656636] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.702939] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 94.704522] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.303Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 94.684169] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst worker2 # [ 94.735537] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.335Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.698987] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 94.712333] niks3-server[1007]: 2026/09/22 14:52:39 INFO Signed narinfos id=25 count=1 lb # [ 94.715586] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls worker2 # [ 94.768690] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.368Z level=INFO msg="Upload complete. (370ms)" lb # [ 94.733849] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 94.742605] niks3-server[1007]: 2026/09/22 14:52:39 INFO Completed upload id=25 worker2 # [ 94.779015] nix-grpc-daemon[770]: ts=2026-09-22T14:52:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 94.745340] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker2 # [ 94.790227] nix-grpc-daemon[770]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 94.795865] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 94.800631] nix-daemon[839]: reaped child process 1200, status = succeeded worker2 # [ 94.813308] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 95.636736] nix-daemon[839]: reaped child process 1214, status = succeeded worker2 # [ 95.741983] nix-daemon[839]: accepted connection from pid 1230, user nix-grpc-daemon (trusted) worker2 # [ 95.762047] nix-daemon[839]: reaped child process 1233, status = succeeded lb # [ 95.729921] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.778931] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.378Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 95.780594] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.378Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 95.775571] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 95.788868] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 95.800302] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker2 # [ 95.842448] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.442Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.809969] niks3-server[1007]: 2026/09/22 14:52:40 INFO Signed narinfos id=26 count=1 lb # [ 95.815558] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 95.822752] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 95.831170] niks3-server[1007]: 2026/09/22 14:52:40 INFO Completed upload id=26 worker2 # [ 95.867620] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.467Z level=INFO msg="Upload complete. (245ms)" lb # [ 95.835692] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker2 # [ 95.881110] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 95.837953] nix-grpc-daemon[766]: ts=2026-09-22T14:52:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker2 # [ 95.898676] nix-daemon[839]: reaped child process 1212, status = succeeded worker1 # [ 95.850562] nix-grpc-daemon[766]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 95.940487] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 95.947478] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=232 client # [ 96.095190] nix-daemon[1166]: reaped child process 1168, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.13 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.24 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 97.003249] nix-daemon[1166]: accepted connection from pid 1248, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 97.129407] nix-grpc-daemon[770]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 97.137491] nix-grpc-daemon[770]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 97.102530] nix-grpc-daemon[766]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 97.116227] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 97.287365] nix-daemon[849]: accepted connection from pid 1158, user nix-grpc-daemon (trusted) worker1 # [ 97.308298] nix-daemon[849]: reaped child process 1161, status = succeeded lb # [ 97.329803] niks3-server[1007]: 2026/09/22 14:52:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.325018] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 97.326614] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.943Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 97.355132] niks3-server[1007]: 2026/09/22 14:52:41 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 97.359078] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.977Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.371997] niks3-server[1007]: 2026/09/22 14:52:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 97.382275] niks3-server[1007]: 2026/09/22 14:52:41 INFO Signed narinfos id=27 count=1 lb # [ 97.390229] niks3-server[1007]: 2026/09/22 14:52:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 97.403127] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 97.410060] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=27 worker1 # [ 97.397016] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.015Z level=INFO msg="Upload complete. (265ms)" lb # [ 97.416105] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 97.408388] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 97.425811] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 97.438061] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 97.444475] nix-daemon[849]: reaped child process 1156, status = succeeded worker1 # [ 97.471576] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 97.487095] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 97.505961] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 97.520629] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 97.672090] nix-daemon[849]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker1 # [ 97.700498] nix-daemon[849]: reaped child process 1170, status = succeeded worker1 # [ 97.791858] nix-daemon[849]: accepted connection from pid 1172, user nix-grpc-daemon (trusted) worker1 # [ 97.810778] nix-daemon[849]: reaped child process 1175, status = succeeded lb # [ 97.828618] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 97.837860] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.833804] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 97.835285] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker1 # [ 97.837129] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 97.873206] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 97.888982] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 97.902062] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls worker1 # [ 97.899072] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.516Z level=INFO msg="Uploading 3 narinfos" lb # [ 97.910995] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 97.915768] niks3-server[1007]: 2026/09/22 14:52:42 INFO Signed narinfos id=28 count=1 lb # [ 97.920320] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 97.922920] niks3-server[1007]: 2026/09/22 14:52:42 INFO Signed narinfos id=29 count=2 lb # [ 97.931581] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 97.944632] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 97.950775] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 97.960712] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=28 lb # [ 97.962109] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 97.965332] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 97.968257] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=29 worker1 # [ 97.955316] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.573Z level=INFO msg="Upload complete. (416ms)" lb # [ 97.970693] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker1 # [ 97.966395] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=2 nar_bytes_in=984 worker1 # [ 97.984996] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 97.993725] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 97.999996] nix-daemon[849]: reaped child process 1165, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 98.018433] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) lb # [ 98.109572] envoy[897]: conn peer=[2001:db8:1::1]:58138 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1599 ms=7169 lb # [ 98.114514] envoy[897]: conn peer=[2001:db8:1::1]:58150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1968 ms=2050 lb # [ 98.119583] envoy[897]: conn peer=[2001:db8:1::1]:57196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=27 lb # [ 98.124336] envoy[897]: conn peer=[2001:db8:1::1]:57208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3899 tx=2178 ms=2375 lb # [ 98.129090] envoy[897]: conn peer=[2001:db8:1::1]:57210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=71 lb # [ 98.133734] envoy[897]: conn peer=[2001:db8:1::1]:57214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2413 ms=1768 lb # [ 98.139186] envoy[897]: conn peer=[2001:db8:1::1]:57228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=68 client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 99.248717] nix-daemon[849]: reaped child process 1179, status = succeeded worker1 # [ 99.349474] nix-daemon[849]: accepted connection from pid 1195, user nix-grpc-daemon (trusted) worker1 # [ 99.372440] nix-daemon[849]: reaped child process 1198, status = succeeded lb # [ 99.384717] niks3-server[1007]: 2026/09/22 14:52:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 99.382508] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.000Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 99.421508] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 99.419852] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.038Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.434432] niks3-server[1007]: 2026/09/22 14:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 99.446721] niks3-server[1007]: 2026/09/22 14:52:44 INFO Signed narinfos id=30 count=1 lb # [ 99.456634] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls worker1 # [ 99.453102] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.071Z level=INFO msg="Upload complete. (221ms)" lb # [ 99.468243] niks3-server[1007]: 2026/09/22 14:52:44 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 99.474294] niks3-server[1007]: 2026/09/22 14:52:44 INFO Completed upload id=30 lb # [ 99.477719] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker1 # [ 99.465833] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 client # copying 1 paths... worker1 # [ 99.482232] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 99.486930] nix-daemon[849]: reaped child process 1177, status = succeeded client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 99.566931] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 99.581040] nix-grpc-daemon[770]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... worker2 # [ 99.599417] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=120 client # [ 99.739730] nix-daemon[1166]: reaped child process 1251, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 99.887145] nix-daemon[1166]: accepted connection from pid 1270, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 100.015544] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:50030 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 99.970191] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 99.981344] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.599Z level=INFO msg="All 1 paths already cached" worker1 # [ 100.001367] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 100.008687] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 100.061592] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 100.078616] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50030 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 100.101922] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 100.117519] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 100.141458] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5) worker2 # [ 100.180555] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 2/5) worker2 # [ 100.218824] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 213 ms (attempt 3/5) worker2 # [ 100.433957] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 4/5) worker2 # [ 100.523044] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 100.529501] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 100.537132] nix-grpc-daemon[770]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 100.719058] nix-daemon[839]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 100.740362] nix-daemon[839]: reaped child process 1249, status = succeeded worker2 # [ 100.844851] nix-daemon[839]: accepted connection from pid 1251, user nix-grpc-daemon (trusted) worker2 # [ 100.862535] nix-daemon[839]: reaped child process 1254, status = succeeded lb # [ 100.831068] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.878086] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 100.879528] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.477Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 100.860476] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 100.876499] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker2 # [ 100.919959] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.520Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.888502] niks3-server[1007]: 2026/09/22 14:52:45 INFO Signed narinfos id=31 count=1 lb # [ 100.891770] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 100.906671] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete worker2 # [ 100.948717] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.548Z level=INFO msg="Upload complete. (390ms)" lb # [ 100.915856] niks3-server[1007]: 2026/09/22 14:52:45 INFO Completed upload id=31 lb # [ 100.918074] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker2 # [ 100.962453] nix-grpc-daemon[770]: ts=2026-09-22T14:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 100.976856] nix-grpc-daemon[770]: ts=2026-09-22T14:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 100.983179] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 100.990740] nix-daemon[839]: reaped child process 1239, status = succeeded worker2 # [ 101.006555] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker2 # [ 101.023838] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 102.728618] nix-daemon[839]: reaped child process 1259, status = succeeded worker2 # [ 102.809868] nix-daemon[839]: accepted connection from pid 1275, user nix-grpc-daemon (trusted) worker2 # [ 102.829983] nix-daemon[839]: reaped child process 1278, status = succeeded lb # [ 102.799139] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 102.848679] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.445Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 102.836155] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker2 # [ 102.881055] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.481Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.849623] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 102.858230] niks3-server[1007]: 2026/09/22 14:52:47 INFO Signed narinfos id=32 count=1 lb # [ 102.867003] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 102.910801] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.511Z level=INFO msg="Upload complete. (272ms)" lb # [ 102.877026] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 102.882826] niks3-server[1007]: 2026/09/22 14:52:47 INFO Completed upload id=32 lb # [ 102.886892] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker2 # [ 102.921132] nix-grpc-daemon[770]: ts=2026-09-22T14:52:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker2 # [ 102.943549] nix-grpc-daemon[770]: ts=2026-09-22T14:52:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50030 worker2 # [ 102.949101] nix-daemon[839]: reaped child process 1256, status = succeeded client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 102.931455] nix-grpc-daemon[766]: ts=2026-09-22T14:52:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 102.945610] nix-grpc-daemon[766]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... worker1 # [ 102.963968] nix-grpc-daemon[766]: ts=2026-09-22T14:52:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=120 client # [ 103.170615] nix-daemon[1166]: reaped child process 1273, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.43 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 103.341311] nix-daemon[1166]: accepted connection from pid 1291, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 103.415578] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:46148 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 103.424412] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 103.433041] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.049Z level=INFO msg="All 1 paths already cached" worker1 # [ 103.445293] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 103.453126] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 103.462269] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 103.473091] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 103.487131] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.105Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 103.497875] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 103.512366] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 103.662076] nix-daemon[849]: accepted connection from pid 1207, user nix-grpc-daemon (trusted) worker1 # [ 103.680333] nix-daemon[849]: reaped child process 1210, status = succeeded worker1 # [ 103.784420] nix-daemon[849]: accepted connection from pid 1212, user nix-grpc-daemon (trusted) worker1 # [ 103.801257] nix-daemon[849]: reaped child process 1215, status = succeeded lb # [ 103.821859] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 103.816630] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.435Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 103.818125] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.435Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 103.849601] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst lb # [ 103.868516] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker1 # [ 103.861580] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.480Z level=INFO msg="Uploading 1 narinfos" lb # [ 103.883675] niks3-server[1007]: 2026/09/22 14:52:48 INFO Signed narinfos id=33 count=1 lb # [ 103.891144] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 103.902668] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker1 # [ 103.897058] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.515Z level=INFO msg="Upload complete. (367ms)" lb # [ 103.910973] niks3-server[1007]: 2026/09/22 14:52:48 INFO Completed upload id=33 lb # [ 103.914081] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker1 # [ 103.908323] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=568 worker1 # [ 103.924502] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 103.931543] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 103.937478] nix-daemon[849]: reaped child process 1205, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker1 # [ 103.951027] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 104.825013] nix-daemon[849]: reaped child process 1219, status = succeeded worker1 # [ 104.936938] nix-daemon[849]: accepted connection from pid 1235, user nix-grpc-daemon (trusted) worker1 # [ 104.956932] nix-daemon[849]: reaped child process 1238, status = succeeded lb # [ 104.978358] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 104.974919] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.592Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 105.016199] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker1 # [ 105.011014] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.629Z level=INFO msg="Uploading 1 narinfos" lb # [ 105.028382] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 105.038011] niks3-server[1007]: 2026/09/22 14:52:49 INFO Signed narinfos id=34 count=1 worker1 # [ 105.034939] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.653Z level=INFO msg="Upload complete. (270ms)" lb # [ 105.048474] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 105.058110] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 105.063374] niks3-server[1007]: 2026/09/22 14:52:49 INFO Completed upload id=34 client # copying 1 paths... worker1 # [ 105.049217] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 lb # [ 105.065960] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 105.109592] nix-grpc-daemon[770]: ts=2026-09-22T14:52:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:50030 worker1 # [ 105.067625] nix-daemon[849]: reaped child process 1217, status = succeeded worker2 # [ 105.126549] nix-grpc-daemon[770]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 105.124001] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 105.130846] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=120 client # [ 105.334863] nix-daemon[1166]: reaped child process 1294, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.16 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.12 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds) worker2 # [ 106.314553] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 106.507686] nix-grpc-daemon[770]: ts=2026-09-22T14:52:51Z level=info event=signal_exit worker2 # [ 107.983099] nix-grpc-daemon[770]: ts=2026-09-22T14:52:52Z level=info event=scheduler_restarting worker1 # [ 107.936577] nix-grpc-daemon[766]: ts=2026-09-22T14:52:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:54534 worker1 # [ 107.941744] nix-grpc-daemon[766]: ts=2026-09-22T14:52:52Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 108.023820] nix-grpc-daemon[770]: ts=2026-09-22T14:52:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 108.057364] nix-daemon[839]: reaped child process 841, status = succeeded lb # [ 108.114502] envoy[897]: conn peer=[2001:db8:1::1]:57234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4254 tx=2069 ms=2528 lb # [ 108.119974] envoy[897]: conn peer=[2001:db8:1::1]:57250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=96 lb # [ 108.126616] envoy[897]: conn peer=[2001:db8:1::1]:57264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2127 ms=3058 lb # [ 108.131662] envoy[897]: conn peer=[2001:db8:1::1]:34342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=101 lb # [ 108.137311] envoy[897]: conn peer=[2001:db8:1::1]:34354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2124 ms=1772 lb # [ 108.139404] envoy[897]: conn peer=[2001:db8:1::1]:34366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=67 lb # [ 108.141555] envoy[897]: conn peer=[2001:db8:1::4]:33140 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=76789 tx=4223 ms=89473 worker2 # [ 108.709701] nix-grpc-daemon[770]: E0000 00:00:1790088773.309486 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 108.716614] nix-grpc-daemon[770]: E0000 00:00:1790088773.309886 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 109.861533] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 109.871730] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 109.877237] systemd[1]: nix-grpc-daemon.service: Consumed 4.535s CPU time over 1min 31.597s wall clock time, 65.2M memory peak, 104.3K incoming IP traffic, 250.9K outgoing IP traffic. worker2 # [ 109.894688] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 109.897966] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 109.899753] systemd[1]: nix-grpc-daemon.socket: Consumed 628us CPU time over 1min 39.792s wall clock time, 256K memory peak, 105K incoming IP traffic, 87.8K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 110.157342] nix-grpc-daemon[766]: ts=2026-09-22T14:52:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46148 worker1 # [ 110.166263] nix-grpc-daemon[766]: ts=2026-09-22T14:52:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46148 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 110.342855] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 110.367669] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 110.858257] systemd[1]: Starting scheduler-token.service... worker2 # [ 110.950962] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 110.961326] systemd[1]: Finished scheduler-token.service. worker2 # [ 110.966481] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 110.974844] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 111.110450] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 111.115782] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 111.138174] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 111.164317] nix-grpc-daemon[1303]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 111.172805] nix-grpc-daemon[1303]: E0000 00:00:1790088775.720049 1303 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.175656] nix-grpc-daemon[1303]: E0000 00:00:1790088775.722323 1303 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.178263] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 111.180051] nix-grpc-daemon[1303]: E0000 00:00:1790088775.733665 1344 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.185362] nix-grpc-daemon[1303]: E0000 00:00:1790088775.754647 1344 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.189686] nix-grpc-daemon[1303]: E0000 00:00:1790088775.754806 1344 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.192289] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 111.193094] nix-grpc-daemon[1303]: E0000 00:00:1790088775.772148 1344 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.195332] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 111.142762] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:54534 worker1 # [ 111.143225] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 111.361714] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=signal_exit lb # [ 111.942853] niks3-server[1007]: 2026/09/22 14:52:56 INFO lead: released remote=192.168.1.3:48398 worker1 # [ 112.921703] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting worker2 # [ 112.973902] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 112.934032] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:54534 worker1 # [ 112.939105] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 112.943592] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 112.948690] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:54534 worker1 # [ 112.953480] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 112.981305] nix-daemon[849]: reaped child process 851, status = succeeded worker2 # [ 113.173718] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.179133] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 113.155067] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 113.380264] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.385368] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 113.586464] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.591624] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 113.551621] nix-grpc-daemon[766]: E0000 00:00:1790088778.170219 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 113.561358] nix-grpc-daemon[766]: E0000 00:00:1790088778.179939 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 113.792791] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 113.798187] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 113.752470] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 113.759552] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 113.765811] systemd[1]: nix-grpc-daemon.service: Consumed 3.833s CPU time over 1min 35.655s wall clock time, 64.1M memory peak, 97.7K incoming IP traffic, 193.6K outgoing IP traffic. worker1 # [ 113.781354] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 113.783449] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 113.788895] systemd[1]: nix-grpc-daemon.socket: Consumed 581us CPU time over 1min 43.616s wall clock time, 256K memory peak, 122.6K incoming IP traffic, 105.5K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 114.000603] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.005225] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.142593] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 114.160782] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 114.189610] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5) worker2 # [ 114.206317] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.211526] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.218750] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 2/5) worker2 # [ 114.267098] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5) worker2 # [ 114.412889] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.418045] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.598289] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 341 ms (attempt 4/5) worker2 # [ 114.619143] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.624430] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.825606] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.830793] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 114.941994] nix-grpc-daemon[1303]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 114.948531] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 114.981720] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 114.999799] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 115.038874] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=niks3_push_started pid=1363 worker2 # [ 115.058301] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 115.062851] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.196338] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.794Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 115.246809] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 115.248953] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.307356] nix-daemon[839]: accepted connection from pid 1376, user nix-grpc-daemon (trusted) worker2 # [ 115.326345] nix-daemon[839]: reaped child process 1379, status = succeeded lb # [ 115.296665] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 115.307256] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 115.355177] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 115.358262] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 115.360614] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 115.356932] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 115.371442] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 115.382239] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 115.389829] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 115.399443] niks3-server[1007]: 2026/09/22 14:53:00 INFO Signed narinfos id=35 count=1 worker2 # [ 115.436136] nix-grpc-daemon[1363]: time=2026-09-22T14:53:00.035Z level=INFO msg="Uploading 3 narinfos" lb # [ 115.405860] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign worker2 # [ 115.450348] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:50051 lb # [ 115.416237] niks3-server[1007]: 2026/09/22 14:53:00 INFO Signed narinfos id=36 count=2 lb # [ 115.418691] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls worker2 # [ 115.455786] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 115.440553] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 115.449103] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 115.452234] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 115.455506] niks3-server[1007]: 2026/09/22 14:53:00 INFO Completed upload id=35 lb # [ 115.458812] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete worker2 # [ 115.495761] nix-grpc-daemon[1363]: time=2026-09-22T14:53:00.095Z level=INFO msg="Upload complete. (300ms)" lb # [ 115.461980] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 115.464537] niks3-server[1007]: 2026/09/22 14:53:00 INFO Completed upload id=36 worker2 # [ 115.508338] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 115.524870] nix-daemon[839]: reaped child process 1361, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.63 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 115.603162] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 115.656937] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.614004] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 115.662195] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.762932] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 115.768337] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 115.770238] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 115.804651] niks3-server[1007]: 2026/09/22 14:53:00 INFO lead: acquired remote=192.168.1.3:39180 lb: waiting until exactly no endpoints of aarch64-linux are unhealthy worker1 # [ 115.790798] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 115.811914] nix-grpc-daemon[1261]: E0000 00:00:1790088780.393006 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 115.863307] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.835879] nix-grpc-daemon[1261]: E0000 00:00:1790088780.395688 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.838878] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 115.840842] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 115.841657] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_take_over worker1 # [ 115.842762] nix-grpc-daemon[1261]: E0000 00:00:1790088780.411581 1303 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 115.845160] nix-grpc-daemon[1261]: E0000 00:00:1790088780.435052 1303 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.847457] nix-grpc-daemon[1261]: E0000 00:00:1790088780.435337 1303 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 115.852811] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 115.856448] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:58970 worker1 # [ 115.858407] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 115.860132] nix-grpc-daemon[1261]: E0000 00:00:1790088780.453822 1303 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.862414] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.863682] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:58970 worker1 # [ 115.865308] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.61 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 115.999372] niks3-server[1007]: 2026/09/22 14:53:00 INFO Shutdown signal received, draining in-flight requests timeout=10s worker2 # [ 116.036613] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_lock_lost status=200 worker1 # [ 115.986979] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_lock_lost status=200 lb # [ 116.018790] niks3-server[1007]: 2026/09/22 14:53:00 INFO lead: released remote=192.168.1.3:39180 lb # [ 116.024903] systemd[1]: Stopping niks3 server... lb # [ 116.027802] systemd[1]: niks3.service: Deactivated successfully. lb # [ 116.029975] systemd[1]: Stopped niks3 server. lb # [ 116.030680] systemd[1]: niks3.service: Consumed 650ms CPU time over 1min 28.340s wall clock time, 27.7M memory peak, 100.6K incoming IP traffic, 142.9K outgoing IP traffic. lb # [ 116.039680] systemd[1]: Starting niks3 server... lb # [ 116.104813] niks3-server[1141]: 2026/09/22 14:53:00 goose: no migrations to run. current version: 20260920000000 lb # [ 116.113156] niks3-server[1141]: 2026/09/22 14:53:00 OK 1_commit_pending_closure.sql (6.34ms) lb # [ 116.116900] niks3-server[1141]: 2026/09/22 14:53:00 OK 2_object_stats_trigger.sql (3.31ms) lb # [ 116.118889] niks3-server[1141]: 2026/09/22 14:53:00 goose: up to current file version: 2 lb # [ 116.120679] niks3-server[1141]: 2026/09/22 14:53:00 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 116.132002] niks3-server[1141]: 2026/09/22 14:53:00 INFO Read proxy enabled — serving cache objects from S3 lb # [ 116.133434] niks3-server[1141]: 2026/09/22 14:53:00 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 116.138722] niks3-server[1141]: 2026/09/22 14:53:00 INFO systemd watchdog enabled interval=15s lb # [ 116.141991] niks3-server[1141]: 2026/09/22 14:53:00 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 116.143496] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 116.657565] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53762 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 116.667836] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker1 # [ 116.649546] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:01Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker1 # [ 116.649981] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:01Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 116.745754] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 116.761573] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 116.777899] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 116.793064] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 116.965034] nix-daemon[839]: accepted connection from pid 1386, user nix-grpc-daemon (trusted) worker2 # [ 116.984539] nix-daemon[839]: reaped child process 1389, status = succeeded lb # [ 117.005985] niks3-server[1141]: 2026/09/22 14:53:01 INFO lead: acquired remote=192.168.1.3:39196 worker2 # [ 117.090231] nix-daemon[839]: accepted connection from pid 1391, user nix-grpc-daemon (trusted) worker2 # [ 117.106194] nix-daemon[839]: reaped child process 1396, status = succeeded lb # [ 117.076889] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 117.144214] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.743Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 117.146228] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.743Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 117.143952] niks3-server[1141]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 117.155430] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign worker2 # [ 117.198364] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.798Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.163882] niks3-server[1141]: 2026/09/22 14:53:01 INFO Signed narinfos id=37 count=1 lb # [ 117.170170] niks3-server[1141]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 117.175104] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 117.186368] niks3-server[1141]: 2026/09/22 14:53:01 INFO Completed upload id=37 worker2 # [ 117.222686] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.822Z level=INFO msg="Upload complete. (412ms)" worker2 # [ 117.232340] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 117.252473] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 117.264281] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 117.267455] nix-daemon[839]: reaped child process 1384, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 117.281644] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart lb # [ 118.115531] envoy[897]: conn peer=[2001:db8:1::1]:34376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=27 lb # [ 118.120690] envoy[897]: conn peer=[2001:db8:1::3]:44134 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14117 tx=3806 ms=93836 lb # [ 118.125875] envoy[897]: conn peer=[2001:db8:1::1]:50548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=281 ms=1381 worker2 # [ 118.186212] nix-daemon[839]: reaped child process 1400, status = succeeded worker2 # [ 118.254864] nix-daemon[839]: accepted connection from pid 1416, user nix-grpc-daemon (trusted) worker2 # [ 118.278013] nix-daemon[839]: reaped child process 1419, status = succeeded lb # [ 118.240652] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 118.291395] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 118.295616] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.893Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 118.295645] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 118.310281] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 118.357069] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.956Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.319193] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 118.324564] niks3-server[1141]: 2026/09/22 14:53:02 INFO Signed narinfos id=38 count=1 lb # [ 118.328619] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 118.337647] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 118.348288] niks3-server[1141]: 2026/09/22 14:53:02 INFO Completed upload id=38 worker2 # [ 118.385199] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.985Z level=INFO msg="Upload complete. (293ms)" lb # [ 118.352415] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 118.397166] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker1 # [ 118.357409] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:02Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker1 # [ 118.357709] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:02Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 118.419880] nix-daemon[839]: reaped child process 1398, status = succeeded worker2 # [ 118.453215] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 118.470173] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 118.485571] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.085Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 118.496727] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 118.510929] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 118.663671] nix-daemon[839]: accepted connection from pid 1423, user nix-grpc-daemon (trusted) worker2 # [ 118.682680] nix-daemon[839]: reaped child process 1426, status = succeeded worker2 # [ 118.789371] nix-daemon[839]: accepted connection from pid 1428, user nix-grpc-daemon (trusted) worker2 # [ 118.807787] nix-daemon[839]: reaped child process 1431, status = succeeded lb # [ 118.774650] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 118.822033] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.422Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 118.823540] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.422Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 118.806211] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 118.822741] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 118.866650] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.466Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.834287] niks3-server[1141]: 2026/09/22 14:53:03 INFO Signed narinfos id=39 count=1 lb # [ 118.843295] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 118.858326] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 118.866974] niks3-server[1141]: 2026/09/22 14:53:03 INFO Completed upload id=39 worker2 # [ 118.903920] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.504Z level=INFO msg="Upload complete. (375ms)" lb # [ 118.872181] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 118.916159] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 118.932680] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 118.938787] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 118.943613] nix-daemon[839]: reaped child process 1421, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 118.957053] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 119.648826] nix-daemon[839]: reaped child process 1435, status = succeeded worker2 # [ 119.743818] nix-daemon[839]: accepted connection from pid 1451, user nix-grpc-daemon (trusted) worker2 # [ 119.762531] nix-daemon[839]: reaped child process 1454, status = succeeded lb # [ 119.731389] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 119.779036] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 119.781078] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.378Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 119.770552] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 119.782574] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 119.792391] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker2 # [ 119.835698] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.435Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.801176] niks3-server[1141]: 2026/09/22 14:53:04 INFO Signed narinfos id=40 count=1 lb # [ 119.803691] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 119.817676] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 119.825661] niks3-server[1141]: 2026/09/22 14:53:04 INFO Completed upload id=40 worker2 # [ 119.861898] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.462Z level=INFO msg="Upload complete. (294ms)" lb # [ 119.828902] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker1 # [ 119.831127] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:04Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker1 # [ 119.837585] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:04Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 119.873185] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 119.895615] nix-daemon[839]: reaped child process 1433, status = succeeded worker2 # [ 119.933290] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 119.950057] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 119.964991] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.565Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 119.976413] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 119.992239] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 120.148117] nix-daemon[839]: accepted connection from pid 1458, user nix-grpc-daemon (trusted) worker2 # [ 120.166274] nix-daemon[839]: reaped child process 1461, status = succeeded worker2 # [ 120.272166] nix-daemon[839]: accepted connection from pid 1463, user nix-grpc-daemon (trusted) worker2 # [ 120.289128] nix-daemon[839]: reaped child process 1466, status = succeeded lb # [ 120.257766] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 120.303643] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 120.305423] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.903Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 120.286397] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker2 # [ 120.345028] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.945Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.301498] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 120.314949] niks3-server[1141]: 2026/09/22 14:53:04 INFO Signed narinfos id=41 count=1 lb # [ 120.320966] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 120.331396] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker2 # [ 120.374363] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.974Z level=INFO msg="Upload complete. (367ms)" lb # [ 120.338255] niks3-server[1141]: 2026/09/22 14:53:04 INFO Completed upload id=41 lb # [ 120.342248] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 120.384322] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 120.398923] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 120.405522] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 120.410762] nix-daemon[839]: reaped child process 1456, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker2 # [ 120.421784] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 121.508645] nix-daemon[839]: reaped child process 1470, status = succeeded worker2 # [ 121.607047] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker2 # [ 121.625939] nix-daemon[839]: reaped child process 1489, status = succeeded lb # [ 121.596892] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 121.644448] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.243Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 121.646165] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.243Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 121.633577] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker2 # [ 121.687832] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.287Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.650500] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 121.662662] niks3-server[1141]: 2026/09/22 14:53:06 INFO Signed narinfos id=42 count=1 lb # [ 121.669619] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls worker2 # [ 121.723275] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.323Z level=INFO msg="Upload complete. (289ms)" lb # [ 121.682242] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 121.691462] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 121.696238] niks3-server[1141]: 2026/09/22 14:53:06 INFO Completed upload id=42 lb # [ 121.699275] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 121.737155] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 121.755566] nix-daemon[839]: reaped child process 1468, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.35 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 6.04 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1 # [ 122.155680] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 122.785560] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:07Z level=info event=signal_exit lb # [ 123.016952] niks3-server[1141]: 2026/09/22 14:53:07 INFO lead: released remote=192.168.1.3:39196 worker2 # [ 124.048878] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 124.002574] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting worker1 # [ 124.003046] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58970 worker1 # [ 124.003178] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 124.003291] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58970 worker1 # [ 124.003398] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 124.003506] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 124.248630] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.254010] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 124.275335] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 124.279889] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 124.283013] systemd[1]: nix-grpc-daemon.service: Consumed 249ms CPU time over 8.664s wall clock time, 15M memory peak, 10K incoming IP traffic, 12.4K outgoing IP traffic. worker1 # [ 124.291781] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 124.295745] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 124.298855] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 8.699s wall clock time, 256K memory peak, 17.8K incoming IP traffic, 16K outgoing IP traffic. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2 # [ 124.455331] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.460801] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.661925] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.667189] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.868381] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.873610] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.076477] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.080563] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.277437] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.282583] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.483708] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.495478] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.692516] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.697809] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.898909] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.904365] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.107376] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.111742] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.308482] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.313801] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.515040] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.520825] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.721906] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.727122] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 126.928348] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 126.933399] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 127.043780] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_take_over lb # [ 127.007784] niks3-server[1141]: 2026/09/22 14:53:11 INFO lead: acquired remote=192.168.1.4:57346 worker2 # [ 127.137051] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.141319] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 127.338059] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 127.364147] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 127.369282] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38230 worker2 # [ 127.374611] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.33 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 127.954680] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 127.964805] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 127.986644] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker2 # [ 127.993636] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 128.032957] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 128.049686] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker2 # [ 128.065998] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 128.081103] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) lb # [ 128.115579] envoy[897]: conn peer=[2001:db8:1::1]:50552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9684 tx=4448 ms=5098 lb # [ 128.120896] envoy[897]: conn peer=[2001:db8:1::3]:35636 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=488 tx=177 ms=8363 worker2 # [ 128.242466] nix-daemon[839]: accepted connection from pid 1522, user nix-grpc-daemon (trusted) worker2 # [ 128.269887] nix-daemon[839]: reaped child process 1525, status = succeeded worker2 # [ 128.367145] nix-daemon[839]: accepted connection from pid 1527, user nix-grpc-daemon (trusted) worker2 # [ 128.386102] nix-daemon[839]: reaped child process 1530, status = succeeded lb # [ 128.354779] niks3-server[1141]: 2026/09/22 14:53:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 128.402148] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 128.403640] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.001Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 128.384597] niks3-server[1141]: 2026/09/22 14:53:12 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 128.399832] niks3-server[1141]: 2026/09/22 14:53:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 128.420495] niks3-server[1141]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 128.442356] niks3-server[1141]: 2026/09/22 14:53:13 INFO Signed narinfos id=43 count=1 worker2 # [ 128.479502] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.079Z level=INFO msg="Uploading 1 narinfos" lb # [ 128.462424] niks3-server[1141]: 2026/09/22 14:53:13 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 128.472976] niks3-server[1141]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo lb # [ 128.488806] niks3-server[1141]: 2026/09/22 14:53:13 INFO Completed upload id=43 worker2 # [ 128.525821] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.126Z level=INFO msg="Upload complete. (428ms)" worker2 # [ 128.537626] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=520 worker2 # [ 128.552329] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 128.566052] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 128.572584] nix-daemon[839]: reaped child process 1520, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker2 # [ 128.590117] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 129.862800] nix-daemon[839]: reaped child process 1534, status = succeeded worker2 # [ 129.982615] nix-daemon[839]: accepted connection from pid 1550, user nix-grpc-daemon (trusted) worker2 # [ 130.004744] nix-daemon[839]: reaped child process 1553, status = succeeded lb # [ 129.967337] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 130.024505] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 130.026367] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.621Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 130.016357] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 130.029329] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker2 # [ 130.079128] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.679Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.042242] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 130.052491] niks3-server[1141]: 2026/09/22 14:53:14 INFO Signed narinfos id=44 count=1 lb # [ 130.055358] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 130.068421] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 130.078218] niks3-server[1141]: 2026/09/22 14:53:14 INFO Completed upload id=44 worker2 # [ 130.114707] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.714Z level=INFO msg="Upload complete. (310ms)" lb # [ 130.081266] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker2 # [ 130.137323] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker2 # [ 130.156379] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker2 # [ 130.163108] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 130.168584] nix-daemon[839]: reaped child process 1532, status = succeeded worker2 # [ 130.178257] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 130.195086] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 client # copying 1 paths to 'lb:50051'... worker2 # [ 130.209801] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.810Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker2 # [ 130.217976] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 130.232285] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 130.384181] nix-daemon[839]: accepted connection from pid 1559, user nix-grpc-daemon (trusted) worker2 # [ 130.411224] nix-daemon[839]: reaped child process 1562, status = succeeded worker2 # [ 130.509461] nix-daemon[839]: accepted connection from pid 1564, user nix-grpc-daemon (trusted) worker2 # [ 130.527031] nix-daemon[839]: reaped child process 1567, status = succeeded lb # [ 130.498349] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 130.545461] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 130.546980] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.145Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 130.530142] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker2 # [ 130.583325] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.183Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.546507] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 130.558265] niks3-server[1141]: 2026/09/22 14:53:15 INFO Signed narinfos id=45 count=1 lb # [ 130.565485] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls worker2 # [ 130.613301] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.213Z level=INFO msg="Upload complete. (366ms)" lb # [ 130.578304] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 130.586330] niks3-server[1141]: 2026/09/22 14:53:15 INFO Completed upload id=45 worker2 # [ 130.623575] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 130.590201] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker2 # [ 130.632882] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 130.643676] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 130.648221] nix-daemon[839]: reaped child process 1557, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker2 # [ 130.660232] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 131.489141] nix-daemon[839]: reaped child process 1571, status = succeeded worker2 # [ 131.594532] nix-daemon[839]: accepted connection from pid 1587, user nix-grpc-daemon (trusted) worker2 # [ 131.615692] nix-daemon[839]: reaped child process 1590, status = succeeded lb # [ 131.586093] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.632394] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.231Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 131.633878] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.231Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 131.637359] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst worker2 # [ 131.683579] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.283Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.654271] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 131.665500] niks3-server[1141]: 2026/09/22 14:53:16 INFO Signed narinfos id=46 count=1 lb # [ 131.670391] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 131.681635] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 131.689497] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 131.694675] niks3-server[1141]: 2026/09/22 14:53:16 INFO Completed upload id=46 worker2 # [ 131.725188] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.325Z level=INFO msg="Upload complete. (254ms)" lb # [ 131.699479] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker2 # [ 131.744872] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker2 # [ 131.764381] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker2 # [ 131.770061] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 131.775137] nix-daemon[839]: reaped child process 1569, status = succeeded worker2 # [ 131.793383] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 131.809633] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 131.826192] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.426Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker2 # [ 131.838387] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 131.852366] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 132.008896] nix-daemon[839]: accepted connection from pid 1596, user nix-grpc-daemon (trusted) worker2 # [ 132.028373] nix-daemon[839]: reaped child process 1599, status = succeeded worker2 # [ 132.138287] nix-daemon[839]: accepted connection from pid 1601, user nix-grpc-daemon (trusted) worker2 # [ 132.159265] nix-daemon[839]: reaped child process 1604, status = succeeded lb # [ 132.120665] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 132.166669] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 132.168316] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.766Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 132.149892] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker2 # [ 132.201494] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.801Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.165982] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 132.177523] niks3-server[1141]: 2026/09/22 14:53:16 INFO Signed narinfos id=47 count=1 lb # [ 132.184584] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 132.195991] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker2 # [ 132.239371] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.839Z level=INFO msg="Upload complete. (371ms)" lb # [ 132.203839] niks3-server[1141]: 2026/09/22 14:53:16 INFO Completed upload id=47 lb # [ 132.207179] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker2 # [ 132.248259] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 132.256287] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 132.267713] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 132.273125] nix-daemon[839]: reaped child process 1594, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker2 # [ 132.287140] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 133.194183] nix-daemon[839]: reaped child process 1608, status = succeeded worker2 # [ 133.266958] nix-daemon[839]: accepted connection from pid 1624, user nix-grpc-daemon (trusted) worker2 # [ 133.284947] nix-daemon[839]: reaped child process 1627, status = succeeded lb # [ 133.257866] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.303966] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 133.305611] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.903Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 133.300190] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst lb # [ 133.312471] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 133.324464] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign worker2 # [ 133.367125] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.967Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.336288] niks3-server[1141]: 2026/09/22 14:53:17 INFO Signed narinfos id=48 count=1 lb # [ 133.338811] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 133.350426] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 133.355448] niks3-server[1141]: 2026/09/22 14:53:17 INFO Completed upload id=48 worker2 # [ 133.392328] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.992Z level=INFO msg="Upload complete. (294ms)" lb # [ 133.358669] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker2 # [ 133.405053] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker2 # [ 133.425720] nix-daemon[839]: reaped child process 1606, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.73 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker1 # [ 133.663032] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 133.672715] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 133.820495] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 133.822405] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 133.826213] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 133.832076] nix-grpc-daemon[1344]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 133.833524] nix-grpc-daemon[1344]: E0000 00:00:1790088798.449262 1344 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 133.876085] nix-grpc-daemon[1344]: E0000 00:00:1790088798.455652 1344 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.878440] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 133.889585] nix-grpc-daemon[1344]: E0000 00:00:1790088798.469728 1386 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.892443] nix-grpc-daemon[1344]: E0000 00:00:1790088798.488989 1386 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.894864] nix-grpc-daemon[1344]: E0000 00:00:1790088798.489126 1386 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.900151] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 133.902774] nix-grpc-daemon[1344]: E0000 00:00:1790088798.507608 1386 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.905482] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.953392] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38230 worker2 # [ 133.959409] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.24 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 134.310829] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 004dacd07350486192513362c1a3877c client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker1 # [ 134.447736] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 134.462808] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) client # [ 134.559237] nix[1417]: don't know how to build these paths: client # [ 134.561406] nix[1417]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 134.543173] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker2 # [ 134.548644] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 134.589660] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 134.605796] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53762 client # [ 134.647920] nix[1417]: copying 1 paths to 'lb:50051'... worker2 # [ 134.621054] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 client # [ 134.650402] nix[1417]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker2 # [ 134.635786] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 134.794741] nix-daemon[839]: accepted connection from pid 1640, user nix-grpc-daemon (trusted) worker2 # [ 134.821692] nix-daemon[839]: reaped child process 1643, status = succeeded worker2 # [ 134.923199] nix-daemon[839]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) worker2 # [ 134.940530] nix-daemon[839]: reaped child process 1648, status = succeeded lb # [ 134.912809] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.965059] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.564Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 134.966608] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.564Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 134.951050] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst worker2 # [ 134.998816] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.599Z level=INFO msg="Uploading 1 narinfos" lb # [ 134.963280] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 134.970445] niks3-server[1141]: 2026/09/22 14:53:19 INFO Signed narinfos id=49 count=1 lb # [ 134.976444] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 134.984527] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 134.990605] niks3-server[1141]: 2026/09/22 14:53:19 INFO Completed upload id=49 worker2 # [ 135.025877] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.626Z level=INFO msg="Upload complete. (372ms)" lb # [ 134.992468] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker2 # [ 135.035849] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 135.044467] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 worker2 # [ 135.060329] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) worker2 # [ 135.063219] nix-daemon[839]: reaped child process 1638, status = succeeded client # [ 135.100360] nix[1417]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker2 # [ 135.076452] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 137.039749] systemd-journald[269]: Received client request to rotate journal, rotating. worker1 # [ 137.064934] systemd-journald[269]: Vacuuming done, freed 0B of archived journals from /var/log/journal/117159a78fbd43749c28f85305e28709. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 137.151081] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 137.153131] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:21Z level=info event=draining worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 lb # [ 138.115433] envoy[897]: conn peer=[2001:db8:1::1]:46672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4371 ms=5463 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 142.508789] nix-daemon[839]: reaped child process 1652, status = succeeded worker2 # [ 142.602838] nix-daemon[839]: accepted connection from pid 1693, user nix-grpc-daemon (trusted) worker2 # [ 142.621772] nix-daemon[839]: reaped child process 1696, status = succeeded lb # [ 142.592907] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 142.644073] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.241Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 142.645587] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.241Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 142.630144] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 142.648473] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst worker2 # [ 142.695209] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.295Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.659744] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 142.668505] niks3-server[1141]: 2026/09/22 14:53:27 INFO Signed narinfos id=50 count=1 lb # [ 142.673127] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 142.681973] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete worker2 # [ 142.723102] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.323Z level=INFO msg="Upload complete. (231ms)" lb # [ 142.686282] niks3-server[1141]: 2026/09/22 14:53:27 INFO Completed upload id=50 lb # [ 142.691262] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker2 # [ 142.740127] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 142.753012] nix-daemon[839]: reaped child process 1650, status = succeeded client # [ 142.798401] systemd[1]: rs.service: Deactivated successfully. client # [ 142.803715] systemd[1]: rs.service: Consumed 198ms CPU time over 8.488s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 143.155618] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:27Z level=info event=signal_exit lb # [ 144.017200] niks3-server[1141]: 2026/09/22 14:53:28 INFO lead: released remote=192.168.1.4:57346 worker1 # [ 144.999631] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 145.052945] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting worker2 # [ 145.058288] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 145.063468] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:38230 worker2 # [ 145.068766] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 145.073681] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:38230 worker2 # [ 145.078412] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 145.105321] nix-daemon[839]: reaped child process 1355, status = succeeded worker2 # [ 145.223332] nix-grpc-daemon[1303]: E0000 00:00:1790088809.823293 1348 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 12, started true, shutdown true worker2 # [ 145.230053] nix-grpc-daemon[1303]: E0000 00:00:1790088809.823606 1348 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 145.199303] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.204776] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 145.274067] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 145.405465] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.410642] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 145.611480] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.616700] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' worker1 # [ 145.817817] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.823106] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 146.025577] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.029688] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 146.089665] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 146.095931] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 146.100918] systemd[1]: nix-grpc-daemon.service: Consumed 3.256s CPU time over 35.116s wall clock time, 46.9M memory peak, 84.3K incoming IP traffic, 179.4K outgoing IP traffic. worker2 # [ 146.120431] systemd[1]: Starting scheduler-token.service... worker2 # [ 146.196354] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 146.201259] systemd[1]: Finished scheduler-token.service. worker2 # [ 146.203113] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 146.225061] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 146.226457] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.231723] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 146.339786] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 146.342153] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 146.345895] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 146.351153] nix-grpc-daemon[1706]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 146.359759] nix-grpc-daemon[1706]: E0000 00:00:1790088810.950732 1706 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 146.325708] niks3-server[1141]: 2026/09/22 14:53:30 INFO lead: acquired remote=192.168.1.4:55028 worker2 # [ 146.367916] nix-grpc-daemon[1706]: E0000 00:00:1790088810.954307 1706 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.374332] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 146.375945] nix-grpc-daemon[1706]: E0000 00:00:1790088810.963209 1748 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.399986] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=scheduler_take_over worker2 # [ 146.401779] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 146.402603] nix-grpc-daemon[1706]: E0000 00:00:1790088810.984981 1748 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.404924] nix-grpc-daemon[1706]: E0000 00:00:1790088810.985130 1748 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.410320] nix-grpc-daemon[1706]: E0000 00:00:1790088810.999399 1748 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.414826] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 146.416167] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 146.433056] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.438351] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 146.616330] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 146.621301] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 146.639318] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.644752] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 146.822328] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 146.827527] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 146.845511] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.850803] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 147.028448] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.033472] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 147.052836] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.106108] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:39434 worker2 # [ 147.111333] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 147.234528] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.248160] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 147.252886] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:39434 worker2 # [ 147.258196] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics lb # [ 148.113922] envoy[897]: conn peer=[2001:db8:1::1]:39064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1538 ms=8259 lb # [ 148.120894] envoy[897]: conn peer=[2001:db8:1::4]:38284 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=25409 tx=1859 ms=34079 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.95 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 148.265033] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: a29aee303cdc402a916a44383c8e40ef client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 148.441666] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39444 worker2 # [ 148.456240] nix-daemon[839]: accepted connection from pid 1706, user nix-grpc-daemon (trusted) client # [ 148.501056] nix[1453]: don't know how to build these paths: client # [ 148.502809] nix[1453]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 148.489505] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 148.495902] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 148.489659] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 148.507192] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5) worker1 # [ 148.545759] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5) worker1 # [ 148.669919] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5) worker1 # [ 148.843598] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 4/5) worker1 # [ 149.044275] nix-grpc-daemon[1344]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 149.050916] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 149.071562] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41454 client # [ 149.164224] nix[1453]: copying 1 paths to 'lb:50051'... worker1 # [ 149.087758] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 client # [ 149.164559] nix[1453]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 149.105484] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) worker1 # [ 149.131624] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=info event=niks3_push_started pid=1464 worker1 # [ 149.282975] nix-grpc-daemon[1464]: time=2026-09-22T14:53:33.901Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 149.401432] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted) worker1 # [ 149.432521] nix-daemon[849]: reaped child process 1483, status = succeeded worker1 # [ 149.553532] nix-daemon[849]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 149.572358] nix-daemon[849]: reaped child process 1489, status = succeeded lb # [ 149.594096] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.592073] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.210Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 149.593574] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.210Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 149.645257] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 149.663718] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 149.677321] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 149.687658] niks3-server[1141]: 2026/09/22 14:53:34 INFO Signed narinfos id=51 count=1 worker1 # [ 149.675167] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.293Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.700253] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 149.713880] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo lb # [ 149.728211] niks3-server[1141]: 2026/09/22 14:53:34 INFO Completed upload id=51 worker1 # [ 149.715753] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.334Z level=INFO msg="Upload complete. (428ms)" worker1 # [ 149.724544] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 149.727386] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 149.739068] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) worker1 # [ 149.743360] nix-daemon[849]: reaped child process 1462, status = succeeded worker1 # [ 149.752726] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) client # [ 149.829857] nix[1453]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 152.213062] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1706 (nix-grpc-daemon) on client request. worker2 # [ 152.232468] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 152.205885] niks3-server[1141]: 2026/09/22 14:53:36 INFO lead: released remote=192.168.1.4:55028 worker1 # [ 152.199291] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:36Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" client # [ 152.277106] nix[1453]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 152.262266] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 152.278897] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 152.291469] systemd[1]: nix-grpc-daemon.service: Consumed 227ms CPU time over 6.040s wall clock time, 14.4M memory peak, 9.3K incoming IP traffic, 14.8K outgoing IP traffic. worker2 # [ 152.293994] systemd[1]: Starting scheduler-token.service... worker2 # [ 152.295545] nix-daemon[839]: reaped child process 1765, status = succeeded worker2 # [ 152.339321] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.344874] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.346009] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.349344] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 152.353037] systemd[1]: Starting scheduler-token.service... worker2 # [ 152.405365] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.410808] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.411905] systemd[1]: scheduler-token.service: Consumed 28ms CPU time over 54ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.416538] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 152.399501] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.524594] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 152.527187] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 152.530984] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 152.536063] nix-grpc-daemon[1799]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 152.537506] nix-grpc-daemon[1799]: E0000 00:00:1790088817.135892 1799 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 152.516393] niks3-server[1141]: 2026/09/22 14:53:37 INFO lead: acquired remote=192.168.1.4:34530 worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds) client: must succeed: sleep 5.0 worker2 # [ 152.574391] nix-grpc-daemon[1799]: E0000 00:00:1790088817.142111 1799 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.589656] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 152.591853] nix-grpc-daemon[1799]: E0000 00:00:1790088817.152933 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.594737] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=scheduler_take_over worker2 # [ 152.595902] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48276 worker2 # [ 152.600513] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=1 worker2 # [ 152.604663] nix-grpc-daemon[1799]: E0000 00:00:1790088817.172097 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.607914] nix-grpc-daemon[1799]: E0000 00:00:1790088817.172255 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.610845] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 152.612301] nix-grpc-daemon[1799]: E0000 00:00:1790088817.189229 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.614949] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.618197] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 152.619476] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48276 worker2 # [ 152.621293] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 157.787924] nix-daemon[849]: reaped child process 1493, status = succeeded client: waiting for success: ! systemctl is-active mid worker1 # [ 157.887687] nix-daemon[849]: accepted connection from pid 1530, user nix-grpc-daemon (trusted) worker1 # [ 157.909799] nix-daemon[849]: reaped child process 1533, status = succeeded lb # [ 157.927171] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 157.931472] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.547Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 157.933865] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.547Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 157.981183] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 157.999767] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 158.011089] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 158.020524] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 158.032532] niks3-server[1141]: 2026/09/22 14:53:42 INFO Signed narinfos id=52 count=1 worker1 # [ 158.020084] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.637Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.040764] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 158.051138] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo lb # [ 158.058457] niks3-server[1141]: 2026/09/22 14:53:42 INFO Completed upload id=52 worker1 # [ 158.045897] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.664Z level=INFO msg="Upload complete. (276ms)" worker1 # [ 158.056794] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 158.067052] nix-daemon[849]: reaped child process 1491, status = succeeded client # [ 158.170564] systemd[1]: mid.service: Deactivated successfully. client # [ 158.176286] systemd[1]: mid.service: Consumed 203ms CPU time over 9.907s wall clock time, 39.2M memory peak, 7.8K incoming IP traffic, 8.8K outgoing IP traffic. lb # [ 158.115895] envoy[897]: conn peer=[2001:db8:1::4]:36860 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=5846 lb # [ 158.120791] envoy[897]: conn peer=[2001:db8:1::1]:42928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1630 ms=9685 client: (finished: waiting for success: ! systemctl is-active mid, in 1.13 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.85 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 159.083327] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: ae507e2abbec46519e96d7fe1381abd4 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 159.240767] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 159.250584] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # [ 159.299117] nix[1489]: don't know how to build these paths: client # [ 159.301129] nix[1489]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker2 # [ 159.281797] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker2 # [ 159.284802] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 159.334739] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 159.350486] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5) worker2 # [ 159.429243] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 2/5) worker2 # [ 159.610810] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 3/5) worker2 # [ 159.946531] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5) worker2 # [ 160.525858] nix-grpc-daemon[1799]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 160.532852] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 160.554877] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 client # [ 160.598304] nix[1489]: copying 1 paths to 'lb:50051'... worker2 # [ 160.575361] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 client # [ 160.603542] nix[1489]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 160.589927] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 160.623337] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=info event=niks3_push_started pid=1877 worker2 # [ 160.769220] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.369Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 160.882335] nix-daemon[839]: accepted connection from pid 1889, user nix-grpc-daemon (trusted) worker2 # [ 160.904660] nix-daemon[839]: reaped child process 1892, status = succeeded worker2 # [ 161.012857] nix-daemon[839]: accepted connection from pid 1894, user nix-grpc-daemon (trusted) lb # [ 160.997074] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 161.035528] nix-daemon[839]: reaped child process 1897, status = succeeded worker2 # [ 161.044195] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 161.045699] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.643Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 161.045910] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker2 # [ 161.095478] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.695Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.064681] niks3-server[1141]: 2026/09/22 14:53:45 INFO Signed narinfos id=53 count=1 lb # [ 161.072152] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 161.087738] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 161.096516] niks3-server[1141]: 2026/09/22 14:53:45 INFO Completed upload id=53 worker2 # [ 161.131919] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.732Z level=INFO msg="Upload complete. (359ms)" lb # [ 161.099980] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo lb # [ 161.107598] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls worker2 # [ 161.144580] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 161.155887] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 161.169720] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 161.175448] nix-daemon[839]: reaped child process 1875, status = succeeded worker2 # [ 161.186792] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # [ 161.213077] nix[1489]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr worker2 # [ 162.720122] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 162.744649] systemd[1]: intr.service: Sent signal SIGINT to main process 1489 (nix) on client request. client # [ 162.754680] nix[1489]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 162.770084] nix[1489]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 162.800332] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 162.805375] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 162.807819] systemd[1]: intr.service: Consumed 176ms CPU time over 3.720s wall clock time, 39.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic. worker2 # [ 162.821308] nix-daemon[839]: reaped child process 1899, status = succeeded worker2 # [ 162.822648] nix-daemon[1901]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 162.838264] nix-daemon[839]: reaped child process 1901, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 3.84 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 162.927442] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 0beaecfb8a0e4143a101faa8c1d90f2e client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.05 seconds) client # [ 163.119602] nix[1510]: don't know how to build these paths: client # [ 163.121715] nix[1510]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 163.042491] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41454 worker2 # [ 163.108402] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker2 # [ 163.115365] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 163.113441] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 163.129859] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41454 client # [ 163.220215] nix[1510]: copying 1 paths to 'lb:50051'... client # [ 163.222161] nix[1510]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 163.145892] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 163.162471] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) worker1 # [ 163.345929] nix-daemon[849]: accepted connection from pid 1561, user nix-grpc-daemon (trusted) worker1 # [ 163.369993] nix-daemon[849]: reaped child process 1564, status = succeeded worker1 # [ 163.480844] nix-daemon[849]: accepted connection from pid 1566, user nix-grpc-daemon (trusted) worker1 # [ 163.504314] nix-daemon[849]: reaped child process 1569, status = succeeded lb # [ 163.517128] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 163.521462] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 163.523007] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.139Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 163.559561] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 163.572523] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 163.565664] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.184Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.582198] niks3-server[1141]: 2026/09/22 14:53:48 INFO Signed narinfos id=54 count=1 lb # [ 163.588291] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 163.601570] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 163.608135] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo lb # [ 163.613787] niks3-server[1141]: 2026/09/22 14:53:48 INFO Completed upload id=54 worker1 # [ 163.601035] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.219Z level=INFO msg="Upload complete. (415ms)" worker1 # [ 163.612299] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 163.628885] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 worker1 # [ 163.635659] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) worker1 # [ 163.641366] nix-daemon[849]: reaped child process 1559, status = succeeded worker1 # [ 163.652542] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted) client # [ 163.728093] nix[1510]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 165.490403] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 165.536675] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 165.875713] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 165.884494] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 worker1 # [ 165.850377] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:50Z level=info event=draining worker2 # [ 165.913018] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker2 # [ 165.919891] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 165.961464] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 165.979944] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 165.998300] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 166.014931] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 166.177828] nix-daemon[839]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker2 # [ 166.204672] nix-daemon[839]: reaped child process 1955, status = succeeded worker2 # [ 166.302620] nix-daemon[839]: accepted connection from pid 1957, user nix-grpc-daemon (trusted) worker2 # [ 166.324311] nix-daemon[839]: reaped child process 1960, status = succeeded lb # [ 166.286122] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 166.333923] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 166.335416] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.933Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 166.327753] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 166.342285] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker2 # [ 166.386577] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.986Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.352368] niks3-server[1141]: 2026/09/22 14:53:50 INFO Signed narinfos id=55 count=1 lb # [ 166.359549] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 166.375367] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 166.385988] niks3-server[1141]: 2026/09/22 14:53:50 INFO Completed upload id=55 worker2 # [ 166.422816] nix-grpc-daemon[1877]: time=2026-09-22T14:53:51.023Z level=INFO msg="Upload complete. (389ms)" lb # [ 166.388853] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 166.433360] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 166.450704] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 166.457689] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 166.465406] nix-daemon[839]: reaped child process 1950, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker2 # [ 166.477530] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain lb # [ 168.115601] envoy[897]: conn peer=[2001:db8:1::1]:52032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2974 tx=853 ms=3520 worker2 # [ 168.168739] nix-daemon[839]: reaped child process 1964, status = succeeded worker2 # [ 168.270393] nix-daemon[839]: accepted connection from pid 1980, user nix-grpc-daemon (trusted) worker2 # [ 168.294193] nix-daemon[839]: reaped child process 1983, status = succeeded lb # [ 168.256499] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.305241] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.904Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 168.312935] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.908Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 168.298324] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 168.314995] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 168.328082] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker2 # [ 168.366227] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.965Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.331774] niks3-server[1141]: 2026/09/22 14:53:52 INFO Signed narinfos id=56 count=1 lb # [ 168.342037] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 168.352741] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 168.359605] niks3-server[1141]: 2026/09/22 14:53:52 INFO Completed upload id=56 worker2 # [ 168.395542] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.995Z level=INFO msg="Upload complete. (301ms)" lb # [ 168.363081] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 168.416299] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 worker2 # [ 168.432437] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker2 # [ 168.437708] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 168.442345] nix-daemon[839]: reaped child process 1962, status = succeeded worker2 # [ 168.461694] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 168.477236] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 168.496622] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.096Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 168.507893] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 168.526192] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 168.686177] nix-daemon[839]: accepted connection from pid 1989, user nix-grpc-daemon (trusted) worker2 # [ 168.715304] nix-daemon[839]: reaped child process 1992, status = succeeded worker2 # [ 168.832142] nix-daemon[839]: accepted connection from pid 1994, user nix-grpc-daemon (trusted) worker2 # [ 168.850963] nix-daemon[839]: reaped child process 1997, status = succeeded lb # [ 168.821272] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.872070] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.472Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 168.874485] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.472Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 168.859756] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker2 # [ 168.914504] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.514Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.877559] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 168.884366] niks3-server[1141]: 2026/09/22 14:53:53 INFO Signed narinfos id=57 count=1 lb # [ 168.891667] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 168.902807] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete worker2 # [ 168.949552] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.549Z level=INFO msg="Upload complete. (411ms)" lb # [ 168.914048] niks3-server[1141]: 2026/09/22 14:53:53 INFO Completed upload id=57 lb # [ 168.919223] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 168.960489] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 168.969029] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 168.981947] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 168.987116] nix-daemon[839]: reaped child process 1987, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker2 # [ 168.998186] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 169.853322] nix-daemon[839]: reaped child process 2001, status = succeeded worker2 # [ 169.967917] nix-daemon[839]: accepted connection from pid 2017, user nix-grpc-daemon (trusted) worker2 # [ 169.986815] nix-daemon[839]: reaped child process 2020, status = succeeded lb # [ 169.959100] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.007625] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.606Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 170.009586] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.606Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 170.000824] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 170.013844] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 170.059610] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.659Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.024990] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 170.033631] niks3-server[1141]: 2026/09/22 14:53:54 INFO Signed narinfos id=58 count=1 lb # [ 170.038666] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 170.047878] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 170.053563] niks3-server[1141]: 2026/09/22 14:53:54 INFO Completed upload id=58 worker2 # [ 170.090662] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.690Z level=INFO msg="Upload complete. (280ms)" lb # [ 170.056677] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker2 # [ 170.110309] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 worker2 # [ 170.128397] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 170.133443] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 170.138312] nix-daemon[839]: reaped child process 1999, status = succeeded worker2 # [ 170.153796] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 170.170533] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 170.188615] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.788Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 170.201192] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 170.216093] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 170.372967] nix-daemon[839]: accepted connection from pid 2026, user nix-grpc-daemon (trusted) worker2 # [ 170.396465] nix-daemon[839]: reaped child process 2029, status = succeeded worker2 # [ 170.503439] nix-daemon[839]: accepted connection from pid 2031, user nix-grpc-daemon (trusted) worker2 # [ 170.524504] nix-daemon[839]: reaped child process 2034, status = succeeded lb # [ 170.491100] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.539583] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.139Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 170.541246] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.139Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 170.522183] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst worker2 # [ 170.574945] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.175Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.538951] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 170.551362] niks3-server[1141]: 2026/09/22 14:53:55 INFO Signed narinfos id=59 count=1 lb # [ 170.558953] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 170.574399] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 170.581945] niks3-server[1141]: 2026/09/22 14:53:55 INFO Completed upload id=59 lb # [ 170.584698] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 170.616675] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.216Z level=INFO msg="Upload complete. (383ms)" worker2 # [ 170.630003] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 170.644962] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 170.652424] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 170.657853] nix-daemon[839]: reaped child process 2024, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 170.672082] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 171.539905] nix-daemon[839]: reaped child process 2038, status = succeeded worker2 # [ 171.657413] nix-daemon[839]: accepted connection from pid 2054, user nix-grpc-daemon (trusted) worker2 # [ 171.679702] nix-daemon[839]: reaped child process 2057, status = succeeded lb # [ 171.642427] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 171.692475] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.291Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 171.694137] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.291Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 171.677634] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 171.691703] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 171.707196] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker2 # [ 171.749012] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.348Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.715267] niks3-server[1141]: 2026/09/22 14:53:56 INFO Signed narinfos id=60 count=1 lb # [ 171.721624] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 171.735571] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete worker2 # [ 171.780144] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.380Z level=INFO msg="Upload complete. (298ms)" lb # [ 171.743372] niks3-server[1141]: 2026/09/22 14:53:56 INFO Completed upload id=60 lb # [ 171.747699] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 171.793297] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 worker2 # [ 171.815345] nix-daemon[839]: reaped child process 2036, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.19 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 172.094709] nix-daemon[849]: reaped child process 1573, status = succeeded worker1 # [ 172.199797] nix-daemon[849]: accepted connection from pid 1614, user nix-grpc-daemon (trusted) worker1 # [ 172.221046] nix-daemon[849]: reaped child process 1617, status = succeeded lb # [ 172.238642] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.237155] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.854Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 172.238829] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.854Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 172.283513] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 172.284422] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.902Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.297813] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 172.309115] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 172.317017] niks3-server[1141]: 2026/09/22 14:53:56 INFO Signed narinfos id=61 count=1 lb # [ 172.320887] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls worker1 # [ 172.315383] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.934Z level=INFO msg="Upload complete. (242ms)" lb # [ 172.331224] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 172.334277] niks3-server[1141]: 2026/09/22 14:53:56 INFO Completed upload id=61 lb # [ 172.338876] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 172.329220] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 worker1 # [ 172.346151] nix-daemon[849]: reaped child process 1571, status = succeeded client # [ 172.443972] systemd[1]: sw.service: Deactivated successfully. client # [ 172.449568] systemd[1]: sw.service: Consumed 190ms CPU time over 9.517s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 172.852366] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:57Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 173.914373] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48276 worker2 # [ 173.919348] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:58Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 173.872440] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:58Z level=info event=scheduler_restarting worker1 # [ 173.889824] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 173.915420] nix-daemon[849]: reaped child process 1403, status = succeeded worker1 # [ 173.949762] nix-grpc-daemon[1344]: E0000 00:00:1790088838.568357 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 173.953211] nix-grpc-daemon[1344]: E0000 00:00:1790088838.568514 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 174.240612] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 174.247163] systemd[1]: nix-grpc-daemon.service: Consumed 1.201s CPU time over 40.572s wall clock time, 36.2M memory peak, 26.7K incoming IP traffic, 39K outgoing IP traffic. worker1 # [ 174.269195] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 174.411867] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 174.414521] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 174.418310] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 174.424080] nix-grpc-daemon[1631]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 174.425646] nix-grpc-daemon[1631]: E0000 00:00:1790088839.041647 1631 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 174.473922] nix-grpc-daemon[1631]: E0000 00:00:1790088839.048339 1631 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.28 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 174.477320] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 174.479111] nix-grpc-daemon[1631]: E0000 00:00:1790088839.055210 1670 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 174.481592] nix-grpc-daemon[1631]: E0000 00:00:1790088839.073726 1670 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 174.487625] nix-grpc-daemon[1631]: E0000 00:00:1790088839.074002 1670 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 174.492660] nix-grpc-daemon[1631]: E0000 00:00:1790088839.091028 1670 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 174.495222] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 174.496143] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 174.542976] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48276 worker2 # [ 174.549713] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.93 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 176.033135] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48290 worker1 # [ 175.994346] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 176.009934] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 176.035141] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5) worker1 # [ 176.110995] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5) worker1 # [ 176.269158] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 3/5) worker1 # [ 176.460121] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 724 ms (attempt 4/5) worker1 # [ 177.188001] nix-grpc-daemon[1631]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 177.190181] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 177.301495] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 177.308352] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 177.314783] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 177.320268] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 177.305981] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 177.326112] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 177.343930] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 177.360841] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 177.396381] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=info event=niks3_push_started pid=1691 worker1 # [ 177.542233] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.160Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 177.654515] nix-daemon[849]: accepted connection from pid 1704, user nix-grpc-daemon (trusted) worker1 # [ 177.675422] nix-daemon[849]: reaped child process 1707, status = succeeded worker1 # [ 177.785873] nix-daemon[849]: accepted connection from pid 1709, user nix-grpc-daemon (trusted) worker1 # [ 177.807591] nix-daemon[849]: reaped child process 1712, status = succeeded lb # [ 177.823054] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.822010] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.439Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 177.823512] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.439Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 177.865755] niks3-server[1141]: 2026/09/22 14:54:02 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 177.873651] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.491Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.885647] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 177.894808] niks3-server[1141]: 2026/09/22 14:54:02 INFO Signed narinfos id=62 count=1 lb # [ 177.911124] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 177.921496] niks3-server[1141]: 2026/09/22 14:54:02 INFO Completed upload id=62 worker1 # [ 177.906154] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.524Z level=INFO msg="Upload complete. (360ms)" lb # [ 177.927977] niks3-server[1141]: 2026/09/22 14:54:02 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 177.916490] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 177.928541] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 177.939951] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 177.945070] nix-daemon[849]: reaped child process 1689, status = succeeded worker1 # [ 177.958670] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) lb # [ 178.116463] envoy[897]: conn peer=[2001:db8:1::1]:51204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4402 ms=5934 lb # [ 178.119429] envoy[897]: conn peer=[2001:db8:1::1]:51190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1517 ms=9305 lb # [ 178.125347] envoy[897]: conn peer=[2001:db8:1::3]:49218 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2445 tx=766 ms=39977 worker1 # [ 178.634086] nix-daemon[849]: reaped child process 1716, status = succeeded worker1 # [ 178.735574] nix-daemon[849]: accepted connection from pid 1732, user nix-grpc-daemon (trusted) worker1 # [ 178.759015] nix-daemon[849]: reaped child process 1735, status = succeeded lb # [ 178.775922] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 178.776561] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 178.778565] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.394Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 178.834917] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker1 # [ 178.832164] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.449Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.849705] niks3-server[1141]: 2026/09/22 14:54:03 INFO Signed narinfos id=63 count=1 lb # [ 178.856109] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 178.865933] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 178.875804] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 178.888767] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete worker1 # [ 178.881339] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.500Z level=INFO msg="Upload complete. (313ms)" lb # [ 178.897191] niks3-server[1141]: 2026/09/22 14:54:03 INFO Completed upload id=63 lb # [ 178.899344] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker2 # [ 178.945658] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:03Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 178.896748] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=1 worker2 # [ 178.956546] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:03Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 178.916597] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 178.925074] nix-daemon[849]: reaped child process 1714, status = succeeded worker1 # [ 178.933876] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 178.947663] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 178.961196] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 179.104927] nix-daemon[849]: accepted connection from pid 1739, user nix-grpc-daemon (trusted) worker1 # [ 179.124496] nix-daemon[849]: reaped child process 1742, status = succeeded worker1 # [ 179.227847] nix-daemon[849]: accepted connection from pid 1744, user nix-grpc-daemon (trusted) worker1 # [ 179.246783] nix-daemon[849]: reaped child process 1747, status = succeeded lb # [ 179.263515] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.260099] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.878Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 179.263014] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.878Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 179.296234] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 179.297414] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.915Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.311100] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 179.322827] niks3-server[1141]: 2026/09/22 14:54:03 INFO Signed narinfos id=64 count=1 lb # [ 179.329163] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 179.340958] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete worker1 # [ 179.332947] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.951Z level=INFO msg="Upload complete. (358ms)" lb # [ 179.348361] niks3-server[1141]: 2026/09/22 14:54:03 INFO Completed upload id=64 lb # [ 179.352508] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 179.345054] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 179.355852] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 179.363668] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 179.369227] nix-daemon[849]: reaped child process 1737, status = succeeded worker1 # [ 179.383224] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 180.896654] nix-daemon[849]: reaped child process 1751, status = succeeded worker1 # [ 180.968967] nix-daemon[849]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 180.991132] nix-daemon[849]: reaped child process 1770, status = succeeded lb # [ 181.013508] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.013784] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 181.015572] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.631Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 181.056063] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 181.068734] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 181.065125] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.683Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.078579] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 181.088338] niks3-server[1141]: 2026/09/22 14:54:05 INFO Signed narinfos id=65 count=1 lb # [ 181.095686] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 181.110851] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 181.116745] niks3-server[1141]: 2026/09/22 14:54:05 INFO Completed upload id=65 worker1 # [ 181.103417] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.722Z level=INFO msg="Upload complete. (308ms)" lb # [ 181.120572] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker2 # [ 181.164716] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:05Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.117237] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=1 worker2 # [ 181.178145] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:05Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 181.138434] nix-daemon[849]: reaped child process 1749, status = succeeded worker1 # [ 181.172886] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 181.191641] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 181.208665] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 181.223176] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 181.387865] nix-daemon[849]: accepted connection from pid 1774, user nix-grpc-daemon (trusted) worker1 # [ 181.415697] nix-daemon[849]: reaped child process 1777, status = succeeded worker1 # [ 181.518392] nix-daemon[849]: accepted connection from pid 1779, user nix-grpc-daemon (trusted) worker1 # [ 181.538775] nix-daemon[849]: reaped child process 1782, status = succeeded lb # [ 181.555234] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.551586] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.170Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 181.553267] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.170Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 181.582491] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 181.598672] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign worker1 # [ 181.593348] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.211Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.612454] niks3-server[1141]: 2026/09/22 14:54:06 INFO Signed narinfos id=66 count=1 lb # [ 181.615681] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 181.617358] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.236Z level=INFO msg="Upload complete. (377ms)" lb # [ 181.630303] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker1 # [ 181.624142] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 181.639537] niks3-server[1141]: 2026/09/22 14:54:06 INFO Completed upload id=66 lb # [ 181.642391] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 181.639681] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 181.646024] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 181.650641] nix-daemon[849]: reaped child process 1772, status = succeeded worker1 # [ 181.663763] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 182.943281] nix-daemon[849]: reaped child process 1786, status = succeeded worker1 # [ 183.046043] nix-daemon[849]: accepted connection from pid 1802, user nix-grpc-daemon (trusted) worker1 # [ 183.066815] nix-daemon[849]: reaped child process 1805, status = succeeded lb # [ 183.087107] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.087564] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.704Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 183.089617] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.704Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 183.129121] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 183.141804] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker1 # [ 183.141770] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.760Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.157865] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 183.168842] niks3-server[1141]: 2026/09/22 14:54:07 INFO Signed narinfos id=67 count=1 lb # [ 183.171834] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls worker1 # [ 183.173356] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.792Z level=INFO msg="Upload complete. (299ms)" lb # [ 183.186787] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 183.191431] niks3-server[1141]: 2026/09/22 14:54:07 INFO Completed upload id=67 lb # [ 183.196167] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 183.233251] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:07Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 183.188247] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=1 worker2 # [ 183.246671] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:07Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 183.208511] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 183.217464] nix-daemon[849]: reaped child process 1784, status = succeeded worker1 # [ 183.226563] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 183.242485] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.861Z level=INFO msg="All 1 paths already cached" worker1 # [ 183.252915] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 183.270080] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 183.416296] nix-daemon[849]: accepted connection from pid 1809, user nix-grpc-daemon (trusted) worker1 # [ 183.439847] nix-daemon[849]: reaped child process 1812, status = succeeded worker1 # [ 183.546894] nix-daemon[849]: accepted connection from pid 1814, user nix-grpc-daemon (trusted) worker1 # [ 183.567074] nix-daemon[849]: reaped child process 1817, status = succeeded lb # [ 183.584596] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.581168] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.199Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.582927] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.199Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 183.615533] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 183.619061] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.237Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.631546] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 183.642482] niks3-server[1141]: 2026/09/22 14:54:08 INFO Signed narinfos id=68 count=1 lb # [ 183.648892] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 183.662329] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker1 # [ 183.656912] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.275Z level=INFO msg="Upload complete. (374ms)" lb # [ 183.672457] niks3-server[1141]: 2026/09/22 14:54:08 INFO Completed upload id=68 lb # [ 183.676317] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 183.668347] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 183.684267] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 183.690965] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 183.696964] nix-daemon[849]: reaped child process 1807, status = succeeded worker1 # [ 183.708998] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 185.024143] nix-daemon[849]: reaped child process 1821, status = succeeded worker1 # [ 185.098172] nix-daemon[849]: accepted connection from pid 1837, user nix-grpc-daemon (trusted) worker1 # [ 185.118725] nix-daemon[849]: reaped child process 1840, status = succeeded lb # [ 185.141213] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.142789] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 185.144661] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.758Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 185.191548] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst worker1 # [ 185.192043] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.810Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.207815] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 185.217548] niks3-server[1141]: 2026/09/22 14:54:09 INFO Signed narinfos id=69 count=1 lb # [ 185.225351] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 185.221085] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.839Z level=INFO msg="Upload complete. (297ms)" lb # [ 185.236245] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 185.244171] niks3-server[1141]: 2026/09/22 14:54:09 INFO Completed upload id=69 worker2 # [ 185.281453] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:09Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 lb # [ 185.247192] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls worker1 # [ 185.233220] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=1 lb # [ 185.252202] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 185.296075] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:09Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 185.256499] nix-daemon[849]: reaped child process 1819, status = succeeded worker1 # [ 185.284467] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 185.303370] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 185.321123] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.939Z level=INFO msg="All 1 paths already cached" worker1 # [ 185.334349] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 185.347355] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 185.504625] nix-daemon[849]: accepted connection from pid 1844, user nix-grpc-daemon (trusted) worker1 # [ 185.527182] nix-daemon[849]: reaped child process 1847, status = succeeded worker1 # [ 185.636999] nix-daemon[849]: accepted connection from pid 1849, user nix-grpc-daemon (trusted) worker1 # [ 185.657101] nix-daemon[849]: reaped child process 1852, status = succeeded lb # [ 185.673971] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.671567] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.289Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 185.673243] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.290Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 185.704495] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 185.704748] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.323Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.719373] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 185.730275] niks3-server[1141]: 2026/09/22 14:54:10 INFO Signed narinfos id=70 count=1 lb # [ 185.736675] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 185.733489] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.352Z level=INFO msg="Upload complete. (369ms)" lb # [ 185.746277] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete worker1 # [ 185.740289] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 185.755233] niks3-server[1141]: 2026/09/22 14:54:10 INFO Completed upload id=70 lb # [ 185.758836] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 185.755711] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 185.762391] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 185.767322] nix-daemon[849]: reaped child process 1842, status = succeeded worker1 # [ 185.779111] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 186.610371] nix-daemon[849]: reaped child process 1856, status = succeeded worker1 # [ 186.717937] nix-daemon[849]: accepted connection from pid 1872, user nix-grpc-daemon (trusted) worker1 # [ 186.740819] nix-daemon[849]: reaped child process 1875, status = succeeded lb # [ 186.757676] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.756580] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.372Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 186.760641] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.372Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 186.796205] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 186.810924] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 186.824613] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 186.817957] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.436Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.836535] niks3-server[1141]: 2026/09/22 14:54:11 INFO Signed narinfos id=71 count=1 lb # [ 186.839251] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 186.853501] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 186.860259] niks3-server[1141]: 2026/09/22 14:54:11 INFO Completed upload id=71 worker1 # [ 186.847204] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.465Z level=INFO msg="Upload complete. (255ms)" lb # [ 186.864365] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 186.911488] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:11Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 186.864526] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=1 worker2 # [ 186.919362] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:11Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 186.880546] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 186.888637] nix-daemon[849]: reaped child process 1854, status = succeeded worker1 # [ 186.893451] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 186.903297] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.521Z level=INFO msg="All 1 paths already cached" worker1 # [ 186.912822] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 186.925108] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 187.070400] nix-daemon[849]: accepted connection from pid 1879, user nix-grpc-daemon (trusted) worker1 # [ 187.096717] nix-daemon[849]: reaped child process 1882, status = succeeded worker1 # [ 187.198802] nix-daemon[849]: accepted connection from pid 1884, user nix-grpc-daemon (trusted) worker1 # [ 187.218021] nix-daemon[849]: reaped child process 1887, status = succeeded lb # [ 187.238487] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.235230] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.853Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 187.236881] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.853Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 187.270658] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 187.273508] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.892Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.287137] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 187.298117] niks3-server[1141]: 2026/09/22 14:54:11 INFO Signed narinfos id=72 count=1 lb # [ 187.305074] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 187.318041] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 187.310618] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.929Z level=INFO msg="Upload complete. (371ms)" lb # [ 187.326629] niks3-server[1141]: 2026/09/22 14:54:11 INFO Completed upload id=72 lb # [ 187.330806] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 187.320332] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 187.336517] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 187.343830] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 187.350460] nix-daemon[849]: reaped child process 1877, status = succeeded worker1 # [ 187.365678] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 188.456761] nix-daemon[849]: reaped child process 1891, status = succeeded worker1 # [ 188.553251] nix-daemon[849]: accepted connection from pid 1907, user nix-grpc-daemon (trusted) worker1 # [ 188.575782] nix-daemon[849]: reaped child process 1910, status = succeeded lb # [ 188.592812] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.590656] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.208Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 188.592406] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.208Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 188.631777] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 188.644147] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst worker1 # [ 188.642345] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.260Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.658322] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 188.667767] niks3-server[1141]: 2026/09/22 14:54:13 INFO Signed narinfos id=73 count=1 lb # [ 188.675262] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls worker1 # [ 188.662446] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.281Z level=INFO msg="Upload complete. (285ms)" lb # [ 188.681608] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 188.687491] niks3-server[1141]: 2026/09/22 14:54:13 INFO Completed upload id=73 lb # [ 188.691113] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 188.673426] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=1 worker2 # [ 188.726032] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:13Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 188.739932] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:13Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 188.696518] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 188.703926] nix-daemon[849]: reaped child process 1889, status = succeeded worker1 # [ 188.712576] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 188.726308] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.344Z level=INFO msg="All 2 paths already cached" worker1 # [ 188.736291] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 188.748772] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 188.893521] nix-daemon[849]: accepted connection from pid 1914, user nix-grpc-daemon (trusted) worker1 # [ 188.915114] nix-daemon[849]: reaped child process 1917, status = succeeded worker1 # [ 189.018712] nix-daemon[849]: accepted connection from pid 1919, user nix-grpc-daemon (trusted) worker1 # [ 189.037641] nix-daemon[849]: reaped child process 1922, status = succeeded lb # [ 189.054902] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.052172] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 189.054149] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.672Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 189.084558] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 189.088478] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.707Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.101117] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 189.112947] niks3-server[1141]: 2026/09/22 14:54:13 INFO Signed narinfos id=74 count=1 lb # [ 189.121317] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 189.122365] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.741Z level=INFO msg="Upload complete. (359ms)" lb # [ 189.137397] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete worker1 # [ 189.132254] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 189.148340] niks3-server[1141]: 2026/09/22 14:54:13 INFO Completed upload id=74 lb # [ 189.151647] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 189.147965] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 189.154323] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 189.159316] nix-daemon[849]: reaped child process 1912, status = succeeded worker1 # [ 189.170883] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 190.640685] nix-daemon[849]: reaped child process 1926, status = succeeded worker1 # [ 190.771894] nix-daemon[849]: accepted connection from pid 1942, user nix-grpc-daemon (trusted) worker1 # [ 190.791337] nix-daemon[849]: reaped child process 1945, status = succeeded lb # [ 190.809407] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.808463] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.424Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 190.810322] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.424Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 190.853419] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 190.866166] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst worker1 # [ 190.865715] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.484Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.879032] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 190.888585] niks3-server[1141]: 2026/09/22 14:54:15 INFO Signed narinfos id=75 count=1 lb # [ 190.894885] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 190.906517] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 190.914373] niks3-server[1141]: 2026/09/22 14:54:15 INFO Completed upload id=75 worker1 # [ 190.901254] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.519Z level=INFO msg="Upload complete. (315ms)" lb # [ 190.918206] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 190.961809] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:15Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 190.914321] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=1 worker1 # [ 190.936451] nix-daemon[849]: reaped child process 1924, status = succeeded worker1 # [ 190.959625] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 190.976668] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 190.991205] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.609Z level=INFO msg="All 2 paths already cached" worker1 # [ 191.003006] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 191.015928] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 191.171662] nix-daemon[849]: accepted connection from pid 1949, user nix-grpc-daemon (trusted) worker1 # [ 191.200475] nix-daemon[849]: reaped child process 1952, status = succeeded worker1 # [ 191.303179] nix-daemon[849]: accepted connection from pid 1954, user nix-grpc-daemon (trusted) worker1 # [ 191.320520] nix-daemon[849]: reaped child process 1957, status = succeeded lb # [ 191.339725] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.337128] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.955Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 191.338659] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.955Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 191.370210] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 191.375147] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.993Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.388477] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 191.395453] niks3-server[1141]: 2026/09/22 14:54:15 INFO Signed narinfos id=76 count=1 lb # [ 191.403239] niks3-server[1141]: 2026/09/22 14:54:16 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 191.413035] niks3-server[1141]: 2026/09/22 14:54:16 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete worker1 # [ 191.406595] nix-grpc-daemon[1691]: time=2026-09-22T14:54:16.025Z level=INFO msg="Upload complete. (375ms)" lb # [ 191.423391] niks3-server[1141]: 2026/09/22 14:54:16 INFO Completed upload id=76 lb # [ 191.429866] niks3-server[1141]: 2026/09/22 14:54:16 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 191.416276] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 191.430385] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 191.436587] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 191.441796] nix-daemon[849]: reaped child process 1947, status = succeeded worker1 # [ 191.455827] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 192.565195] nix-daemon[849]: reaped child process 1961, status = succeeded worker1 # [ 192.637111] nix-daemon[849]: accepted connection from pid 1977, user nix-grpc-daemon (trusted) worker1 # [ 192.659539] nix-daemon[849]: reaped child process 1980, status = succeeded lb # [ 192.671574] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.665560] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.283Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 192.667069] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.283Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 192.693731] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 192.700232] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker1 # [ 192.693749] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.312Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.706282] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 192.713551] niks3-server[1141]: 2026/09/22 14:54:17 INFO Signed narinfos id=77 count=1 lb # [ 192.715206] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 192.723786] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 192.729408] niks3-server[1141]: 2026/09/22 14:54:17 INFO Completed upload id=77 lb # [ 192.731639] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 192.715677] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.334Z level=INFO msg="Upload complete. (249ms)" worker2 # [ 192.773918] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:17Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 192.725871] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=1 worker1 # [ 192.738634] nix-daemon[849]: reaped child process 1959, status = succeeded worker1 # [ 192.771265] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 192.786473] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 192.800585] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.419Z level=INFO msg="All 2 paths already cached" worker1 # [ 192.810807] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 192.824906] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 192.974370] nix-daemon[849]: accepted connection from pid 1984, user nix-grpc-daemon (trusted) worker1 # [ 192.995709] nix-daemon[849]: reaped child process 1987, status = succeeded worker1 # [ 193.097293] nix-daemon[849]: accepted connection from pid 1989, user nix-grpc-daemon (trusted) worker1 # [ 193.114045] nix-daemon[849]: reaped child process 1992, status = succeeded lb # [ 193.133970] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.129895] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.748Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 193.131402] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.748Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 193.161855] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 193.178882] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 193.171933] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.790Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.191263] niks3-server[1141]: 2026/09/22 14:54:17 INFO Signed narinfos id=78 count=1 lb # [ 193.198038] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 193.210120] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete worker1 # [ 193.203861] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.822Z level=INFO msg="Upload complete. (364ms)" lb # [ 193.218795] niks3-server[1141]: 2026/09/22 14:54:17 INFO Completed upload id=78 lb # [ 193.223207] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 193.212290] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 193.228383] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 193.234890] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 193.240135] nix-daemon[849]: reaped child process 1982, status = succeeded worker1 # [ 193.253869] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 194.303419] nix-daemon[849]: reaped child process 1996, status = succeeded worker1 # [ 194.406639] nix-daemon[849]: accepted connection from pid 2012, user nix-grpc-daemon (trusted) worker1 # [ 194.430647] nix-daemon[849]: reaped child process 2015, status = succeeded lb # [ 194.443185] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.442603] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.059Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 194.445438] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.060Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 194.485886] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 194.500185] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 194.511339] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign worker1 # [ 194.504499] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.122Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.521930] niks3-server[1141]: 2026/09/22 14:54:19 INFO Signed narinfos id=79 count=1 lb # [ 194.527075] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 194.537961] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 194.544969] niks3-server[1141]: 2026/09/22 14:54:19 INFO Completed upload id=79 worker1 # [ 194.531679] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.150Z level=INFO msg="Upload complete. (267ms)" lb # [ 194.547589] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 194.545272] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=1 worker1 # [ 194.565209] nix-daemon[849]: reaped child process 1994, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.87 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 194.830520] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 02e125853a064f0fa72234cfcded51b0 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 194.948759] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58180 client # [ 195.033188] nix[1590]: don't know how to build these paths: client # [ 195.034177] nix[1590]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker1 # [ 194.961578] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client # [ 195.064894] nix[1590]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker worker2 # [ 195.038208] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:19Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 198.116695] envoy[897]: conn peer=[2001:db8:1::1]:39718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27101 tx=12377 ms=18579 client # [ 225.071123] nix[1590]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 225.084128] nix[1590]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 225.124848] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 225.131837] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 225.136780] systemd[1]: gpu.service: Consumed 199ms CPU time over 30.299s wall clock time, 35.4M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.71 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.22 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 227.458502] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 227.994374] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 228.003945] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58180 worker2 # [ 228.076152] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 228.081902] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 228.121066] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 228.137323] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... lb # [ 228.114720] envoy[897]: conn peer=[2001:db8:1::1]:39130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=579 ms=30077 worker2 # [ 228.159289] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 228.174281] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 228.339681] nix-daemon[839]: accepted connection from pid 2091, user nix-grpc-daemon (trusted) worker2 # [ 228.361339] nix-daemon[839]: reaped child process 2094, status = succeeded worker2 # [ 228.459649] nix-daemon[839]: accepted connection from pid 2096, user nix-grpc-daemon (trusted) worker2 # [ 228.479231] nix-daemon[839]: reaped child process 2099, status = succeeded lb # [ 228.442372] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.491845] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.091Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 228.493609] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.091Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 228.481355] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 228.540685] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.140Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.504484] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 228.507808] niks3-server[1141]: 2026/09/22 14:54:53 INFO Signed narinfos id=80 count=1 lb # [ 228.517018] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 228.525598] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 228.533127] niks3-server[1141]: 2026/09/22 14:54:53 INFO Completed upload id=80 worker2 # [ 228.569955] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.170Z level=INFO msg="Upload complete. (379ms)" worker2 # [ 228.580305] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 228.547999] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 228.595855] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 228.602719] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 228.607883] nix-daemon[839]: reaped child process 2089, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 228.621675] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 229.455741] nix-daemon[839]: reaped child process 2103, status = succeeded worker2 # [ 229.562269] nix-daemon[839]: accepted connection from pid 2119, user nix-grpc-daemon (trusted) worker2 # [ 229.582682] nix-daemon[839]: reaped child process 2122, status = succeeded lb # [ 229.546814] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 229.596405] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 229.597885] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.194Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 229.588342] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 229.600739] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 229.647046] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.247Z level=INFO msg="Uploading 1 narinfos" lb # [ 229.613901] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 229.620863] niks3-server[1141]: 2026/09/22 14:54:54 INFO Signed narinfos id=81 count=1 lb # [ 229.626579] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 229.634477] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 229.641421] niks3-server[1141]: 2026/09/22 14:54:54 INFO Completed upload id=81 lb # [ 229.644833] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 229.677521] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.277Z level=INFO msg="Upload complete. (246ms)" worker2 # [ 229.693904] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=1 worker2 # [ 229.709622] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 229.713765] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 229.718152] nix-daemon[839]: reaped child process 2101, status = succeeded worker2 # [ 229.737432] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 229.751655] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 229.765946] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.366Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 229.777141] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 229.793477] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 229.935015] nix-daemon[839]: accepted connection from pid 2128, user nix-grpc-daemon (trusted) worker2 # [ 229.954577] nix-daemon[839]: reaped child process 2131, status = succeeded worker2 # [ 230.054825] nix-daemon[839]: accepted connection from pid 2133, user nix-grpc-daemon (trusted) worker2 # [ 230.070907] nix-daemon[839]: reaped child process 2136, status = succeeded lb # [ 230.041529] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 230.088747] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.688Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 230.090219] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.688Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 230.071726] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 230.123007] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.723Z level=INFO msg="Uploading 1 narinfos" lb # [ 230.086838] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 230.096589] niks3-server[1141]: 2026/09/22 14:54:54 INFO Signed narinfos id=82 count=1 lb # [ 230.102283] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 230.112361] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 230.117745] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo lb # [ 230.121805] niks3-server[1141]: 2026/09/22 14:54:54 INFO Completed upload id=82 worker2 # [ 230.158310] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.758Z level=INFO msg="Upload complete. (353ms)" worker2 # [ 230.166923] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 230.181056] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 230.187209] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 230.191670] nix-daemon[839]: reaped child process 2126, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 230.204181] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 231.113830] nix-daemon[839]: reaped child process 2140, status = succeeded worker2 # [ 231.185371] nix-daemon[839]: accepted connection from pid 2156, user nix-grpc-daemon (trusted) worker2 # [ 231.203915] nix-daemon[839]: reaped child process 2159, status = succeeded lb # [ 231.173884] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 231.224136] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.820Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 231.225772] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.820Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 231.215339] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst worker2 # [ 231.265986] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.866Z level=INFO msg="Uploading 1 narinfos" lb # [ 231.229703] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 231.239518] niks3-server[1141]: 2026/09/22 14:54:55 INFO Signed narinfos id=83 count=1 lb # [ 231.246412] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 231.256248] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls worker2 # [ 231.299095] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.899Z level=INFO msg="Upload complete. (283ms)" lb # [ 231.264839] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 231.268705] niks3-server[1141]: 2026/09/22 14:54:55 INFO Completed upload id=83 lb # [ 231.272465] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 231.313122] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=1 worker2 # [ 231.334190] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 231.343996] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 231.352739] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 231.359148] nix-daemon[839]: reaped child process 2138, status = succeeded worker2 # [ 231.363237] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48290 client # copying 1 paths to 'lb:50051'... worker2 # [ 231.369400] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.968Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 231.374563] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 231.386393] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 231.517758] nix-daemon[839]: accepted connection from pid 2165, user nix-grpc-daemon (trusted) worker2 # [ 231.537078] nix-daemon[839]: reaped child process 2168, status = succeeded worker2 # [ 231.638210] nix-daemon[839]: accepted connection from pid 2170, user nix-grpc-daemon (trusted) worker2 # [ 231.653333] nix-daemon[839]: reaped child process 2173, status = succeeded lb # [ 231.623226] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 231.669478] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 231.670977] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.269Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 231.652186] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 231.703509] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.303Z level=INFO msg="Uploading 1 narinfos" lb # [ 231.668490] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 231.678333] niks3-server[1141]: 2026/09/22 14:54:56 INFO Signed narinfos id=84 count=1 lb # [ 231.688561] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete worker2 # [ 231.732143] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.332Z level=INFO msg="Upload complete. (335ms)" lb # [ 231.698727] niks3-server[1141]: 2026/09/22 14:54:56 INFO Completed upload id=84 worker2 # [ 231.739404] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 231.705211] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo lb # [ 231.709338] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 231.748733] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 worker2 # [ 231.759660] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) worker2 # [ 231.764739] nix-daemon[839]: reaped child process 2163, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 231.777397] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 232.856782] nix-daemon[839]: reaped child process 2177, status = succeeded worker2 # [ 232.962744] nix-daemon[839]: accepted connection from pid 2193, user nix-grpc-daemon (trusted) worker2 # [ 232.980808] nix-daemon[839]: reaped child process 2196, status = succeeded lb # [ 232.950871] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 233.000552] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.599Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 233.002882] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.599Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 232.982396] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 232.989865] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker2 # [ 233.026702] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.626Z level=INFO msg="Uploading 1 narinfos" lb # [ 232.995666] niks3-server[1141]: 2026/09/22 14:54:57 INFO Signed narinfos id=85 count=1 lb # [ 232.999627] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 233.005495] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 233.013560] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete worker2 # [ 233.049949] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.650Z level=INFO msg="Upload complete. (261ms)" lb # [ 233.015901] niks3-server[1141]: 2026/09/22 14:54:57 INFO Completed upload id=85 lb # [ 233.020490] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 233.060947] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=1 worker2 # [ 233.075464] nix-daemon[839]: reaped child process 2175, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.32 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 234.461946] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:59Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.15 seconds) (finished: run the VM test script, in 235.52 seconds) test script finished in 235.77s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.53 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh