vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #477
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.7suAKGOSEa', fmt=raw size=107374182414lb # Disk image does not exist, creating the virtualisation disk image...15worker2 # mke2fs 1.47.4 (6-Mar-2025)16worker2: QEMU running (pid 52)17worker1: QEMU running (pid 50)18client # Disk image does not exist, creating the virtualisation disk image...19client: QEMU running (pid 49)20lb: QEMU running (pid 51)21worker1 # Disk image does not exist, creating the virtualisation disk image...22worker2 # Discarding device blocks: 0/262144 done23client # Formatting '/build/vm-state-client/tmp.D5d4tdloP6', fmt=raw size=107374182424lb # Formatting '/build/vm-state-lb/tmp.oXKTJw60uU', fmt=raw size=107374182425(finished: start all VMs, in 0.51 seconds)26client # mke2fs 1.47.4 (6-Mar-2025)27lb # mke2fs 1.47.4 (6-Mar-2025)28client: waiting for unit network-addresses-eth1.service29client # Discarding device blocks: 0/262144 done30lb # Discarding device blocks: 0/262144 done31client: waiting for the VM to finish booting32client # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # Formatting '/build/vm-state-worker1/tmp.FpfAU0Zy09', fmt=raw size=107374182436client # Filesystem UUID: efbc24ab-f24b-4b02-a40e-0beba281942137lb # Filesystem UUID: e48e31f4-8c81-4952-a5bb-5360b0ff2bc838worker2 # Filesystem UUID: 2f7a45d4-390c-434d-9704-89d8651d2b8139worker1 # mke2fs 1.47.4 (6-Mar-2025)40client # Superblock backups stored on blocks:41lb # Superblock backups stored on blocks:42worker2 # Superblock backups stored on blocks:43worker1 # Discarding device blocks: 0/262144 done44client # 32768, 98304, 163840, 22937645lb # 32768, 98304, 163840, 22937646worker2 # 32768, 98304, 163840, 22937647worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes48client # 49lb # 50worker2 # 51worker1 # Filesystem UUID: 6fb13776-fea5-401c-bb9b-b55dc9d947d252client # Allocating group tables: 0/8 done53lb # Allocating group tables: 0/8 done54client # Writing inode tables: 0/8 done55lb # Writing inode tables: 0/8 done56worker2 # Allocating group tables: 0/8 done57worker1 # Superblock backups stored on blocks:58client # Creating journal (8192 blocks): done59lb # Creating journal (8192 blocks): done60worker2 # Writing inode tables: 0/8 done61worker1 # 32768, 98304, 163840, 22937662client # Writing superblocks and filesystem accounting information: 0/8 done63lb # Writing superblocks and filesystem accounting information: 0/8 done64worker2 # Creating journal (8192 blocks): done65worker1 # 66client # 67lb # 68worker2 # Writing superblocks and filesystem accounting information: 0/8 done69worker1 # Allocating group tables: 0/8 done70client # Virtualisation disk image created.71lb # Virtualisation disk image created.72worker2 # 73worker1 # Writing inode tables: 0/8 done74worker2 # Virtualisation disk image created.75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84worker2 # [ 0.000000] efi: UEFI not found.85worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]87worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]88worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]89worker2 # [ 0.000000] Zone ranges:90client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202691worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]92client # [ 0.000000] KASLR enabled93worker2 # [ 0.000000] DMA32 empty94client # [ 0.000000] random: crng init done95worker2 # [ 0.000000] Normal empty96client # [ 0.000000] Machine model: linux,dummy-virt97worker2 # [ 0.000000] Device empty98client # [ 0.000000] efi: UEFI not found.99worker2 # [ 0.000000] Movable zone start for each node100worker2 # [ 0.000000] Early memory node ranges101client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT102worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]103client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]104worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]105client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]106client # [ 0.000000] Zone ranges:107worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000108worker2 # [ 0.000000] psci: probing for conduit method from DT.109client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]110client # [ 0.000000] DMA32 empty111worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.112client # [ 0.000000] Normal empty113client # [ 0.000000] Device empty114worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs115client # [ 0.000000] Movable zone start for each node116worker2 # [ 0.000000] psci: Trusted OS migration not required117client # [ 0.000000] Early memory node ranges118worker2 # [ 0.000000] psci: SMC Calling Convention v1.1119client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]120worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)121client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]122worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296123client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000124worker2 # [ 0.000000] Detected PIPT I-cache on CPU0125client # [ 0.000000] psci: probing for conduit method from DT.126client # [ 0.000000] psci: PSCIv1.3 detected in firmware.127worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)128client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs129worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface130client # [ 0.000000] psci: Trusted OS migration not required131worker2 # [ 0.000000] CPU features: detected: Spectre-v4132client # [ 0.000000] psci: SMC Calling Convention v1.1133worker2 # [ 0.000000] CPU features: detected: Spectre-BHB134worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38135client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)136worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23137client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296138worker2 # [ 0.000000] alternatives: applying boot alternatives139client # [ 0.000000] Detected PIPT I-cache on CPU0140client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)141client # [ 0.000000] CPU features: detected: GICv3 CPU interface142client # [ 0.000000] CPU features: detected: Spectre-v4143client # [ 0.000000] CPU features: detected: Spectre-BHB144client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38145worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sc06wwlk0999388hf04bpwgqk4k0sq2n-nixos-system-worker2-test/init regInfo=/nix/store/x9v7xr8nnl06ymz6zg72v5anpb3alhd1-closure-info/registration console=ttyAMA0,115200n8 console=tty0146client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23147client # [ 0.000000] alternatives: applying boot alternatives148worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x9v7xr8nnl06ymz6zg72v5anpb3alhd1-closure-info/registration", will be passed to user space.149worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes150worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)151worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)152client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test/init regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration console=ttyAMA0,115200n8 console=tty0153worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB154worker2 # [ 0.000000] software IO TLB: area num 1.155worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB156client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration", will be passed to user space.157worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)158client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes159worker2 # [ 0.000000] Fallback order for Node 0: 0160client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)161worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216162worker2 # [ 0.000000] Policy zone: DMA163client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)164worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off165client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB166worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1167client # [ 0.000000] software IO TLB: area num 1.168worker2 # [ 0.000000] allocated 3145728 bytes of page_ext169client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB170worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages171worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups172client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)173worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.174client # [ 0.000000] Fallback order for Node 0: 0175worker2 # [ 0.000000] rcu: RCU event tracing is enabled.176client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216177client # [ 0.000000] Policy zone: DMA178worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.179worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.180client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off181worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.182client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1183worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.184client # [ 0.000000] allocated 3145728 bytes of page_ext185worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.186client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages187client # [ 0.000000] ftrace: allocated 294 pages with 4 groups188worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1189client # [ 0.000000] rcu: Hierarchical RCU implementation.190client # [ 0.000000] rcu: RCU event tracing is enabled.191worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.193worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.194client # [ 0.000000] Trampoline variant of Tasks RCU enabled.195client # [ 0.000000] Rude variant of Tasks RCU enabled.196worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.197client # [ 0.000000] Tracing variant of Tasks RCU enabled.198worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0199client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.200worker2 # [ 0.000000] GICv3: 256 SPIs implemented201worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented202worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]203client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1204worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq205worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI206client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.207worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0208worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026209worker1 # [ 0.000000] KASLR enabled210worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000211worker1 # [ 0.000000] random: crng init done212client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.213lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]214worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]215worker1 # [ 0.000000] Machine model: linux,dummy-virt216worker1 # [ 0.000000] efi: UEFI not found.217client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.218worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)219client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0220lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026221worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT222lb # [ 0.000000] KASLR enabled223client # [ 0.000000] GICv3: 256 SPIs implemented224lb # [ 0.000000] random: crng init done225client # [ 0.000000] GICv3: 0 Extended SPIs implemented226worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]227worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)228lb # [ 0.000000] Machine model: linux,dummy-virt229client # [ 0.000000] Root IRQ handler: gic_handle_irq230worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]231lb # [ 0.000000] efi: UEFI not found.232worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000233worker1 # [ 0.000000] Zone ranges:234client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI235worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000236worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]237client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0238lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT239worker1 # [ 0.000000] DMA32 empty240worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.241worker1 # [ 0.000000] Normal empty242client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000243lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]244worker1 # [ 0.000000] Device empty245client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]246lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]247worker1 # [ 0.000000] Movable zone start for each node248lb # [ 0.000000] Zone ranges:249worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns250worker1 # [ 0.000000] Early memory node ranges251lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]252client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)253worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).254worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]255lb # [ 0.000000] DMA32 empty256lb # [ 0.000000] Normal empty257lb # [ 0.000000] Device empty258worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]259client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)260worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns261lb # [ 0.000000] Movable zone start for each node262worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000263client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000264lb # [ 0.000000] Early memory node ranges265worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns266worker1 # [ 0.000000] psci: probing for conduit method from DT.267lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]268worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.269client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000270worker2 # [ 0.000032] arm-pv: using stolen time PV271worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs272lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]273client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.274worker1 # [ 0.000000] psci: Trusted OS migration not required275worker2 # [ 0.000514] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)276lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000277worker2 # [ 0.000670] Console: colour dummy device 80x25278worker1 # [ 0.000000] psci: SMC Calling Convention v1.1279lb # [ 0.000000] psci: probing for conduit method from DT.280client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns281worker2 # [ 0.000678] printk: legacy console [tty0] enabled282lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.283client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).284worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)285lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs286worker2 # [ 0.000873] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)287worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296288lb # [ 0.000000] psci: Trusted OS migration not required289client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns290worker1 # [ 0.000000] Detected PIPT I-cache on CPU0291worker2 # [ 0.000880] pid_max: default: 32768 minimum: 301292lb # [ 0.000000] psci: SMC Calling Convention v1.1293worker2 # [ 0.000975] LSM: initializing lsm=capability,landlock,yama,bpf,ima294client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns295worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)296worker2 # [ 0.001128] landlock: Up and running.297lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)298client # [ 0.000032] arm-pv: using stolen time PV299worker2 # [ 0.001131] Yama: becoming mindful.300worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface301lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296302worker1 # [ 0.000000] CPU features: detected: Spectre-v4303worker2 # [ 0.001701] LSM support for eBPF active304lb # [ 0.000000] Detected PIPT I-cache on CPU0305client # [ 0.000514] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)306worker1 # [ 0.000000] CPU features: detected: Spectre-BHB307worker2 # [ 0.001867] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)308client # [ 0.000688] Console: colour dummy device 80x25309worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38310lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)311client # [ 0.000696] printk: legacy console [tty0] enabled312worker2 # [ 0.001905] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)313worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23314lb # [ 0.000000] CPU features: detected: GICv3 CPU interface315worker2 # [ 0.002945] cacheinfo: Unable to detect cache hierarchy for CPU 0316lb # [ 0.000000] CPU features: detected: Spectre-v4317worker1 # [ 0.000000] alternatives: applying boot alternatives318client # [ 0.000883] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)319worker2 # [ 0.003655] rcu: Hierarchical SRCU implementation.320lb # [ 0.000000] CPU features: detected: Spectre-BHB321client # [ 0.000890] pid_max: default: 32768 minimum: 301322worker2 # [ 0.003659] rcu: Max phase no-delay instances is 1000.323lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38324client # [ 0.000958] LSM: initializing lsm=capability,landlock,yama,bpf,ima325worker2 # [ 0.004856] fsl-mc MSI: its@8080000 domain created326lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23327client # [ 0.001121] landlock: Up and running.328worker2 # [ 0.004946] EFI services will not be available.329client # [ 0.001124] Yama: becoming mindful.330lb # [ 0.000000] alternatives: applying boot alternatives331worker2 # [ 0.005012] smp: Bringing up secondary CPUs ...332client # [ 0.001683] LSM support for eBPF active333worker2 # [ 0.005020] smp: Brought up 1 node, 1 CPU334worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d5kbzdn7i28wfhzsv2q0mrw9hp16b4pw-nixos-system-worker1-test/init regInfo=/nix/store/j03mxcdq3gkpl9sxpbhv30jk4b26hkg9-closure-info/registration console=ttyAMA0,115200n8 console=tty0335client # [ 0.001822] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)336worker2 # [ 0.005024] SMP: Total of 1 processors activated.337worker2 # [ 0.005029] CPU: All CPU(s) started at EL1338client # [ 0.001854] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)339worker2 # [ 0.005041] CPU features: detected: Branch Target Identification340client # [ 0.003015] cacheinfo: Unable to detect cache hierarchy for CPU 0341worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/j03mxcdq3gkpl9sxpbhv30jk4b26hkg9-closure-info/registration", will be passed to user space.342worker2 # [ 0.005046] CPU features: detected: ARMv8.4 Translation Table Level343client # [ 0.003774] rcu: Hierarchical SRCU implementation.344worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes345lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yr75snp71djsm3hdcvg8c01kgvckkd38-nixos-system-lb-test/init regInfo=/nix/store/id8l0i993q7c01fvycym66biyd895hnq-closure-info/registration console=ttyAMA0,115200n8 console=tty0346client # [ 0.003779] rcu: Max phase no-delay instances is 1000.347worker2 # [ 0.005049] CPU features: detected: Instruction cache invalidation not required for I/D coherence348worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)349client # [ 0.004964] fsl-mc MSI: its@8080000 domain created350client # [ 0.005068] EFI services will not be available.351worker2 # [ 0.005052] CPU features: detected: Data cache clean to the PoU not required for I/D coherence352lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/id8l0i993q7c01fvycym66biyd895hnq-closure-info/registration", will be passed to user space.353worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)354client # [ 0.005150] smp: Bringing up secondary CPUs ...355worker2 # [ 0.005056] CPU features: detected: Common not Private translations356client # [ 0.005158] smp: Brought up 1 node, 1 CPU357lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes358worker2 # [ 0.005059] CPU features: detected: CRC32 instructions359client # [ 0.005161] SMP: Total of 1 processors activated.360worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB361client # [ 0.005166] CPU: All CPU(s) started at EL1362lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)363worker1 # [ 0.000000] software IO TLB: area num 1.364worker2 # [ 0.005062] CPU features: detected: Data cache clean to Point of Deep Persistence365client # [ 0.005177] CPU features: detected: Branch Target Identification366lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)367worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB368worker2 # [ 0.005066] CPU features: detected: Data cache clean to Point of Persistence369client # [ 0.005182] CPU features: detected: ARMv8.4 Translation Table Level370worker2 # [ 0.005069] CPU features: detected: Data independent timing control (DIT)371worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)372worker1 # [ 0.000000] Fallback order for Node 0: 0373lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB374worker2 # [ 0.005072] CPU features: detected: E0PD375client # [ 0.005185] CPU features: detected: Instruction cache invalidation not required for I/D coherence376lb # [ 0.000000] software IO TLB: area num 1.377worker2 # [ 0.005074] CPU features: detected: Enhanced Counter Virtualization378worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216379lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB380client # [ 0.005189] CPU features: detected: Data cache clean to the PoU not required for I/D coherence381worker1 # [ 0.000000] Policy zone: DMA382worker2 # [ 0.005077] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)383worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off384client # [ 0.005192] CPU features: detected: Common not Private translations385lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)386worker2 # [ 0.005081] CPU features: detected: Enhanced Virtualization Traps387lb # [ 0.000000] Fallback order for Node 0: 0388client # [ 0.005195] CPU features: detected: CRC32 instructions389worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1390worker2 # [ 0.005084] CPU features: detected: Fine Grained Traps391worker1 # [ 0.000000] allocated 3145728 bytes of page_ext392lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216393client # [ 0.005198] CPU features: detected: Data cache clean to Point of Deep Persistence394lb # [ 0.000000] Policy zone: DMA395worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages396worker2 # [ 0.005088] CPU features: detected: Generic authentication (architected QARMA5 algorithm)397worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups398lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off399worker2 # [ 0.005093] CPU features: detected: RCpc load-acquire (LDAPR)400worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.401client # [ 0.005202] CPU features: detected: Data cache clean to Point of Persistence402worker2 # [ 0.005096] CPU features: detected: LSE atomic instructions403lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1404worker1 # [ 0.000000] rcu: RCU event tracing is enabled.405lb # [ 0.000000] allocated 3145728 bytes of page_ext406worker2 # [ 0.005099] CPU features: detected: Privileged Access Never407client # [ 0.005205] CPU features: detected: Data independent timing control (DIT)408worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.409worker2 # [ 0.005102] CPU features: detected: PMUv3410client # [ 0.005208] CPU features: detected: E0PD411lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages412worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.413worker2 # [ 0.005105] CPU features: detected: RAS Extension Support414lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups415client # [ 0.005211] CPU features: detected: Enhanced Counter Virtualization416worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.417lb # [ 0.000000] rcu: Hierarchical RCU implementation.418worker2 # [ 0.005108] CPU features: detected: RASv1p1 Extension Support419client # [ 0.005214] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)420worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.421lb # [ 0.000000] rcu: RCU event tracing is enabled.422worker2 # [ 0.005110] CPU features: detected: Random Number Generator423client # [ 0.005217] CPU features: detected: Enhanced Virtualization Traps424worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.425lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.426client # [ 0.005220] CPU features: detected: Fine Grained Traps427lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.428worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1429worker2 # [ 0.005113] CPU features: detected: Speculation barrier (SB)430lb # [ 0.000000] Rude variant of Tasks RCU enabled.431client # [ 0.005223] CPU features: detected: Generic authentication (architected QARMA5 algorithm)432worker2 # [ 0.005116] CPU features: detected: Stage-2 Force Write-Back433lb # [ 0.000000] Tracing variant of Tasks RCU enabled.434worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.435client # [ 0.005228] CPU features: detected: RCpc load-acquire (LDAPR)436worker2 # [ 0.005119] CPU features: detected: TLB range maintenance instructions437lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.438client # [ 0.005231] CPU features: detected: LSE atomic instructions439worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.440worker2 # [ 0.005124] CPU features: detected: Speculative Store Bypassing Safe (SSBS)441client # [ 0.005235] CPU features: detected: Privileged Access Never442lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1443worker2 # [ 0.005165] alternatives: applying system-wide alternatives444client # [ 0.005237] CPU features: detected: PMUv3445worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.446lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.447client # [ 0.005240] CPU features: detected: RAS Extension Support448worker2 # [ 0.008089] CPU features: detected: BBM Level 2 without TLB conflict abort449worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0450worker1 # [ 0.000000] GICv3: 256 SPIs implemented451client # [ 0.005243] CPU features: detected: RASv1p1 Extension Support452lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.453worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented454client # [ 0.005245] CPU features: detected: Random Number Generator455worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq456worker2 # [ 0.008314] Memory: 1406792K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)457client # [ 0.005248] CPU features: detected: Speculation barrier (SB)458worker2 # [ 0.008720] devtmpfs: initialized459lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.460worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI461client # [ 0.005251] CPU features: detected: Stage-2 Force Write-Back462lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0463worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0464worker2 # [ 0.010664] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)465lb # [ 0.000000] GICv3: 256 SPIs implemented466client # [ 0.005254] CPU features: detected: TLB range maintenance instructions467worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000468lb # [ 0.000000] GICv3: 0 Extended SPIs implemented469worker2 # [ 0.010687] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).470worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]471lb # [ 0.000000] Root IRQ handler: gic_handle_irq472client # [ 0.005258] CPU features: detected: Speculative Store Bypassing Safe (SSBS)473worker2 # [ 0.010888] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL474lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI475client # [ 0.005297] alternatives: applying system-wide alternatives476worker2 # [ 0.010893] 0 pages in range for non-PLT usage477worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)478lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0479worker2 # [ 0.010894] 508288 pages in range for PLT usage480client # [ 0.008337] CPU features: detected: BBM Level 2 without TLB conflict abort481lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000482worker2 # [ 0.011011] pinctrl core: initialized pinctrl subsystem483worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)484lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]485worker2 # [ 0.011738] DMI not present or invalid.486worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000487client # [ 0.008540] Memory: 1406496K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)488worker2 # [ 0.015255] NET: Registered PF_NETLINK/PF_ROUTE protocol family489lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)490client # [ 0.008938] devtmpfs: initialized491worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000492worker2 # [ 0.017477] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations493client # [ 0.010764] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)494worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.495lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)496worker2 # [ 0.017667] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations497lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000498client # [ 0.010791] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).499worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns500worker2 # [ 0.017877] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations501lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000502client # [ 0.010979] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL503worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).504worker2 # [ 0.017901] audit: initializing netlink subsys (disabled)505client # [ 0.010983] 0 pages in range for non-PLT usage506lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.507client # [ 0.010984] 508288 pages in range for PLT usage508worker2 # [ 0.018497] thermal_sys: Registered thermal governor 'fair_share'509worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns510worker2 # [ 0.018499] thermal_sys: Registered thermal governor 'bang_bang'511client # [ 0.011097] pinctrl core: initialized pinctrl subsystem512lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns513client # [ 0.011893] DMI not present or invalid.514worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns515worker2 # [ 0.018502] thermal_sys: Registered thermal governor 'step_wise'516lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).517client # [ 0.015369] NET: Registered PF_NETLINK/PF_ROUTE protocol family518worker1 # [ 0.000029] arm-pv: using stolen time PV519worker2 # [ 0.018508] thermal_sys: Registered thermal governor 'user_space'520client # [ 0.017654] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations521lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns522worker2 # [ 0.018511] thermal_sys: Registered thermal governor 'power_allocator'523worker1 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)524client # [ 0.017835] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations525worker1 # [ 0.000597] Console: colour dummy device 80x25526worker2 # [ 0.018535] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1527lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns528worker1 # [ 0.000605] printk: legacy console [tty0] enabled529lb # [ 0.000028] arm-pv: using stolen time PV530worker2 # [ 0.018544] cpuidle: using governor ladder531client # [ 0.018038] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations532worker2 # [ 0.018550] cpuidle: using governor menu533client # [ 0.018060] audit: initializing netlink subsys (disabled)534worker1 # [ 0.000796] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)535lb # [ 0.000392] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)536worker2 # [ 0.018745] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.537client # [ 0.018708] thermal_sys: Registered thermal governor 'fair_share'538lb # [ 0.000572] Console: colour dummy device 80x25539worker1 # [ 0.000803] pid_max: default: 32768 minimum: 301540worker2 # [ 0.018763] ASID allocator initialised with 65536 entries541lb # [ 0.000580] printk: legacy console [tty0] enabled542client # [ 0.018710] thermal_sys: Registered thermal governor 'bang_bang'543worker1 # [ 0.000874] LSM: initializing lsm=capability,landlock,yama,bpf,ima544worker2 # [ 0.019997] Serial: AMBA PL011 UART driver545worker1 # [ 0.001048] landlock: Up and running.546client # [ 0.018713] thermal_sys: Registered thermal governor 'step_wise'547worker1 # [ 0.001050] Yama: becoming mindful.548lb # [ 0.000770] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)549worker2 # [ 0.025396] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1550worker1 # [ 0.001498] LSM support for eBPF active551client # [ 0.018719] thermal_sys: Registered thermal governor 'user_space'552lb # [ 0.000776] pid_max: default: 32768 minimum: 301553worker2 # [ 0.025552] printk: console [ttyAMA0] enabled554client # [ 0.018722] thermal_sys: Registered thermal governor 'power_allocator'555worker1 # [ 0.001640] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)556lb # [ 0.000851] LSM: initializing lsm=capability,landlock,yama,bpf,ima557lb # [ 0.000978] landlock: Up and running.558client # [ 0.018756] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1559worker1 # [ 0.001672] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)560lb # [ 0.000981] Yama: becoming mindful.561client # [ 0.018764] cpuidle: using governor ladder562lb # [ 0.001411] LSM support for eBPF active563worker1 # [ 0.002802] cacheinfo: Unable to detect cache hierarchy for CPU 0564client # [ 0.018770] cpuidle: using governor menu565worker1 # [ 0.003500] rcu: Hierarchical SRCU implementation.566lb # [ 0.001576] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)567client # [ 0.018962] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.568worker1 # [ 0.003504] rcu: Max phase no-delay instances is 1000.569lb # [ 0.001609] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)570client # [ 0.018979] ASID allocator initialised with 65536 entries571worker1 # [ 0.004794] fsl-mc MSI: its@8080000 domain created572client # [ 0.020231] Serial: AMBA PL011 UART driver573worker1 # [ 0.004879] EFI services will not be available.574lb # [ 0.002979] cacheinfo: Unable to detect cache hierarchy for CPU 0575worker1 # [ 0.004961] smp: Bringing up secondary CPUs ...576lb # [ 0.003692] rcu: Hierarchical SRCU implementation.577client # [ 0.025635] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1578worker1 # [ 0.004970] smp: Brought up 1 node, 1 CPU579lb # [ 0.003696] rcu: Max phase no-delay instances is 1000.580client # [ 0.025799] printk: console [ttyAMA0] enabled581worker1 # [ 0.004973] SMP: Total of 1 processors activated.582lb # [ 0.004919] fsl-mc MSI: its@8080000 domain created583worker1 # [ 0.004979] CPU: All CPU(s) started at EL1584lb # [ 0.005017] EFI services will not be available.585worker2 # [ 0.144042] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages586worker1 # [ 0.004989] CPU features: detected: Branch Target Identification587lb # [ 0.005097] smp: Bringing up secondary CPUs ...588worker2 # [ 0.144059] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page589lb # [ 0.005104] smp: Brought up 1 node, 1 CPU590worker1 # [ 0.004994] CPU features: detected: ARMv8.4 Translation Table Level591lb # [ 0.005107] SMP: Total of 1 processors activated.592worker2 # [ 0.144064] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages593lb # [ 0.005113] CPU: All CPU(s) started at EL1594worker1 # [ 0.004997] CPU features: detected: Instruction cache invalidation not required for I/D coherence595worker2 # [ 0.144069] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page596lb # [ 0.005124] CPU features: detected: Branch Target Identification597worker2 # [ 0.144073] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages598worker1 # [ 0.005001] CPU features: detected: Data cache clean to the PoU not required for I/D coherence599lb # [ 0.005129] CPU features: detected: ARMv8.4 Translation Table Level600worker2 # [ 0.144077] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page601worker1 # [ 0.005004] CPU features: detected: Common not Private translations602lb # [ 0.005132] CPU features: detected: Instruction cache invalidation not required for I/D coherence603worker1 # [ 0.005008] CPU features: detected: CRC32 instructions604worker2 # [ 0.144082] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages605worker2 # [ 0.144086] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page606lb # [ 0.005135] CPU features: detected: Data cache clean to the PoU not required for I/D coherence607worker1 # [ 0.005010] CPU features: detected: Data cache clean to Point of Deep Persistence608lb # [ 0.005139] CPU features: detected: Common not Private translations609worker1 # [ 0.005014] CPU features: detected: Data cache clean to Point of Persistence610lb # [ 0.005142] CPU features: detected: CRC32 instructions611worker1 # [ 0.005017] CPU features: detected: Data independent timing control (DIT)612worker1 # [ 0.005020] CPU features: detected: E0PD613worker2 # [ 0.151224] fbcon: Taking over console614lb # [ 0.005145] CPU features: detected: Data cache clean to Point of Deep Persistence615worker2 # [ 0.151239] ACPI: Interpreter disabled.616worker1 # [ 0.005023] CPU features: detected: Enhanced Counter Virtualization617lb # [ 0.005149] CPU features: detected: Data cache clean to Point of Persistence618client # [ 0.147610] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages619worker1 # [ 0.005026] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)620client # [ 0.147628] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page621lb # [ 0.005152] CPU features: detected: Data independent timing control (DIT)622lb # [ 0.005155] CPU features: detected: E0PD623worker1 # [ 0.005029] CPU features: detected: Enhanced Virtualization Traps624client # [ 0.147634] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages625lb # [ 0.005157] CPU features: detected: Enhanced Counter Virtualization626worker1 # [ 0.005032] CPU features: detected: Fine Grained Traps627client # [ 0.147638] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page628lb # [ 0.005160] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)629worker1 # [ 0.005035] CPU features: detected: Generic authentication (architected QARMA5 algorithm)630client # [ 0.147643] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages631lb # [ 0.005164] CPU features: detected: Enhanced Virtualization Traps632worker1 # [ 0.005040] CPU features: detected: RCpc load-acquire (LDAPR)633client # [ 0.147647] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page634lb # [ 0.005167] CPU features: detected: Fine Grained Traps635worker1 # [ 0.005042] CPU features: detected: LSE atomic instructions636client # [ 0.147651] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages637worker1 # [ 0.005045] CPU features: detected: Privileged Access Never638lb # [ 0.005170] CPU features: detected: Generic authentication (architected QARMA5 algorithm)639client # [ 0.147655] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page640worker1 # [ 0.005048] CPU features: detected: PMUv3641lb # [ 0.005175] CPU features: detected: RCpc load-acquire (LDAPR)642worker1 # [ 0.005050] CPU features: detected: RAS Extension Support643lb # [ 0.005178] CPU features: detected: LSE atomic instructions644worker1 # [ 0.005053] CPU features: detected: RASv1p1 Extension Support645lb # [ 0.005181] CPU features: detected: Privileged Access Never646worker1 # [ 0.005056] CPU features: detected: Random Number Generator647client # [ 0.154920] fbcon: Taking over console648lb # [ 0.005184] CPU features: detected: PMUv3649worker2 # [ 0.160213] iommu: Default domain type: Translated650client # [ 0.154934] ACPI: Interpreter disabled.651worker1 # [ 0.005058] CPU features: detected: Speculation barrier (SB)652lb # [ 0.005186] CPU features: detected: RAS Extension Support653worker2 # [ 0.160223] iommu: DMA domain TLB invalidation policy: strict mode654worker1 # [ 0.005061] CPU features: detected: Stage-2 Force Write-Back655client # [ 0.156771] iommu: Default domain type: Translated656lb # [ 0.005189] CPU features: detected: RASv1p1 Extension Support657worker2 # [ 0.160562] SCSI subsystem initialized658worker1 # [ 0.005064] CPU features: detected: TLB range maintenance instructions659client # [ 0.156781] iommu: DMA domain TLB invalidation policy: strict mode660lb # [ 0.005192] CPU features: detected: Random Number Generator661lb # [ 0.005194] CPU features: detected: Speculation barrier (SB)662worker2 # [ 0.162444] usbcore: registered new interface driver usbfs663worker1 # [ 0.005068] CPU features: detected: Speculative Store Bypassing Safe (SSBS)664client # [ 0.158420] SCSI subsystem initialized665lb # [ 0.005197] CPU features: detected: Stage-2 Force Write-Back666worker2 # [ 0.162476] usbcore: registered new interface driver hub667worker2 # [ 0.162501] usbcore: registered new device driver usb668lb # [ 0.005200] CPU features: detected: TLB range maintenance instructions669worker2 # [ 0.162805] pps_core: LinuxPPS API ver. 1 registered670lb # [ 0.005205] CPU features: detected: Speculative Store Bypassing Safe (SSBS)671worker2 # [ 0.162811] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>672lb # [ 0.005243] alternatives: applying system-wide alternatives673worker2 # [ 0.162821] PTP clock support registered674worker2 # [ 0.162872] EDAC MC: Ver: 3.0.0675lb # [ 0.008212] CPU features: detected: BBM Level 2 without TLB conflict abort676worker2 # [ 0.167456] scmi_core: SCMI protocol bus registered677lb # [ 0.008404] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)678worker2 # [ 0.168432] FPGA manager framework679lb # [ 0.008788] devtmpfs: initialized680worker2 # [ 0.169374] vgaarb: loaded681client # [ 0.163934] usbcore: registered new interface driver usbfs682lb # [ 0.010611] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)683client # [ 0.163966] usbcore: registered new interface driver hub684worker2 # [ 0.169997] clocksource: Switched to clocksource arch_sys_counter685client # [ 0.163984] usbcore: registered new device driver usb686lb # [ 0.010633] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).687client # [ 0.164258] pps_core: LinuxPPS API ver. 1 registered688lb # [ 0.010821] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL689worker1 # [ 0.005103] alternatives: applying system-wide alternatives690lb # [ 0.010825] 0 pages in range for non-PLT usage691client # [ 0.164264] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>692lb # [ 0.010826] 508288 pages in range for PLT usage693worker1 # [ 0.007973] CPU features: detected: BBM Level 2 without TLB conflict abort694client # [ 0.164275] PTP clock support registered695client # [ 0.164323] EDAC MC: Ver: 3.0.0696lb # [ 0.010916] pinctrl core: initialized pinctrl subsystem697lb # [ 0.011675] DMI not present or invalid.698client # [ 0.168861] scmi_core: SCMI protocol bus registered699worker2 # [ 0.173835] VFS: Disk quotas dquot_6.6.0700worker1 # [ 0.008128] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)701lb # [ 0.015004] NET: Registered PF_NETLINK/PF_ROUTE protocol family702worker1 # [ 0.008498] devtmpfs: initialized703worker2 # [ 0.173871] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)704client # [ 0.169798] FPGA manager framework705client # [ 0.170692] vgaarb: loaded706worker1 # [ 0.010337] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)707client # [ 0.171324] clocksource: Switched to clocksource arch_sys_counter708worker1 # [ 0.010359] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).709worker1 # [ 0.010545] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL710worker2 # [ 0.177454] netfs: FS-Cache loaded711worker1 # [ 0.010549] 0 pages in range for non-PLT usage712worker2 # [ 0.177572] pnp: PnP ACPI: disabled713worker1 # [ 0.010550] 508288 pages in range for PLT usage714worker1 # [ 0.010661] pinctrl core: initialized pinctrl subsystem715worker1 # [ 0.011566] DMI not present or invalid.716worker1 # [ 0.014730] NET: Registered PF_NETLINK/PF_ROUTE protocol family717client # [ 0.175041] VFS: Disk quotas dquot_6.6.0718worker1 # [ 0.016976] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations719client # [ 0.175075] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)720worker1 # [ 0.017151] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations721worker2 # [ 0.181459] NET: Registered PF_INET protocol family722worker1 # [ 0.017354] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations723worker2 # [ 0.181718] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)724worker1 # [ 0.017375] audit: initializing netlink subsys (disabled)725lb # [ 0.017247] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations726client # [ 0.178733] netfs: FS-Cache loaded727worker1 # [ 0.017928] thermal_sys: Registered thermal governor 'fair_share'728client # [ 0.178857] pnp: PnP ACPI: disabled729lb # [ 0.017428] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations730worker1 # [ 0.017930] thermal_sys: Registered thermal governor 'bang_bang'731lb # [ 0.017624] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations732worker1 # [ 0.017934] thermal_sys: Registered thermal governor 'step_wise'733lb # [ 0.017645] audit: initializing netlink subsys (disabled)734worker1 # [ 0.017939] thermal_sys: Registered thermal governor 'user_space'735lb # [ 0.018223] thermal_sys: Registered thermal governor 'fair_share'736worker1 # [ 0.017942] thermal_sys: Registered thermal governor 'power_allocator'737lb # [ 0.018225] thermal_sys: Registered thermal governor 'bang_bang'738worker1 # [ 0.017967] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1739lb # [ 0.018229] thermal_sys: Registered thermal governor 'step_wise'740client # [ 0.182697] NET: Registered PF_INET protocol family741worker1 # [ 0.017975] cpuidle: using governor ladder742lb # [ 0.018234] thermal_sys: Registered thermal governor 'user_space'743worker1 # [ 0.017981] cpuidle: using governor menu744client # [ 0.182961] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)745lb # [ 0.018237] thermal_sys: Registered thermal governor 'power_allocator'746worker1 # [ 0.018175] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.747worker1 # [ 0.018189] ASID allocator initialised with 65536 entries748lb # [ 0.018261] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1749worker1 # [ 0.019354] Serial: AMBA PL011 UART driver750lb # [ 0.018270] cpuidle: using governor ladder751lb # [ 0.018275] cpuidle: using governor menu752worker1 # [ 0.024397] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1753lb # [ 0.018479] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.754worker1 # [ 0.024553] printk: console [ttyAMA0] enabled755lb # [ 0.018493] ASID allocator initialised with 65536 entries756lb # [ 0.019637] Serial: AMBA PL011 UART driver757worker1 # [ 0.143812] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages758lb # [ 0.024817] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1759worker1 # [ 0.143829] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page760lb # [ 0.024933] printk: console [ttyAMA0] enabled761worker1 # [ 0.143834] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages762worker1 # [ 0.143838] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page763lb # [ 0.144200] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages764worker1 # [ 0.143843] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages765lb # [ 0.144216] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page766worker1 # [ 0.143847] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page767lb # [ 0.144222] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages768worker1 # [ 0.143851] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages769lb # [ 0.144226] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page770worker1 # [ 0.143856] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page771lb # [ 0.144231] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages772lb # [ 0.144235] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page773lb # [ 0.144239] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages774worker1 # [ 0.151325] fbcon: Taking over console775lb # [ 0.144244] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page776worker1 # [ 0.151339] ACPI: Interpreter disabled.777lb # [ 0.151667] fbcon: Taking over console778lb # [ 0.151681] ACPI: Interpreter disabled.779lb # [ 0.153587] iommu: Default domain type: Translated780lb # [ 0.153596] iommu: DMA domain TLB invalidation policy: strict mode781lb # [ 0.155254] SCSI subsystem initialized782worker1 # [ 0.159833] iommu: Default domain type: Translated783worker1 # [ 0.159842] iommu: DMA domain TLB invalidation policy: strict mode784worker1 # [ 0.160211] SCSI subsystem initialized785worker1 # [ 0.162248] usbcore: registered new interface driver usbfs786lb # [ 0.160553] usbcore: registered new interface driver usbfs787worker1 # [ 0.162281] usbcore: registered new interface driver hub788lb # [ 0.160591] usbcore: registered new interface driver hub789worker1 # [ 0.162307] usbcore: registered new device driver usb790lb # [ 0.160609] usbcore: registered new device driver usb791worker1 # [ 0.162576] pps_core: LinuxPPS API ver. 1 registered792lb # [ 0.160912] pps_core: LinuxPPS API ver. 1 registered793worker1 # [ 0.162582] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>794lb # [ 0.160918] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>795worker1 # [ 0.162592] PTP clock support registered796worker1 # [ 0.162639] EDAC MC: Ver: 3.0.0797lb # [ 0.160928] PTP clock support registered798lb # [ 0.160977] EDAC MC: Ver: 3.0.0799worker1 # [ 0.167264] scmi_core: SCMI protocol bus registered800lb # [ 0.165573] scmi_core: SCMI protocol bus registered801worker1 # [ 0.168224] FPGA manager framework802worker1 # [ 0.169148] vgaarb: loaded803lb # [ 0.166561] FPGA manager framework804worker2 # [ 0.210574] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)805lb # [ 0.167550] vgaarb: loaded806worker2 # [ 0.210615] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)807lb # [ 0.168170] clocksource: Switched to clocksource arch_sys_counter808worker1 # [ 0.169776] clocksource: Switched to clocksource arch_sys_counter809worker1 # [ 0.170348] VFS: Disk quotas dquot_6.6.0810worker2 # [ 0.210642] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)811worker2 # [ 0.210715] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)812worker1 # [ 0.171605] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)813worker2 # [ 0.210860] TCP: Hash tables configured (established 16384 bind 16384)814worker1 # [ 0.172756] netfs: FS-Cache loaded815worker1 # [ 0.172877] pnp: PnP ACPI: disabled816worker2 # [ 0.210936] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)817lb # [ 0.171945] VFS: Disk quotas dquot_6.6.0818worker2 # [ 0.210970] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)819lb # [ 0.171975] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)820worker2 # [ 0.211006] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)821client # [ 0.212671] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)822worker2 # [ 0.211091] NET: Registered PF_UNIX/PF_LOCAL protocol family823client # [ 0.212720] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)824worker2 # [ 0.211114] NET: Registered PF_XDP protocol family825worker2 # [ 0.211130] PCI: CLS 0 bytes, default 64826client # [ 0.212750] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)827worker2 # [ 0.211365] Trying to unpack rootfs image as initramfs...828lb # [ 0.175707] netfs: FS-Cache loaded829client # [ 0.212827] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)830lb # [ 0.175830] pnp: PnP ACPI: disabled831client # [ 0.212971] TCP: Hash tables configured (established 16384 bind 16384)832client # [ 0.213060] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)833client # [ 0.213105] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)834client # [ 0.213141] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)835worker1 # [ 0.178630] NET: Registered PF_INET protocol family836client # [ 0.213240] NET: Registered PF_UNIX/PF_LOCAL protocol family837lb # [ 0.179638] NET: Registered PF_INET protocol family838client # [ 0.213263] NET: Registered PF_XDP protocol family839worker1 # [ 0.178890] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)840lb # [ 0.179909] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)841client # [ 0.213281] PCI: CLS 0 bytes, default 64842client # [ 0.213525] Trying to unpack rootfs image as initramfs...843worker2 # [ 0.227547] kvm [1]: HYP mode not available844client # [ 0.229006] kvm [1]: HYP mode not available845worker1 # [ 0.207434] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846worker1 # [ 0.207477] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847worker1 # [ 0.207504] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848worker1 # [ 0.207574] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849worker1 # [ 0.207718] TCP: Hash tables configured (established 16384 bind 16384)850worker1 # [ 0.207813] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851lb # [ 0.208700] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)852worker1 # [ 0.207849] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)853lb # [ 0.208746] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)854worker1 # [ 0.207884] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)855lb # [ 0.208774] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)856worker1 # [ 0.207972] NET: Registered PF_UNIX/PF_LOCAL protocol family857lb # [ 0.208844] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)858worker1 # [ 0.207996] NET: Registered PF_XDP protocol family859worker1 # [ 0.208014] PCI: CLS 0 bytes, default 64860lb # [ 0.208987] TCP: Hash tables configured (established 16384 bind 16384)861worker1 # [ 0.208264] Trying to unpack rootfs image as initramfs...862lb # [ 0.209068] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)863lb # [ 0.209104] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)864lb # [ 0.209141] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.209236] NET: Registered PF_UNIX/PF_LOCAL protocol family866lb # [ 0.209258] NET: Registered PF_XDP protocol family867lb # [ 0.209276] PCI: CLS 0 bytes, default 64868lb # [ 0.209523] Trying to unpack rootfs image as initramfs...869worker1 # [ 0.223473] kvm [1]: HYP mode not available870lb # [ 0.225853] kvm [1]: HYP mode not available871worker2 # [ 0.318541] Initialise system trusted keyrings872worker2 # [ 0.319362] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.320642] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.321408] 9p: Installing v9fs 9p2000 file system support875client # [ 0.319867] Initialise system trusted keyrings876client # [ 0.320614] workingset: timestamp_bits=42 max_order=19 bucket_order=0877client # [ 0.321912] squashfs: version 4.0 (2009/01/31) Phillip Lougher878client # [ 0.322653] 9p: Installing v9fs 9p2000 file system support879worker2 # [ 0.341998] Key type asymmetric registered880worker2 # [ 0.350061] Asymmetric key parser 'x509' registered881client # [ 0.346613] Key type asymmetric registered882worker2 # [ 0.350140] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)883client # [ 0.346638] Asymmetric key parser 'x509' registered884worker2 # [ 0.352225] io scheduler mq-deadline registered885worker2 # [ 0.352236] io scheduler kyber registered886client # [ 0.346706] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)887worker2 # [ 0.362124] pl061_gpio 9030000.pl061: PL061 GPIO chip registered888worker1 # [ 0.314331] Initialise system trusted keyrings889client # [ 0.355391] io scheduler mq-deadline registered890worker1 # [ 0.315075] workingset: timestamp_bits=42 max_order=19 bucket_order=0891client # [ 0.355411] io scheduler kyber registered892worker1 # [ 0.316333] squashfs: version 4.0 (2009/01/31) Phillip Lougher893client # [ 0.360571] pl061_gpio 9030000.pl061: PL061 GPIO chip registered894worker2 # [ 0.363472] ledtrig-cpu: registered to indicate activity on CPUs895worker2 # [ 0.363875] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:896worker1 # [ 0.317112] 9p: Installing v9fs 9p2000 file system support897worker2 # [ 0.363892] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000898lb # [ 0.316719] Initialise system trusted keyrings899lb # [ 0.317461] workingset: timestamp_bits=42 max_order=19 bucket_order=0900worker2 # [ 0.363905] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000901lb # [ 0.318768] squashfs: version 4.0 (2009/01/31) Phillip Lougher902worker2 # [ 0.363915] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000903worker2 # [ 0.363937] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits904lb # [ 0.319550] 9p: Installing v9fs 9p2000 file system support905client # [ 0.361868] ledtrig-cpu: registered to indicate activity on CPUs906worker2 # [ 0.363959] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]907client # [ 0.362227] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:908worker2 # [ 0.364032] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00909worker2 # [ 0.364042] pci_bus 0000:00: root bus resource [bus 00-ff]910client # [ 0.362245] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000911worker2 # [ 0.364048] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]912client # [ 0.362257] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000913worker2 # [ 0.364054] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]914client # [ 0.362265] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000915worker2 # [ 0.364059] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]916client # [ 0.362285] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits917worker2 # [ 0.364148] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint918worker2 # [ 0.364584] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint919client # [ 0.362305] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]920worker2 # [ 0.364770] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]921client # [ 0.362378] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00922worker2 # [ 0.364787] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]923client # [ 0.362388] pci_bus 0000:00: root bus resource [bus 00-ff]924worker2 # [ 0.364817] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]925client # [ 0.362393] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]926worker2 # [ 0.364833] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]927client # [ 0.362398] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]928worker2 # [ 0.365289] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint929client # [ 0.362403] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]930worker2 # [ 0.365470] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]931client # [ 0.362460] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint932worker2 # [ 0.365486] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]933client # [ 0.362906] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint934worker2 # [ 0.365517] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]935client # [ 0.363093] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]936worker2 # [ 0.365977] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint937client # [ 0.363109] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]938client # [ 0.363139] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]939worker1 # [ 0.337693] Key type asymmetric registered940client # [ 0.363155] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]941worker1 # [ 0.337716] Asymmetric key parser 'x509' registered942client # [ 0.363628] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint943client # [ 0.363813] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]944client # [ 0.363828] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]945worker2 # [ 0.386403] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]946client # [ 0.363858] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]947worker2 # [ 0.386422] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]948worker2 # [ 0.386452] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]949client # [ 0.364317] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint950client # [ 0.364499] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]951worker2 # [ 0.386907] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint952client # [ 0.364515] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]953worker2 # [ 0.387089] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]954client # [ 0.364544] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955worker2 # [ 0.387105] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]956worker2 # [ 0.387136] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]957client # [ 0.365017] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint958worker1 # [ 0.345811] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)959client # [ 0.365202] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]960worker2 # [ 0.387591] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint961worker1 # [ 0.346829] io scheduler mq-deadline registered962client # [ 0.365217] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]963worker2 # [ 0.387772] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]964worker1 # [ 0.346840] io scheduler kyber registered965client # [ 0.365250] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966worker2 # [ 0.387788] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]967worker2 # [ 0.387818] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]968client # [ 0.365752] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint969client # [ 0.365937] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]970worker2 # [ 0.388275] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint971client # [ 0.365952] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]972worker2 # [ 0.388459] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]973lb # [ 0.348281] Key type asymmetric registered974client # [ 0.365981] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975lb # [ 0.348306] Asymmetric key parser 'x509' registered976worker2 # [ 0.388474] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]977worker2 # [ 0.388503] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978lb # [ 0.348367] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)979client # [ 0.366432] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint980lb # [ 0.350412] io scheduler mq-deadline registered981client # [ 0.366614] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]982worker2 # [ 0.388952] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint983lb # [ 0.350423] io scheduler kyber registered984client # [ 0.366630] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]985worker2 # [ 0.389135] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]986client # [ 0.366659] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]987worker2 # [ 0.389150] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]988worker2 # [ 0.389180] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]989client # [ 0.367115] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint990worker2 # [ 0.389196] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]991client # [ 0.367299] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]992client # [ 0.367315] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]993worker2 # [ 0.389685] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint994worker2 # [ 0.389875] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]995worker2 # [ 0.389905] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker1 # [ 0.357929] pl061_gpio 9030000.pl061: PL061 GPIO chip registered997worker1 # [ 0.359281] ledtrig-cpu: registered to indicate activity on CPUs998worker1 # [ 0.359656] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:999lb # [ 0.360300] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1000worker1 # [ 0.359673] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001001worker1 # [ 0.359684] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001002worker1 # [ 0.359693] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001003lb # [ 0.361728] ledtrig-cpu: registered to indicate activity on CPUs1004client # [ 0.407402] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1005worker1 # [ 0.359713] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1006lb # [ 0.362139] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1007client # [ 0.407427] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1008worker1 # [ 0.359735] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1009lb # [ 0.362156] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001010client # [ 0.407925] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1011worker1 # [ 0.359814] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001012worker2 # [ 0.414415] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1013lb # [ 0.362169] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001014client # [ 0.408119] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1015worker1 # [ 0.359823] pci_bus 0000:00: root bus resource [bus 00-ff]1016worker2 # [ 0.414616] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1017worker1 # [ 0.359829] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1018client # [ 0.408149] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019lb # [ 0.362178] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001020worker2 # [ 0.414646] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021worker1 # [ 0.359833] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1022client # [ 0.408602] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1023lb # [ 0.362205] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1024worker2 # [ 0.415039] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1025client # [ 0.408789] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1026worker1 # [ 0.359838] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1027worker2 # [ 0.415221] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1028lb # [ 0.362228] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1029client # [ 0.408819] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker1 # [ 0.359895] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1031lb # [ 0.362303] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001032worker2 # [ 0.415474] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1033client # [ 0.409229] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1034lb # [ 0.362313] pci_bus 0000:00: root bus resource [bus 00-ff]1035worker1 # [ 0.360331] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1036worker2 # [ 0.415740] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1037client # [ 0.409414] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1038worker1 # [ 0.360519] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1039worker2 # [ 0.415757] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1040lb # [ 0.362319] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1041worker1 # [ 0.360536] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1042client # [ 0.409663] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1043lb # [ 0.362324] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1044worker2 # [ 0.415787] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker1 # [ 0.360577] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046client # [ 0.409946] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1047lb # [ 0.362329] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1048worker2 # [ 0.416244] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1049worker1 # [ 0.360593] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1050client # [ 0.409962] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1051worker2 # [ 0.416427] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1052lb # [ 0.362384] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1053client # [ 0.409991] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054worker2 # [ 0.416443] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1055worker1 # [ 0.361056] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1056lb # [ 0.362835] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1057worker1 # [ 0.361239] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1058worker2 # [ 0.416472] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059client # [ 0.410448] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1060lb # [ 0.363020] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1061worker1 # [ 0.361254] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1062client # [ 0.410631] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1063worker2 # [ 0.417047] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1064lb # [ 0.363036] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1065client # [ 0.410647] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker1 # [ 0.361284] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067worker2 # [ 0.417059] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1068lb # [ 0.363065] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069client # [ 0.410677] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070worker1 # [ 0.361728] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1071lb # [ 0.363081] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1072worker2 # [ 0.417066] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1073client # [ 0.411259] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1074lb # [ 0.363540] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1075worker1 # [ 0.381388] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1076worker2 # [ 0.417111] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1077client # [ 0.411270] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1078worker1 # [ 0.381407] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1079lb # [ 0.363724] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1080worker2 # [ 0.417159] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1081lb # [ 0.363739] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1082client # [ 0.411275] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1083worker1 # [ 0.381437] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker2 # [ 0.417205] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1085lb # [ 0.363769] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086client # [ 0.411319] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1087worker2 # [ 0.417253] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1088worker2 # [ 0.417300] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1089worker1 # [ 0.383980] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1090worker2 # [ 0.417349] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1091lb # [ 0.382305] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1092worker1 # [ 0.384171] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1093lb # [ 0.382498] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1094worker1 # [ 0.384186] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.417400] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1096lb # [ 0.382514] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker1 # [ 0.384219] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker2 # [ 0.417448] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1099lb # [ 0.382543] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker1 # [ 0.384690] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1101worker2 # [ 0.417494] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1102lb # [ 0.383009] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1103worker1 # [ 0.384873] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1104worker2 # [ 0.417564] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1105lb # [ 0.383190] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1106worker1 # [ 0.384889] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1107lb # [ 0.383205] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1108worker2 # [ 0.417610] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1109worker1 # [ 0.384918] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110lb # [ 0.383237] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker2 # [ 0.417632] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1112worker1 # [ 0.385374] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1113worker2 # [ 0.417653] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1114lb # [ 0.383689] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1115worker1 # [ 0.385559] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1116lb # [ 0.383869] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1117worker2 # [ 0.417676] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1118worker1 # [ 0.385574] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1119lb # [ 0.383884] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1120worker2 # [ 0.417700] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1121worker1 # [ 0.385603] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122lb # [ 0.383913] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.417734] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1124client # [ 0.435405] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1125worker2 # [ 0.417757] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1126client # [ 0.435454] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1127worker2 # [ 0.417779] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1128client # [ 0.435500] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1129worker2 # [ 0.417801] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1130client # [ 0.435548] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1131worker2 # [ 0.417823] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1132worker2 # [ 0.417846] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1133client # [ 0.435595] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1134worker2 # [ 0.417869] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1135client # [ 0.435643] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1136worker2 # [ 0.417891] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1137client # [ 0.435691] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1138worker2 # [ 0.417913] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1139client # [ 0.435738] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1140worker2 # [ 0.417934] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1141lb # [ 0.396445] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1142worker2 # [ 0.417956] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1143client # [ 0.435858] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1144lb # [ 0.396642] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1145worker2 # [ 0.417978] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1146lb # [ 0.396659] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1147client # [ 0.435905] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1148lb # [ 0.396688] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149client # [ 0.435927] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1150client # [ 0.435948] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1151lb # [ 0.397145] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1152lb # [ 0.397328] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1153client # [ 0.435970] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1154lb # [ 0.397344] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1155client # [ 0.435995] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1156lb # [ 0.397373] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157client # [ 0.436016] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1158lb # [ 0.397388] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1159client # [ 0.436038] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1160lb # [ 0.397837] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1161client # [ 0.436059] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1162worker1 # [ 0.402119] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1163lb # [ 0.398022] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1164client # [ 0.436081] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1165worker1 # [ 0.402318] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1166lb # [ 0.398052] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167worker1 # [ 0.402334] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1168client # [ 0.436103] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1169worker2 # [ 0.458041] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1170lb # [ 0.398521] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1171worker1 # [ 0.402363] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172client # [ 0.436127] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1173worker2 # [ 0.458107] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1174lb # [ 0.398708] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1175worker1 # [ 0.402380] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1176client # [ 0.436148] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1177worker2 # [ 0.458134] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1178lb # [ 0.398737] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179client # [ 0.436170] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1180worker1 # [ 0.402869] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1181worker2 # [ 0.458157] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1182client # [ 0.436191] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1183lb # [ 0.399125] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1184worker1 # [ 0.403060] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker2 # [ 0.458191] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1186client # [ 0.436213] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1187lb # [ 0.399308] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1188worker1 # [ 0.403089] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189worker2 # [ 0.458201] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1190client # [ 0.436234] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1191lb # [ 0.399550] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1192worker2 # [ 0.458206] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1193worker1 # [ 0.403547] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1194client # [ 0.436255] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1195lb # [ 0.399800] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1196worker2 # [ 0.459407] pci 0000:00:0a.0: enabling device (0000 -> 0002)1197worker1 # [ 0.403737] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1198client # [ 0.436276] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1199lb # [ 0.399817] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1200worker1 # [ 0.403767] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1201client # [ 0.436298] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1202lb # [ 0.399846] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203client # [ 0.436319] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1204worker1 # [ 0.404173] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1205client # [ 0.436341] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1206worker1 # [ 0.404355] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1207client # [ 0.436371] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1208worker1 # [ 0.404613] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1209client # [ 0.436380] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1210worker1 # [ 0.404883] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1211client # [ 0.436385] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1212worker1 # [ 0.404900] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1213client # [ 0.437598] pci 0000:00:0a.0: enabling device (0000 -> 0002)1214worker1 # [ 0.404930] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1215worker1 # [ 0.405391] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1216worker1 # [ 0.405574] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1217worker1 # [ 0.405590] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1218worker1 # [ 0.405619] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1219lb # [ 0.420339] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1220worker1 # [ 0.406202] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1221lb # [ 0.420529] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1222worker1 # [ 0.406213] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1223lb # [ 0.420544] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1224worker1 # [ 0.406219] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1225lb # [ 0.420573] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1226worker1 # [ 0.406263] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1227lb # [ 0.421140] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1228worker1 # [ 0.406310] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1229lb # [ 0.421151] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1230worker1 # [ 0.406356] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1231lb # [ 0.421157] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1232worker1 # [ 0.406402] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1233lb # [ 0.421200] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1234worker1 # [ 0.406449] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1235lb # [ 0.421246] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1236worker1 # [ 0.406496] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1237lb # [ 0.421292] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1238worker1 # [ 0.406541] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1239lb # [ 0.421337] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1240worker1 # [ 0.406588] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1241lb # [ 0.421383] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1242worker2 # [ 0.484653] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1243worker1 # [ 0.406634] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1244lb # [ 0.421429] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1245worker1 # [ 0.406701] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1246lb # [ 0.421475] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1247worker1 # [ 0.406747] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1248lb # [ 0.421522] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1249worker1 # [ 0.406769] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1250lb # [ 0.421568] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1251worker1 # [ 0.406790] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1252lb # [ 0.421655] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1253worker1 # [ 0.406820] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1254lb # [ 0.421701] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1255worker1 # [ 0.406845] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1256lb # [ 0.421723] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1257worker1 # [ 0.406867] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1258client # [ 0.485729] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1259worker1 # [ 0.406889] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1260lb # [ 0.421744] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1261lb # [ 0.421765] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1262worker1 # [ 0.406910] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1263worker2 # [ 0.490619] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1264lb # [ 0.421788] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1265worker1 # [ 0.406933] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266worker1 # [ 0.406955] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1267lb # [ 0.421810] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1268worker2 # [ 0.492685] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1269worker1 # [ 0.406977] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1270lb # [ 0.421832] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1271worker1 # [ 0.406999] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1272lb # [ 0.421853] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1273worker1 # [ 0.407020] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1274lb # [ 0.421875] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1275worker1 # [ 0.407042] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1276lb # [ 0.421897] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1277worker1 # [ 0.407063] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1278lb # [ 0.421918] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1279worker1 # [ 0.407084] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1280lb # [ 0.421941] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1281worker1 # [ 0.407105] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1282lb # [ 0.421962] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1283worker1 # [ 0.407126] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1284lb # [ 0.421983] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1285worker1 # [ 0.407148] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1286lb # [ 0.422004] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1287worker1 # [ 0.407169] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1288lb # [ 0.422026] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1289worker1 # [ 0.407191] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1290lb # [ 0.422047] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1291worker1 # [ 0.407216] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1292lb # [ 0.422068] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1293worker1 # [ 0.407225] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1294lb # [ 0.422089] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1295worker1 # [ 0.407230] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1296lb # [ 0.422110] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1297worker2 # [ 0.502578] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1298worker1 # [ 0.408402] pci 0000:00:0a.0: enabling device (0000 -> 0002)1299lb # [ 0.422132] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1300worker2 # [ 0.504530] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1301lb # [ 0.422158] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1302lb # [ 0.422167] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1303client # [ 0.495504] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1304lb # [ 0.422172] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1305client # [ 0.498946] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1306lb # [ 0.423319] pci 0000:00:0a.0: enabling device (0000 -> 0002)1307client # [ 0.501047] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1308client # [ 0.502993] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1309worker2 # [ 0.514169] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1310worker2 # [ 0.517016] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1311worker2 # [ 0.519010] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1312worker2 # [ 0.520985] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1313client # [ 0.520668] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1314client # [ 0.522903] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1315client # [ 0.526554] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1316worker2 # [ 0.531111] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1317worker1 # [ 0.484065] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1318worker2 # [ 0.533520] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1319lb # [ 0.483498] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1320client # [ 0.529322] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1321lb # [ 0.489441] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1322lb # [ 0.491561] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1323client # [ 0.539494] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1324client # [ 0.541950] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1325worker1 # [ 0.493992] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1326worker2 # [ 0.551222] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1327worker1 # [ 0.497342] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1328worker1 # [ 0.499427] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1329worker1 # [ 0.501304] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1330worker2 # [ 0.553703] msm_serial: driver initialized1331worker2 # [ 0.553859] SuperH (H)SCI(F) driver initialized1332worker2 # [ 0.553912] STM32 USART driver initialized1333lb # [ 0.501687] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1334lb # [ 0.503587] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1335client # [ 0.555645] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1336worker1 # [ 0.511214] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1337client # [ 0.558300] msm_serial: driver initialized1338worker1 # [ 0.513308] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1339client # [ 0.558446] SuperH (H)SCI(F) driver initialized1340client # [ 0.558500] STM32 USART driver initialized1341lb # [ 0.514772] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1342worker1 # [ 0.524748] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1343lb # [ 0.521115] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1344lb # [ 0.523173] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345worker1 # [ 0.531071] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346worker1 # [ 0.532993] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1347worker2 # [ 0.584838] loop: module loaded1348worker2 # [ 0.585000] virtio_blk virtio5: 1/0/0 default/read/poll queues1349lb # [ 0.533025] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1350worker2 # [ 0.585738] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1351lb # [ 0.534839] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1352worker1 # [ 0.543188] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1353client # [ 0.591899] loop: module loaded1354client # [ 0.592062] virtio_blk virtio5: 1/0/0 default/read/poll queues1355lb # [ 0.545890] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1356client # [ 0.592799] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1357worker1 # [ 0.555018] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1358worker2 # [ 0.598592] megasas: 07.734.00.00-rc11359worker2 # [ 0.599296] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1360worker1 # [ 0.557522] msm_serial: driver initialized1361worker1 # [ 0.557658] SuperH (H)SCI(F) driver initialized1362worker1 # [ 0.557711] STM32 USART driver initialized1363worker2 # [ 0.601470] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001364lb # [ 0.556432] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1365worker2 # [ 0.601525] Intel/Sharp Extended Query Table at 0x00311366worker2 # [ 0.603102] Using buffer write method1367worker2 # [ 0.603167] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1368lb # [ 0.558997] msm_serial: driver initialized1369lb # [ 0.559164] SuperH (H)SCI(F) driver initialized1370worker2 # [ 0.604827] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001371lb # [ 0.559221] STM32 USART driver initialized1372worker2 # [ 0.604855] Intel/Sharp Extended Query Table at 0x00311373client # [ 0.603940] megasas: 07.734.00.00-rc11374client # [ 0.604650] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1375client # [ 0.606534] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001376client # [ 0.606560] Intel/Sharp Extended Query Table at 0x00311377client # [ 0.608189] Using buffer write method1378client # [ 0.608245] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1379client # [ 0.609958] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001380client # [ 0.609984] Intel/Sharp Extended Query Table at 0x00311381worker2 # [ 0.623633] Using buffer write method1382worker2 # [ 0.623666] Concatenating MTD devices:1383worker2 # [ 0.623670] (0): "0.flash"1384worker2 # [ 0.623674] (1): "0.flash"1385worker2 # [ 0.623678] into device "0.flash"1386client # [ 0.627366] Using buffer write method1387client # [ 0.627391] Concatenating MTD devices:1388client # [ 0.627395] (0): "0.flash"1389client # [ 0.627399] (1): "0.flash"1390client # [ 0.627403] into device "0.flash"1391worker1 # [ 0.588457] loop: module loaded1392lb # [ 0.587169] loop: module loaded1393worker1 # [ 0.588637] virtio_blk virtio5: 1/0/0 default/read/poll queues1394lb # [ 0.587343] virtio_blk virtio5: 1/0/0 default/read/poll queues1395worker1 # [ 0.589365] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1396lb # [ 0.588062] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397worker1 # [ 0.594333] megasas: 07.734.00.00-rc11398worker1 # [ 0.595021] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399lb # [ 0.600781] megasas: 07.734.00.00-rc11400worker1 # [ 0.606376] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001401lb # [ 0.601477] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1402worker1 # [ 0.606405] Intel/Sharp Extended Query Table at 0x00311403worker1 # [ 0.607906] Using buffer write method1404lb # [ 0.603845] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001405worker1 # [ 0.607964] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1406lb # [ 0.603876] Intel/Sharp Extended Query Table at 0x00311407lb # [ 0.605452] Using buffer write method1408lb # [ 0.605519] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1409lb # [ 0.607130] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410lb # [ 0.607158] Intel/Sharp Extended Query Table at 0x00311411worker1 # [ 0.618369] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412worker1 # [ 0.618401] Intel/Sharp Extended Query Table at 0x00311413worker1 # [ 0.619734] Using buffer write method1414worker1 # [ 0.619762] Concatenating MTD devices:1415worker1 # [ 0.619766] (0): "0.flash"1416worker1 # [ 0.619770] (1): "0.flash"1417worker1 # [ 0.619774] into device "0.flash"1418lb # [ 0.624731] Using buffer write method1419lb # [ 0.624771] Concatenating MTD devices:1420lb # [ 0.624775] (0): "0.flash"1421lb # [ 0.624779] (1): "0.flash"1422lb # [ 0.624783] into device "0.flash"1423worker2 # [ 0.840229] Freeing initrd memory: 26148K1424worker2 # [ 0.846150] tun: Universal TUN/TAP device driver, 1.61425worker2 # [ 0.849800] thunder_xcv, ver 1.01426worker2 # [ 0.849845] thunder_bgx, ver 1.01427worker2 # [ 0.849869] nicpf, ver 1.01428worker2 # [ 0.851610] e1000: Intel(R) PRO/1000 Network Driver1429worker2 # [ 0.851621] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker2 # [ 0.851646] e1000e: Intel(R) PRO/1000 Network Driver1431worker2 # [ 0.851656] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker2 # [ 0.851686] igb: Intel(R) Gigabit Ethernet Network Driver1433worker2 # [ 0.851692] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker2 # [ 0.851716] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435worker2 # [ 0.851722] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436client # [ 0.853811] Freeing initrd memory: 26160K1437worker2 # [ 0.851859] sky2: driver version 1.301438worker2 # [ 0.853437] usbcore: registered new interface driver usb-storage1439worker2 # [ 0.853519] usbcore: registered new interface driver usbserial_generic1440worker2 # [ 0.853534] usbserial: USB Serial support registered for generic1441worker2 # [ 0.854725] ehci-pci 0000:00:0a.0: EHCI Host Controller1442worker2 # [ 0.854750] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11443client # [ 0.859861] tun: Universal TUN/TAP device driver, 1.61444worker2 # [ 0.855011] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001445worker2 # [ 0.865568] hv_vmbus: registering driver hyperv_keyboard1446worker2 # [ 0.866052] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001447worker2 # [ 0.866368] hub 1-0:1.0: USB hub found1448worker2 # [ 0.866395] hub 1-0:1.0: 6 ports detected1449worker2 # [ 0.869451] rtc-pl031 9010000.pl031: registered as rtc01450worker2 # [ 0.869480] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:48:44 UTC (1789998524)1451worker2 # [ 0.869820] i2c_dev: i2c /dev entries driver1452client # [ 0.863544] thunder_xcv, ver 1.01453client # [ 0.863584] thunder_bgx, ver 1.01454client # [ 0.863607] nicpf, ver 1.01455client # [ 0.864159] e1000: Intel(R) PRO/1000 Network Driver1456client # [ 0.864166] e1000: Copyright (c) 1999-2006 Intel Corporation.1457client # [ 0.864190] e1000e: Intel(R) PRO/1000 Network Driver1458client # [ 0.864199] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1459client # [ 0.864230] igb: Intel(R) Gigabit Ethernet Network Driver1460client # [ 0.864235] igb: Copyright (c) 2007-2014 Intel Corporation.1461worker2 # [ 0.874764] sdhci: Secure Digital Host Controller Interface driver1462client # [ 0.864262] igbvf: Intel(R) Gigabit Virtual Function Network Driver1463worker2 # [ 0.874773] sdhci: Copyright(c) Pierre Ossman1464client # [ 0.864267] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1465client # [ 0.864399] sky2: driver version 1.301466worker2 # [ 0.875042] Synopsys Designware Multimedia Card Interface Driver1467worker2 # [ 0.875414] sdhci-pltfm: SDHCI platform and OF driver helper1468client # [ 0.866027] usbcore: registered new interface driver usb-storage1469worker2 # [ 0.877095] hid: raw HID events driver (C) Jiri Kosina1470client # [ 0.866107] usbcore: registered new interface driver usbserial_generic1471worker2 # [ 0.877343] usbcore: registered new interface driver usbhid1472client # [ 0.866120] usbserial: USB Serial support registered for generic1473worker2 # [ 0.877349] usbhid: USB HID core driver1474client # [ 0.866707] hv_vmbus: registering driver hyperv_keyboard1475client # [ 0.867952] ehci-pci 0000:00:0a.0: EHCI Host Controller1476client # [ 0.867976] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11477worker2 # [ 0.882940] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1478client # [ 0.868130] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001479worker2 # [ 0.884449] drop_monitor: Initializing network drop monitor service1480client # [ 0.879839] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001481client # [ 0.880149] hub 1-0:1.0: USB hub found1482worker2 # [ 0.884595] NET: Registered PF_INET6 protocol family1483client # [ 0.880165] hub 1-0:1.0: 6 ports detected1484worker2 # [ 0.887600] Segment Routing with IPv61485worker2 # [ 0.887619] In-situ OAM (IOAM) with IPv61486client # [ 0.883090] rtc-pl031 9010000.pl031: registered as rtc01487worker2 # [ 0.887649] NET: Registered PF_PACKET protocol family1488client # [ 0.883119] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:48:44 UTC (1789998524)1489worker2 # [ 0.889295] 9pnet: Installing 9P2000 support1490client # [ 0.885235] i2c_dev: i2c /dev entries driver1491worker1 # [ 0.844151] Freeing initrd memory: 26148K1492worker2 # [ 0.892066] Key type dns_resolver registered1493client # [ 0.888484] sdhci: Secure Digital Host Controller Interface driver1494client # [ 0.888494] sdhci: Copyright(c) Pierre Ossman1495client # [ 0.888759] Synopsys Designware Multimedia Card Interface Driver1496client # [ 0.889143] sdhci-pltfm: SDHCI platform and OF driver helper1497client # [ 0.890820] hid: raw HID events driver (C) Jiri Kosina1498worker1 # [ 0.850037] tun: Universal TUN/TAP device driver, 1.61499client # [ 0.891073] usbcore: registered new interface driver usbhid1500client # [ 0.891079] usbhid: USB HID core driver1501worker2 # [ 0.898333] registered taskstats version 11502lb # [ 0.851280] Freeing initrd memory: 26140K1503worker1 # [ 0.853712] thunder_xcv, ver 1.01504worker2 # [ 0.898491] Loading compiled-in X.509 certificates1505worker1 # [ 0.853756] thunder_bgx, ver 1.01506worker1 # [ 0.853779] nicpf, ver 1.01507client # [ 0.896643] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1508client # [ 0.898155] drop_monitor: Initializing network drop monitor service1509client # [ 0.898334] NET: Registered PF_INET6 protocol family1510client # [ 0.901361] Segment Routing with IPv61511client # [ 0.901389] In-situ OAM (IOAM) with IPv61512client # [ 0.901420] NET: Registered PF_PACKET protocol family1513worker2 # [ 0.906922] Demotion targets for Node 0: null1514worker2 # [ 0.907027] Key type .fscrypt registered1515worker1 # [ 0.855498] e1000: Intel(R) PRO/1000 Network Driver1516client # [ 0.903062] 9pnet: Installing 9P2000 support1517worker2 # [ 0.907033] Key type fscrypt-provisioning registered1518worker1 # [ 0.855505] e1000: Copyright (c) 1999-2006 Intel Corporation.1519lb # [ 0.857400] tun: Universal TUN/TAP device driver, 1.61520worker2 # [ 0.907128] ima: No TPM chip found, activating TPM-bypass!1521worker1 # [ 0.855530] e1000e: Intel(R) PRO/1000 Network Driver1522worker2 # [ 0.907146] ima: Allocated hash algorithm: sha11523worker1 # [ 0.855539] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1524worker2 # [ 0.907167] ima: No architecture policies found1525worker1 # [ 0.855567] igb: Intel(R) Gigabit Ethernet Network Driver1526worker1 # [ 0.855573] igb: Copyright (c) 2007-2014 Intel Corporation.1527worker2 # [ 0.911112] input: gpio-keys as /devices/platform/gpio-keys/input/input01528client # [ 0.905921] Key type dns_resolver registered1529worker1 # [ 0.855596] igbvf: Intel(R) Gigabit Virtual Function Network Driver1530worker1 # [ 0.855602] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1531worker1 # [ 0.855734] sky2: driver version 1.301532worker1 # [ 0.857398] usbcore: registered new interface driver usb-storage1533worker1 # [ 0.857488] usbcore: registered new interface driver usbserial_generic1534worker1 # [ 0.857502] usbserial: USB Serial support registered for generic1535worker1 # [ 0.858521] ehci-pci 0000:00:0a.0: EHCI Host Controller1536lb # [ 0.861126] thunder_xcv, ver 1.01537lb # [ 0.861174] thunder_bgx, ver 1.01538lb # [ 0.861200] nicpf, ver 1.01539worker1 # [ 0.858546] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11540lb # [ 0.861763] e1000: Intel(R) PRO/1000 Network Driver1541worker1 # [ 0.858828] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001542lb # [ 0.861770] e1000: Copyright (c) 1999-2006 Intel Corporation.1543lb # [ 0.861796] e1000e: Intel(R) PRO/1000 Network Driver1544lb # [ 0.861804] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1545worker1 # [ 0.869523] hv_vmbus: registering driver hyperv_keyboard1546client # [ 0.912421] registered taskstats version 11547lb # [ 0.861835] igb: Intel(R) Gigabit Ethernet Network Driver1548worker1 # [ 0.869840] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001549client # [ 0.912580] Loading compiled-in X.509 certificates1550worker1 # [ 0.870164] hub 1-0:1.0: USB hub found1551lb # [ 0.861841] igb: Copyright (c) 2007-2014 Intel Corporation.1552worker1 # [ 0.870221] hub 1-0:1.0: 6 ports detected1553lb # [ 0.861864] igbvf: Intel(R) Gigabit Virtual Function Network Driver1554lb # [ 0.861870] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1555lb # [ 0.862008] sky2: driver version 1.301556worker1 # [ 0.873457] rtc-pl031 9010000.pl031: registered as rtc01557lb # [ 0.863675] usbcore: registered new interface driver usb-storage1558worker1 # [ 0.873485] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:48:44 UTC (1789998524)1559lb # [ 0.863754] usbcore: registered new interface driver usbserial_generic1560worker1 # [ 0.873778] i2c_dev: i2c /dev entries driver1561lb # [ 0.863771] usbserial: USB Serial support registered for generic1562lb # [ 0.864852] ehci-pci 0000:00:0a.0: EHCI Host Controller1563client # [ 0.921042] Demotion targets for Node 0: null1564client # [ 0.921146] Key type .fscrypt registered1565lb # [ 0.864882] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11566client # [ 0.921152] Key type fscrypt-provisioning registered1567client # [ 0.921242] ima: No TPM chip found, activating TPM-bypass!1568lb # [ 0.865093] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001569client # [ 0.921261] ima: Allocated hash algorithm: sha11570lb # [ 0.876230] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001571client # [ 0.921281] ima: No architecture policies found1572lb # [ 0.876544] hub 1-0:1.0: USB hub found1573worker2 # [ 0.928420] clk: Disabling unused clocks1574lb # [ 0.876559] hub 1-0:1.0: 6 ports detected1575client # [ 0.925378] input: gpio-keys as /devices/platform/gpio-keys/input/input01576worker2 # [ 0.928442] PM: genpd: Disabling unused power domains1577worker1 # [ 0.878632] sdhci: Secure Digital Host Controller Interface driver1578lb # [ 0.879519] hv_vmbus: registering driver hyperv_keyboard1579worker1 # [ 0.878641] sdhci: Copyright(c) Pierre Ossman1580worker1 # [ 0.878901] Synopsys Designware Multimedia Card Interface Driver1581worker1 # [ 0.879267] sdhci-pltfm: SDHCI platform and OF driver helper1582worker2 # [ 0.932600] Freeing unused kernel memory: 4736K1583worker2 # [ 0.932809] Run /init as init process1584lb # [ 0.881133] rtc-pl031 9010000.pl031: registered as rtc01585worker1 # [ 0.880890] hid: raw HID events driver (C) Jiri Kosina1586worker1 # [ 0.881130] usbcore: registered new interface driver usbhid1587lb # [ 0.881164] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:48:44 UTC (1789998524)1588worker1 # [ 0.881136] usbhid: USB HID core driver1589lb # [ 0.881478] i2c_dev: i2c /dev entries driver1590worker1 # [ 0.886595] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1591worker1 # [ 0.888061] drop_monitor: Initializing network drop monitor service1592worker1 # [ 0.888241] NET: Registered PF_INET6 protocol family1593lb # [ 0.886476] sdhci: Secure Digital Host Controller Interface driver1594lb # [ 0.886486] sdhci: Copyright(c) Pierre Ossman1595worker1 # [ 0.891274] Segment Routing with IPv61596lb # [ 0.886761] Synopsys Designware Multimedia Card Interface Driver1597worker1 # [ 0.891293] In-situ OAM (IOAM) with IPv61598worker1 # [ 0.891321] NET: Registered PF_PACKET protocol family1599lb # [ 0.887142] sdhci-pltfm: SDHCI platform and OF driver helper1600worker1 # [ 0.892982] 9pnet: Installing 9P2000 support1601lb # [ 0.891554] hid: raw HID events driver (C) Jiri Kosina1602lb # [ 0.891810] usbcore: registered new interface driver usbhid1603lb # [ 0.891817] usbhid: USB HID core driver1604worker1 # [ 0.895864] Key type dns_resolver registered1605lb # [ 0.894769] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1606worker2 # [ 0.948147] systemd[1]: Successfully made /usr/ read-only.1607lb # [ 0.897465] drop_monitor: Initializing network drop monitor service1608lb # [ 0.897613] NET: Registered PF_INET6 protocol family1609client # [ 0.943448] clk: Disabling unused clocks1610lb # [ 0.899578] Segment Routing with IPv61611client # [ 0.943471] PM: genpd: Disabling unused power domains1612lb # [ 0.899599] In-situ OAM (IOAM) with IPv61613lb # [ 0.899654] NET: Registered PF_PACKET protocol family1614client # [ 0.947715] Freeing unused kernel memory: 4736K1615client # [ 0.947912] Run /init as init process1616worker1 # [ 0.902255] registered taskstats version 11617lb # [ 0.901487] 9pnet: Installing 9P2000 support1618worker1 # [ 0.902394] Loading compiled-in X.509 certificates1619lb # [ 0.903563] Key type dns_resolver registered1620worker1 # [ 0.910784] Demotion targets for Node 0: null1621worker1 # [ 0.910884] Key type .fscrypt registered1622worker1 # [ 0.910890] Key type fscrypt-provisioning registered1623worker1 # [ 0.910987] ima: No TPM chip found, activating TPM-bypass!1624worker1 # [ 0.911006] ima: Allocated hash algorithm: sha11625worker1 # [ 0.911029] ima: No architecture policies found1626lb # [ 0.910523] registered taskstats version 11627lb # [ 0.910669] Loading compiled-in X.509 certificates1628worker1 # [ 0.914951] input: gpio-keys as /devices/platform/gpio-keys/input/input01629client # [ 0.965549] systemd[1]: Successfully made /usr/ read-only.1630lb # [ 0.919028] Demotion targets for Node 0: null1631lb # [ 0.919149] Key type .fscrypt registered1632lb # [ 0.919155] Key type fscrypt-provisioning registered1633lb # [ 0.919245] ima: No TPM chip found, activating TPM-bypass!1634lb # [ 0.919264] ima: Allocated hash algorithm: sha11635lb # [ 0.919286] ima: No architecture policies found1636lb # [ 0.923384] input: gpio-keys as /devices/platform/gpio-keys/input/input01637worker1 # [ 0.932009] clk: Disabling unused clocks1638worker1 # [ 0.932040] PM: genpd: Disabling unused power domains1639worker1 # [ 0.936248] Freeing unused kernel memory: 4736K1640worker1 # [ 0.936437] Run /init as init process1641lb # [ 0.940614] clk: Disabling unused clocks1642lb # [ 0.940641] PM: genpd: Disabling unused power domains1643lb # [ 0.944829] Freeing unused kernel memory: 4736K1644lb # [ 0.945035] Run /init as init process1645worker1 # [ 0.951982] systemd[1]: Successfully made /usr/ read-only.1646lb # [ 0.962437] systemd[1]: Successfully made /usr/ read-only.1647worker2 # [ 1.114055] usb 1-1: new high-speed USB device number 2 using ehci-pci1648client # [ 1.127411] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.117846] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.124265] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker2 # [ 1.263973] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652client # [ 1.279804] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653worker2 # [ 1.282647] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654worker2 # [ 1.294715] systemd[1]: Detected virtualization qemu.1655worker2 # [ 1.296746] systemd[1]: Detected architecture arm64.1656worker2 # [ 1.298763] systemd[1]: Running in initrd.1657worker2 # [ 1.301325] systemd[1]: Initializing machine ID from random generator.1658worker2 # [ 1.304273] systemd[1]: Hostname set to <worker2>.1659client # [ 1.300497] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1660client # [ 1.312376] systemd[1]: Detected virtualization qemu.1661client # [ 1.314366] systemd[1]: Detected architecture arm64.1662client # [ 1.316333] systemd[1]: Running in initrd.1663worker1 # [ 1.270131] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11664client # [ 1.318944] systemd[1]: Initializing machine ID from random generator.1665lb # [ 1.273924] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11666client # [ 1.321936] systemd[1]: Hostname set to <client>.1667worker1 # [ 1.286810] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668worker1 # [ 1.298776] systemd[1]: Detected virtualization qemu.1669worker1 # [ 1.300817] systemd[1]: Detected architecture arm64.1670worker2 # [ 1.350200] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01671worker1 # [ 1.302781] systemd[1]: Running in initrd.1672worker1 # [ 1.305415] systemd[1]: Initializing machine ID from random generator.1673lb # [ 1.297191] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1674lb # [ 1.306491] systemd[1]: Detected virtualization qemu.1675worker1 # [ 1.308452] systemd[1]: Hostname set to <worker1>.1676lb # [ 1.307933] systemd[1]: Detected architecture arm64.1677lb # [ 1.309432] systemd[1]: Running in initrd.1678lb # [ 1.311274] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.313385] systemd[1]: Hostname set to <lb>.1680client # [ 1.367622] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker1 # [ 1.358048] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.360437] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker2 # [ 1.470072] usb 1-2: new high-speed USB device number 3 using ehci-pci1684client # [ 1.487407] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.481862] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.480265] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.567570] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker2 # [ 1.627960] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21689worker2 # [ 1.630706] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01690client # [ 1.627052] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691client # [ 1.647929] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692worker2 # [ 1.657635] systemd[1]: Queued start job for default target Initrd Default Target.1693client # [ 1.653055] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694lb # [ 1.610134] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695worker1 # [ 1.612307] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696worker2 # [ 1.668418] systemd[1]: Created slice Slice /system/modprobe.1697worker2 # [ 1.669640] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1698worker2 # [ 1.671116] systemd[1]: Expecting device /dev/disk/by-label/nixos...1699worker2 # [ 1.671155] systemd[1]: Reached target Path Units.1700worker2 # [ 1.671178] systemd[1]: Reached target Slice Units.1701worker2 # [ 1.671199] systemd[1]: Reached target Swaps.1702worker2 # [ 1.671219] systemd[1]: Reached target Timer Units.1703worker2 # [ 1.671432] systemd[1]: Listening on D-Bus System Message Bus Socket.1704worker2 # [ 1.671591] systemd[1]: Listening on Journal Socket (/dev/log).1705worker2 # [ 1.671741] systemd[1]: Listening on Journal Sockets.1706worker2 # [ 1.671879] systemd[1]: Listening on udev Control Socket.1707worker2 # [ 1.671992] systemd[1]: Listening on udev Kernel Socket.1708worker2 # [ 1.672015] systemd[1]: Reached target Socket Units.1709worker2 # [ 1.683427] systemd[1]: Starting Create List of Static Device Nodes...1710worker1 # [ 1.640206] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21711worker2 # [ 1.693940] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1712worker2 # [ 1.695316] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1713worker1 # [ 1.645338] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01714lb # [ 1.642779] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21715lb # [ 1.643069] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01716worker2 # [ 1.707365] systemd[1]: Mounting Kernel Configuration File System...1717worker2 # [ 1.726252] systemd[1]: Starting Journal Service...1718worker2 # [ 1.732560] systemd[1]: Starting Load Kernel Modules...1719worker2 # [ 1.733535] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1720client # [ 1.739633] systemd[1]: Queued start job for default target Initrd Default Target.1721worker2 # [ 1.750372] systemd[1]: Starting Coldplug All udev Devices...1722client # [ 1.749698] systemd[1]: Created slice Slice /system/modprobe.1723client # [ 1.750880] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1724client # [ 1.752277] systemd[1]: Expecting device /dev/disk/by-label/nixos...1725client # [ 1.753444] systemd[1]: Reached target Path Units.1726client # [ 1.754252] systemd[1]: Reached target Slice Units.1727client # [ 1.755080] systemd[1]: Reached target Swaps.1728client # [ 1.755885] systemd[1]: Reached target Timer Units.1729client # [ 1.756885] systemd[1]: Listening on D-Bus System Message Bus Socket.1730client # [ 1.758118] systemd[1]: Listening on Journal Socket (/dev/log).1731client # [ 1.759238] systemd[1]: Listening on Journal Sockets.1732client # [ 1.760308] systemd[1]: Listening on udev Control Socket.1733client # [ 1.761335] systemd[1]: Listening on udev Kernel Socket.1734client # [ 1.762233] systemd[1]: Reached target Socket Units.1735worker2 # [ 1.768358] systemd[1]: Finished Create List of Static Device Nodes.1736client # [ 1.764645] systemd[1]: Starting Create List of Static Device Nodes...1737worker2 # [ 1.775617] systemd-journald[73]: Collecting audit messages is disabled.1738lb # [ 1.724104] systemd[1]: Queued start job for default target Initrd Default Target.1739worker1 # [ 1.726618] systemd[1]: Queued start job for default target Initrd Default Target.1740worker2 # [ 1.778583] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1741worker2 # [ 1.779839] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1742client # [ 1.775135] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1743worker2 # [ 1.780897] systemd[1]: Mounted Kernel Configuration File System.1744client # [ 1.776404] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1745lb # [ 1.734031] systemd[1]: Created slice Slice /system/modprobe.1746worker1 # [ 1.736271] systemd[1]: Created slice Slice /system/modprobe.1747lb # [ 1.734988] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748worker1 # [ 1.737491] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1749lb # [ 1.736002] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750lb # [ 1.737059] systemd[1]: Reached target Path Units.1751worker1 # [ 1.738925] systemd[1]: Expecting device /dev/disk/by-label/nixos...1752lb # [ 1.737707] systemd[1]: Reached target Slice Units.1753worker1 # [ 1.740036] systemd[1]: Reached target Path Units.1754lb # [ 1.738353] systemd[1]: Reached target Swaps.1755worker1 # [ 1.740868] systemd[1]: Reached target Slice Units.1756lb # [ 1.738943] systemd[1]: Reached target Timer Units.1757worker1 # [ 1.741692] systemd[1]: Reached target Swaps.1758lb # [ 1.739730] systemd[1]: Listening on D-Bus System Message Bus Socket.1759worker2 # [ 1.790444] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1760worker1 # [ 1.742508] systemd[1]: Reached target Timer Units.1761lb # [ 1.740718] systemd[1]: Listening on Journal Socket (/dev/log).1762client # [ 1.784809] systemd[1]: Mounting Kernel Configuration File System...1763lb # [ 1.741580] systemd[1]: Listening on Journal Sockets.1764worker1 # [ 1.743500] systemd[1]: Listening on D-Bus System Message Bus Socket.1765lb # [ 1.742353] systemd[1]: Listening on udev Control Socket.1766worker1 # [ 1.744741] systemd[1]: Listening on Journal Socket (/dev/log).1767lb # [ 1.743150] systemd[1]: Listening on udev Kernel Socket.1768lb # [ 1.743842] systemd[1]: Reached target Socket Units.1769worker1 # [ 1.745934] systemd[1]: Listening on Journal Sockets.1770worker1 # [ 1.747018] systemd[1]: Listening on udev Control Socket.1771worker1 # [ 1.748027] systemd[1]: Listening on udev Kernel Socket.1772worker2 # [ 1.794369] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1773lb # [ 1.745777] systemd[1]: Starting Create List of Static Device Nodes...1774worker1 # [ 1.748939] systemd[1]: Reached target Socket Units.1775worker1 # [ 1.751404] systemd[1]: Starting Create List of Static Device Nodes...1776lb # [ 1.752379] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1777lb # [ 1.753283] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1778client # [ 1.805657] systemd[1]: Starting Journal Service...1779worker1 # [ 1.761990] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1780worker1 # [ 1.763137] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1781worker2 # [ 1.810485] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1782worker2 # [ 1.815122] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01783worker1 # [ 1.766465] systemd[1]: Mounting Kernel Configuration File System...1784worker2 # [ 1.815345] [drm] features: -virgl +edid -resource_blob -host_visible1785worker2 # [ 1.815355] [drm] features: -context_init1786worker2 # [ 1.816052] [drm] number of scanouts: 11787lb # [ 1.765506] systemd[1]: Mounting Kernel Configuration File System...1788worker2 # [ 1.816071] [drm] number of cap sets: 01789client # [ 1.815470] systemd[1]: Starting Load Kernel Modules...1790client # [ 1.815578] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1791worker1 # [ 1.778116] systemd[1]: Starting Journal Service...1792client # [ 1.827159] systemd[1]: Starting Coldplug All udev Devices...1793lb # [ 1.784423] systemd[1]: Starting Journal Service...1794worker2 # [ 1.834577] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1795worker2 # [ 1.836962] systemd[1]: Starting Create Static Device Nodes in /dev...1796worker1 # [ 1.790206] systemd[1]: Starting Load Kernel Modules...1797worker1 # [ 1.790315] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1798lb # [ 1.790601] systemd[1]: Starting Load Kernel Modules...1799lb # [ 1.791562] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1800client # [ 1.847024] systemd-journald[73]: Collecting audit messages is disabled.1801worker2 # [ 1.846137] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1802worker2 # [ 1.846157] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01803worker1 # [ 1.805538] systemd[1]: Starting Coldplug All udev Devices...1804lb # [ 1.805368] systemd[1]: Starting Coldplug All udev Devices...1805client # [ 1.856196] systemd[1]: Finished Create List of Static Device Nodes.1806client # [ 1.857227] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1807client # [ 1.857542] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1808worker1 # [ 1.821964] systemd[1]: Finished Create List of Static Device Nodes.1809worker1 # [ 1.823017] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1810worker1 # [ 1.823305] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1811worker1 # [ 1.823543] systemd[1]: Mounted Kernel Configuration File System.1812client # [ 1.872075] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1813worker2 # [ 1.873801] Console: switching to colour frame buffer device 160x501814lb # [ 1.829371] systemd[1]: Finished Create List of Static Device Nodes.1815worker2 # [ 1.880293] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1816lb # [ 1.830581] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1817worker1 # [ 1.834290] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1818worker2 # [ 1.883645] systemd[1]: Finished Create Static Device Nodes in /dev.1819worker2 # [ 1.883842] systemd[1]: Reached target Preparation for Local File Systems.1820worker2 # [ 1.883876] systemd[1]: Reached target Local File Systems.1821lb # [ 1.836973] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1822lb # [ 1.839782] systemd-journald[73]: Collecting audit messages is disabled.1823lb # [ 1.845320] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1824client # [ 1.892707] systemd[1]: Mounted Kernel Configuration File System.1825worker2 # [ 1.894453] systemd[1]: Starting Rule-based Manager for Device Events and Files...1826lb # [ 1.848661] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827lb # [ 1.848907] systemd[1]: Mounted Kernel Configuration File System.1828worker1 # [ 1.859563] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1829worker2 # [ 1.911757] systemd[1]: Finished Load Kernel Modules.1830worker1 # [ 1.864390] systemd-journald[73]: Collecting audit messages is disabled.1831lb # [ 1.862235] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1832worker2 # [ 1.914857] systemd[1]: Starting Apply Kernel Variables...1833client # [ 1.914727] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1834lb # [ 1.868272] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01835lb # [ 1.868493] [drm] features: -virgl +edid -resource_blob -host_visible1836lb # [ 1.868503] [drm] features: -context_init1837lb # [ 1.869192] [drm] number of scanouts: 11838lb # [ 1.869209] [drm] number of cap sets: 01839worker1 # [ 1.878095] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840worker1 # [ 1.885874] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01841worker1 # [ 1.886101] [drm] features: -virgl +edid -resource_blob -host_visible1842worker1 # [ 1.886110] [drm] features: -context_init1843worker1 # [ 1.886798] [drm] number of scanouts: 11844client # [ 1.927082] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1845worker1 # [ 1.886816] [drm] number of cap sets: 01846client # [ 1.931715] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01847client # [ 1.931948] [drm] features: -virgl +edid -resource_blob -host_visible1848client # [ 1.931957] [drm] features: -context_init1849client # [ 1.932663] [drm] number of scanouts: 11850client # [ 1.932681] [drm] number of cap sets: 01851lb # [ 1.892738] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1852client # [ 1.940029] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853worker1 # [ 1.898455] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1854lb # [ 1.896533] systemd[1]: Starting Create Static Device Nodes in /dev...1855worker1 # [ 1.906110] systemd[1]: Starting Create Static Device Nodes in /dev...1856client # [ 1.947674] systemd[1]: Starting Create Static Device Nodes in /dev...1857lb # [ 1.900692] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1858lb # [ 1.900712] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01859client # [ 1.955763] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1860client # [ 1.955784] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01861worker1 # [ 1.910257] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1862worker1 # [ 1.910276] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01863worker2 # [ 1.966177] systemd[1]: Finished Apply Kernel Variables.1864worker2 # [ 1.979485] systemd[1]: Started Journal Service.1865worker2 # [ 1.968698] systemd-modules-load[74]: Inserted module 'dm_mod'1866lb # [ 1.924546] Console: switching to colour frame buffer device 160x501867worker2 # [ 1.969944] systemd-modules-load[74]: Module 'virtio_balloon' is built in1868worker1 # [ 1.930087] Console: switching to colour frame buffer device 160x501869client # [ 1.983425] systemd[1]: Finished Create Static Device Nodes in /dev.1870client # [ 1.983621] systemd[1]: Reached target Preparation for Local File Systems.1871client # [ 1.983650] systemd[1]: Reached target Local File Systems.1872worker2 # [ 1.982238] systemd-modules-load[74]: Module 'virtio_console' is built in1873worker2 # [ 1.983369] systemd-modules-load[74]: Inserted module 'virtio_gpu'1874worker2 # [ 1.984554] systemd-modules-load[74]: Module 'virtio_rng' is built in1875client # [ 1.991713] systemd[1]: Starting Rule-based Manager for Device Events and Files...1876lb # [ 1.948716] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1877worker2 # [ 1.988278] systemd[1]: Starting Create System Files and Directories...1878lb # [ 1.953494] systemd[1]: Finished Create Static Device Nodes in /dev.1879lb # [ 1.955437] systemd[1]: Reached target Preparation for Local File Systems.1880lb # [ 1.957285] systemd[1]: Reached target Local File Systems.1881worker2 # [ 1.996613] systemd-udevd[80]: Using default interface naming scheme 'v261'.1882worker1 # [ 1.962898] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1883client # [ 1.999680] Console: switching to colour frame buffer device 160x501884client # [ 2.006417] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885worker1 # [ 1.964624] systemd[1]: Finished Create Static Device Nodes in /dev.1886worker1 # [ 1.964839] systemd[1]: Reached target Preparation for Local File Systems.1887worker1 # [ 1.964884] systemd[1]: Reached target Local File Systems.1888worker1 # [ 1.968870] systemd[1]: Starting Rule-based Manager for Device Events and Files...1889lb # [ 1.968536] systemd[1]: Starting Rule-based Manager for Device Events and Files...1890worker2 # [ 2.016647] systemd[1]: Finished Create System Files and Directories.1891lb # [ 1.980391] systemd[1]: Finished Load Kernel Modules.1892lb # [ 1.983331] systemd[1]: Starting Apply Kernel Variables...1893worker1 # [ 1.985963] systemd[1]: Finished Load Kernel Modules.1894client # [ 2.032107] systemd[1]: Finished Load Kernel Modules.1895worker1 # [ 1.990168] systemd[1]: Starting Apply Kernel Variables...1896client # [ 2.039725] systemd[1]: Starting Apply Kernel Variables...1897worker2 # [ 2.034963] systemd[1]: Started Rule-based Manager for Device Events and Files.1898client # [ 2.051145] systemd[1]: Started Journal Service.1899client # [ 2.048343] systemd-modules-load[74]: Inserted module 'dm_mod'1900client # [ 2.049750] systemd-modules-load[74]: Module 'virtio_balloon' is built in1901client # [ 2.050846] systemd-modules-load[74]: Module 'virtio_console' is built in1902client # [ 2.051891] systemd-modules-load[74]: Inserted module 'virtio_gpu'1903lb # [ 2.024489] systemd[1]: Started Journal Service.1904lb # [ 2.011355] systemd-modules-load[74]: Inserted module 'dm_mod'1905lb # [ 2.012550] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906lb # [ 2.020307] systemd-modules-load[74]: Module 'virtio_console' is built in1907client # [ 2.068413] systemd-modules-load[74]: Module 'virtio_rng' is built in1908worker1 # [ 2.024612] systemd-modules-load[74]: Inserted module 'dm_mod'1909worker1 # [ 2.043536] systemd[1]: Started Journal Service.1910lb # [ 2.028333] systemd-modules-load[74]: Inserted module 'virtio_gpu'1911client # [ 2.077027] systemd[1]: Starting Create System Files and Directories...1912worker1 # [ 2.032419] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913lb # [ 2.032246] systemd-modules-load[74]: Module 'virtio_rng' is built in1914lb # [ 2.034965] systemd[1]: Starting Create System Files and Directories...1915worker1 # [ 2.040114] systemd-modules-load[74]: Module 'virtio_console' is built in1916client # [ 2.087677] systemd-udevd[80]: Using default interface naming scheme 'v261'.1917lb # [ 2.041388] systemd[1]: Finished Apply Kernel Variables.1918worker2 # [ 2.096130] systemd[1]: Starting Virtual Console Setup...1919worker1 # [ 2.044430] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920worker1 # [ 2.048408] systemd-modules-load[74]: Module 'virtio_rng' is built in1921worker1 # [ 2.049485] systemd[1]: Starting Create System Files and Directories...1922client # [ 2.101714] systemd[1]: Finished Apply Kernel Variables.1923worker1 # [ 2.057249] systemd[1]: Finished Apply Kernel Variables.1924client # [ 2.116412] systemd[1]: Finished Create System Files and Directories.1925lb # [ 2.069313] systemd-udevd[80]: Using default interface naming scheme 'v261'.1926worker1 # [ 2.076521] systemd-udevd[80]: Using default interface naming scheme 'v261'.1927lb # [ 2.081002] systemd[1]: Finished Create System Files and Directories.1928client # [ 2.134246] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker2 # [ 2.144464] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1930worker1 # [ 2.094606] systemd[1]: Finished Create System Files and Directories.1931worker2 # [ 2.147818] systemd[1]: Finished Virtual Console Setup.1932lb # [ 2.101215] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker1 # [ 2.118048] systemd[1]: Started Rule-based Manager for Device Events and Files.1934client # [ 2.196113] systemd[1]: Starting Virtual Console Setup...1935lb # [ 2.164099] systemd[1]: Starting Virtual Console Setup...1936worker1 # [ 2.176107] systemd[1]: Starting Virtual Console Setup...1937client # [ 2.249261] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938client # [ 2.251971] systemd[1]: Finished Virtual Console Setup.1939lb # [ 2.212499] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940lb # [ 2.215874] systemd[1]: Finished Virtual Console Setup.1941worker1 # [ 2.228521] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker1 # [ 2.231826] systemd[1]: Finished Virtual Console Setup.1943worker2 # [ 2.748705] systemd[1]: Finished Coldplug All udev Devices.1944worker2 # [ 2.749661] systemd[1]: Reached target System Initialization.1945worker2 # [ 2.752103] systemd[1]: Reached target Basic System.1946client # [ 2.833252] systemd[1]: Finished Coldplug All udev Devices.1947client # [ 2.834253] systemd[1]: Reached target System Initialization.1948client # [ 2.835074] systemd[1]: Reached target Basic System.1949worker1 # [ 2.802499] systemd[1]: Finished Coldplug All udev Devices.1950worker1 # [ 2.803472] systemd[1]: Reached target System Initialization.1951worker1 # [ 2.804332] systemd[1]: Reached target Basic System.1952lb # [ 2.812162] systemd[1]: Finished Coldplug All udev Devices.1953lb # [ 2.813715] systemd[1]: Reached target System Initialization.1954lb # [ 2.814575] systemd[1]: Reached target Basic System.1955worker2 # [ 2.883838] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1956worker2 # [ 2.919405] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1957worker2 # [ 2.929395] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1958client # [ 2.983485] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1959worker2 # [ 2.999939] systemd[1]: Found device /dev/disk/by-label/nixos.1960worker2 # [ 3.002049] systemd[1]: Reached target Initrd Root Device.1961worker2 # [ 3.006828] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1962client # [ 3.012228] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1963client # [ 3.016412] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1964worker1 # [ 2.971792] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965lb # [ 2.978210] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966worker1 # [ 2.979732] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1967worker1 # [ 2.983326] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1968lb # [ 2.985757] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 3.052486] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1970lb # [ 3.003636] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1971worker2 # [ 3.060089] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1972worker2 # [ 3.062707] systemd[1]: Mounting /sysroot...1973client # [ 3.077054] systemd[1]: Found device /dev/disk/by-label/nixos.1974client # [ 3.079608] systemd[1]: Reached target Initrd Root Device.1975client # [ 3.085225] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976worker1 # [ 3.040602] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker2 # [ 3.106476] EXT4-fs (vda): mounted filesystem 2f7a45d4-390c-434d-9704-89d8651d2b81 r/w with ordered data mode. Quota mode: none.1978worker1 # [ 3.042350] systemd[1]: Reached target Initrd Root Device.1979worker2 # [ 3.097566] systemd[1]: Mounted /sysroot.1980worker1 # [ 3.045264] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981worker2 # [ 3.099380] systemd[1]: Reached target Initrd Root File System.1982worker2 # [ 3.104087] systemd[1]: Starting Mountpoints Configured in the Real Root...1983lb # [ 3.052860] systemd[1]: Found device /dev/disk/by-label/nixos.1984lb # [ 3.055702] systemd[1]: Reached target Initrd Root Device.1985lb # [ 3.061186] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986worker2 # [ 3.130407] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1987worker2 # [ 3.136087] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1988worker2 # [ 3.139381] systemd[1]: Reloading...1989client # [ 3.135761] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1990worker1 # [ 3.094813] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1991client # [ 3.144085] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992client # [ 3.148643] systemd[1]: Mounting /sysroot...1993worker1 # [ 3.102460] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker1 # [ 3.108102] systemd[1]: Mounting /sysroot...1995lb # [ 3.109233] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1996lb # [ 3.116411] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997lb # [ 3.126359] systemd[1]: Mounting /sysroot...1998client # [ 3.199305] EXT4-fs (vda): mounted filesystem efbc24ab-f24b-4b02-a40e-0beba2819421 r/w with ordered data mode. Quota mode: none.1999client # [ 3.189406] systemd[1]: Mounted /sysroot.2000worker1 # [ 3.159873] EXT4-fs (vda): mounted filesystem 6fb13776-fea5-401c-bb9b-b55dc9d947d2 r/w with ordered data mode. Quota mode: none.
loading 1,826 hidden lines…
3827worker2 # [ 29.304942] nix-grpc-daemon[779]: ts=2026-09-21T13:49:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3828worker1 # [ 29.325946] nix-grpc-daemon[777]: ts=2026-09-21T13:49:12Z level=info event=scheduler_connected addr=lb:500513829worker1 # [ 29.330989] nix-grpc-daemon[777]: ts=2026-09-21T13:49:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3830worker2 # [ 29.506040] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513831worker2 # [ 29.511213] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3832worker1 # [ 29.531928] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513833worker1 # [ 29.536984] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3834worker2 # [ 29.712536] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513835worker2 # [ 29.717686] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3836worker1 # [ 29.737606] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513837worker1 # [ 29.742631] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3838worker2 # [ 29.918738] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513839worker2 # [ 29.924178] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3840worker1 # [ 29.945377] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513841worker1 # [ 29.949444] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842lb: must succeed: curl -sf localhost:9901/clusters3843worker2 # [ 30.127160] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513844worker2 # [ 30.131424] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3845lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3846worker1 # [ 30.145987] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513847worker1 # [ 30.151070] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3848worker2 # [ 30.328258] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513849worker2 # [ 30.333264] nix-grpc-daemon[779]: ts=2026-09-21T13:49:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850worker1 # [ 30.351923] nix-grpc-daemon[777]: ts=2026-09-21T13:49:13Z level=info event=scheduler_connected addr=lb:500513851worker1 # [ 30.357300] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3852worker2 # [ 30.534430] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513853worker2 # [ 30.539703] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3854worker1 # [ 30.557984] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513855worker1 # [ 30.563085] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3856worker2 # [ 30.740846] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513857worker2 # [ 30.745990] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3858worker1 # [ 30.763959] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513859worker1 # [ 30.768893] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3860worker2 # [ 30.947106] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513861worker2 # [ 30.952355] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3862worker1 # [ 30.971378] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513863worker1 # [ 30.975562] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3864lb: must succeed: curl -sf localhost:9901/clusters3865worker2 # [ 31.155408] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513866worker2 # [ 31.159583] nix-grpc-daemon[779]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3868worker1 # [ 31.171898] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_connected addr=lb:500513869worker1 # [ 31.176940] nix-grpc-daemon[777]: ts=2026-09-21T13:49:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3870worker2 # [ 31.356829] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513871worker2 # [ 31.361873] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3872worker1 # [ 31.377603] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513873worker1 # [ 31.382529] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3874worker2 # [ 31.562913] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513875worker2 # [ 31.567906] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3876worker1 # [ 31.583336] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513877worker1 # [ 31.588609] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3878worker2 # [ 31.769013] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513879worker2 # [ 31.774142] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3880worker1 # [ 31.789197] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513881worker1 # [ 31.797142] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3882worker2 # [ 31.975098] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513883worker2 # [ 31.980472] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3884worker1 # [ 31.997165] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513885worker1 # [ 32.001220] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886worker2 # [ 32.183203] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513887worker2 # [ 32.187404] nix-grpc-daemon[779]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3888lb: must succeed: curl -sf localhost:9901/clusters3889worker1 # [ 32.197476] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_connected addr=lb:500513890worker1 # [ 32.202431] nix-grpc-daemon[777]: ts=2026-09-21T13:49:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3892worker2 # [ 32.383801] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513893worker2 # [ 32.388215] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3894worker1 # [ 32.403309] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513895worker1 # [ 32.408437] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3896worker2 # [ 32.589212] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513897worker2 # [ 32.594147] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3898worker1 # [ 32.609144] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513899worker1 # [ 32.614408] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3900worker2 # [ 32.795203] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513901worker2 # [ 32.800253] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3902worker1 # [ 32.811298] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513903worker1 # [ 32.816453] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3904worker2 # [ 33.001359] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513905worker2 # [ 33.006352] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3906worker1 # [ 33.018858] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513907worker1 # [ 33.022940] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3908worker2 # [ 33.209582] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513909worker2 # [ 33.213646] nix-grpc-daemon[779]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3910worker1 # [ 33.219500] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_connected addr=lb:500513911worker1 # [ 33.225051] nix-grpc-daemon[777]: ts=2026-09-21T13:49:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3912lb: must succeed: curl -sf localhost:9901/clusters3913lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3914worker2 # [ 33.410448] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513915worker2 # [ 33.415581] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3916worker1 # [ 33.425747] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513917worker1 # [ 33.430754] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3918worker2 # [ 33.616598] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513919worker2 # [ 33.621548] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3920worker1 # [ 33.631582] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513921worker1 # [ 33.636724] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3922worker2 # [ 33.822431] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513923worker2 # [ 33.827438] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3924worker1 # [ 33.837351] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513925worker1 # [ 33.842315] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3926worker2 # [ 34.028470] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513927worker2 # [ 34.033337] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3928worker1 # [ 34.044654] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513929worker1 # [ 34.048640] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3930worker2 # [ 34.236111] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513931worker2 # [ 34.240104] nix-grpc-daemon[779]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3932worker1 # [ 34.244842] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_connected addr=lb:500513933worker1 # [ 34.249725] nix-grpc-daemon[777]: ts=2026-09-21T13:49:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3934lb: must succeed: curl -sf localhost:9901/clusters3935worker2 # [ 34.436506] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513936worker2 # [ 34.441359] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3938worker1 # [ 34.450581] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513939worker1 # [ 34.455645] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3940worker2 # [ 34.642388] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513941worker2 # [ 34.647505] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3942worker1 # [ 34.656332] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513943worker1 # [ 34.662055] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3944worker2 # [ 34.849372] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513945worker2 # [ 34.853459] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3946worker1 # [ 34.862005] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513947worker1 # [ 34.867069] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3948worker2 # [ 35.050842] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513949worker2 # [ 35.056141] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3950worker1 # [ 35.069554] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513951worker1 # [ 35.079102] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3952worker2 # [ 35.259229] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513953worker2 # [ 35.263561] nix-grpc-daemon[779]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3954worker1 # [ 35.270162] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_connected addr=lb:500513955worker1 # [ 35.275381] nix-grpc-daemon[777]: ts=2026-09-21T13:49:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3956lb: must succeed: curl -sf localhost:9901/clusters3957worker2 # [ 35.460008] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513958worker2 # [ 35.465276] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3959lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3960worker1 # [ 35.476204] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513961worker1 # [ 35.481083] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3962worker2 # [ 35.666188] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513963worker2 # [ 35.668962] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3964worker1 # [ 35.681815] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513965worker1 # [ 35.686955] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3966worker2 # [ 35.870077] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513967worker2 # [ 35.875252] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3968worker1 # [ 35.887490] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513969worker1 # [ 35.892527] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3970worker2 # [ 36.076259] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513971worker2 # [ 36.081161] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3972worker1 # [ 36.095050] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513973worker1 # [ 36.099164] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3974worker2 # [ 36.283972] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513975worker2 # [ 36.287939] nix-grpc-daemon[779]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3976worker1 # [ 36.295276] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_connected addr=lb:500513977worker1 # [ 36.300335] nix-grpc-daemon[777]: ts=2026-09-21T13:49:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3978worker2 # [ 36.484663] nix-grpc-daemon[779]: ts=2026-09-21T13:49:20Z level=info event=scheduler_connected addr=lb:500513979worker2 # [ 36.489574] nix-grpc-daemon[779]: ts=2026-09-21T13:49:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3980lb: must succeed: curl -sf localhost:9901/clusters3981worker1 # [ 36.478813] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=scheduler_take_over3982lb # [ 36.478925] niks3-server[995]: 2026/09/21 13:49:20 INFO lead: acquired remote=192.168.1.3:437403983worker1 # [ 36.501051] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=scheduler_connected addr=lb:500513984worker1 # [ 36.505932] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3985lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)3986worker2 # [ 36.690618] nix-grpc-daemon[779]: ts=2026-09-21T13:49:20Z level=info event=scheduler_connected addr=lb:500513987worker2 # [ 36.695609] nix-grpc-daemon[779]: ts=2026-09-21T13:49:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3988worker1 # [ 36.706597] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=scheduler_connected addr=lb:500513989worker1 # [ 36.728629] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:565803990worker1 # [ 36.733718] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03991worker2 # [ 36.896601] nix-grpc-daemon[779]: ts=2026-09-21T13:49:20Z level=info event=scheduler_connected addr=lb:500513992worker1 # [ 36.859509] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=oidc_jwks provider=mock keys=13993worker1 # [ 36.863894] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:565803994worker1 # [ 36.869309] nix-grpc-daemon[777]: ts=2026-09-21T13:49:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03995lb: must succeed: curl -sf localhost:9901/clusters3996lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3997worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3998worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3999lb: must succeed: curl -sf localhost:9901/clusters4000lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4001lb: must succeed: curl -sf localhost:9901/clusters4002lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4003lb: must succeed: curl -sf localhost:9901/clusters4004lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4005lb: must succeed: curl -sf localhost:9901/clusters4006lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4007worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4008worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)4009worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4010worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4011lb: must succeed: curl -sf localhost:9901/clusters4012lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4013lb: must succeed: curl -sf localhost:9901/clusters4014lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4015worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4016worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4017worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4018worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4019worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4020worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4021worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4022worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4023(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.61 seconds)4024subtest: balancer auth: client cert, bearer token, nothing, unknown CN4025worker1 # [ 38.871352] nix-grpc-daemon[777]: ts=2026-09-21T13:49:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565924026worker1 # [ 38.890908] systemd[1]: Started Nix Daemon.4027worker1 # [ 39.181026] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4028worker1 # [ 39.206389] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)4029worker1 # [ 39.250953] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)4030worker1 # [ 39.344827] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 3/5)4031worker1 # [ 39.536390] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 537 ms (attempt 4/5)4032worker1 # [ 40.083667] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4033worker1 # [ 40.089965] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4034worker1 # [ 40.108263] nix-grpc-daemon[777]: ts=2026-09-21T13:49:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565924035worker1 # [ 40.123018] nix-grpc-daemon[777]: ts=2026-09-21T13:49:23Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4036worker2 # [ 40.205265] nix-grpc-daemon[779]: ts=2026-09-21T13:49:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544037worker2 # [ 40.225579] systemd[1]: Started Nix Daemon.4038worker2 # [ 40.478237] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4039worker2 # [ 40.502233] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)4040worker2 # [ 40.544820] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 2/5)4041worker2 # [ 40.666137] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 3/5)4042worker2 # [ 40.926164] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 577 ms (attempt 4/5)4043worker2 # [ 41.513672] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4044worker2 # [ 41.519335] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4045worker2 # [ 41.532734] nix-grpc-daemon[779]: ts=2026-09-21T13:49:25Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4046client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4047client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)4048client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4049client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4050worker2 # [ 41.984263] nix-grpc-daemon[779]: ts=2026-09-21T13:49:25Z level=info event=oidc_jwks provider=mock keys=14051worker2 # [ 41.989443] nix-grpc-daemon[779]: ts=2026-09-21T13:49:25Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:543544052worker2 # [ 42.005929] nix-grpc-daemon[779]: ts=2026-09-21T13:49:25Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:543544053worker1 # [ 41.970204] nix-grpc-daemon[777]: ts=2026-09-21T13:49:25Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:565924054worker2 # [ 42.017571] nix-grpc-daemon[779]: ts=2026-09-21T13:49:25Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4055worker1 # [ 41.982364] nix-grpc-daemon[777]: ts=2026-09-21T13:49:25Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4056client: must succeed: rm -r /var/lib/nix-grpc-store4057client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4058worker1 # [ 42.313089] nix-grpc-daemon[777]: ts=2026-09-21T13:49:25Z level=info event=denied method=QueryMissing cn=- role=none4059worker1 # [ 42.570099] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=info event=denied method=QueryMissing cn=stranger role=none4060(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.12 seconds)4061subtest: DAG build is scheduled across workers and lands in the cache4062client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24063client # don't know how to build these paths:4064client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4065worker2 # [ 42.875572] nix-grpc-daemon[779]: ts=2026-09-21T13:49:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543544066worker2 # [ 42.883567] nix-grpc-daemon[779]: ts=2026-09-21T13:49:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544067worker1 # [ 42.860302] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04068worker1 # [ 42.867635] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14069worker1 # [ 42.913193] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924070worker1 # [ 42.934617] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924071client # copying 1 paths to 'lb:50051'...4072worker1 # [ 42.951073] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924073client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4074worker1 # [ 42.977702] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4075worker1 # [ 43.017534] nix-grpc-daemon[777]: ts=2026-09-21T13:49:26Z level=info event=niks3_push_started pid=8754076worker1 # [ 43.296897] nix-grpc-daemon[875]: time=2026-09-21T13:49:26.939Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4077worker1 # [ 43.483691] nix-daemon[860]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4078worker1 # [ 43.500148] nix-daemon[860]: reaped child process 891, status = succeeded4079worker1 # [ 43.669284] nix-daemon[860]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4080worker1 # [ 43.684554] nix-daemon[860]: reaped child process 896, status = succeeded4081lb # [ 43.697306] niks3-server[995]: 2026/09/21 13:49:27 INFO Received uploads request method=POST path=/api/pending_closures4082worker1 # [ 43.723638] nix-grpc-daemon[875]: time=2026-09-21T13:49:27.367Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4083worker1 # [ 43.727217] nix-grpc-daemon[875]: time=2026-09-21T13:49:27.369Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4084lb # [ 43.769095] niks3-server[995]: 2026/09/21 13:49:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4085lb # [ 43.800089] niks3-server[995]: 2026/09/21 13:49:27 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4086lb # [ 43.816058] niks3-server[995]: 2026/09/21 13:49:27 INFO Signed narinfos id=1 count=14087worker1 # [ 43.819493] nix-grpc-daemon[875]: time=2026-09-21T13:49:27.463Z level=INFO msg="Uploading 1 narinfos"4088lb # [ 43.828191] niks3-server[995]: 2026/09/21 13:49:27 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4089lb # [ 43.835716] niks3-server[995]: 2026/09/21 13:49:27 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4090lb # [ 43.845458] niks3-server[995]: 2026/09/21 13:49:27 INFO Completed upload id=14091worker1 # [ 43.846477] nix-grpc-daemon[875]: time=2026-09-21T13:49:27.490Z level=INFO msg="Upload complete. (541ms)"4092lb # [ 43.851395] niks3-server[995]: 2026/09/21 13:49:27 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4093worker1 # [ 43.856390] nix-grpc-daemon[777]: ts=2026-09-21T13:49:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=5044094worker1 # [ 43.876361] nix-grpc-daemon[777]: ts=2026-09-21T13:49:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924095worker1 # [ 43.888215] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4096worker1 # [ 43.890998] nix-daemon[860]: reaped child process 873, status = succeeded4097client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4098client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4099worker1 # [ 43.904446] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4100client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4101client # farm-a> LOG-farm-a-t14102worker1 # [ 45.129491] nix-daemon[860]: reaped child process 900, status = succeeded4103worker1 # [ 45.289275] nix-daemon[860]: accepted connection from pid 917, user nix-grpc-daemon (trusted)4104worker1 # [ 45.306513] nix-daemon[860]: reaped child process 920, status = succeeded4105lb # [ 45.319318] niks3-server[995]: 2026/09/21 13:49:28 INFO Received uploads request method=POST path=/api/pending_closures4106worker1 # [ 45.335369] nix-grpc-daemon[875]: time=2026-09-21T13:49:28.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4107worker1 # [ 45.337612] nix-grpc-daemon[875]: time=2026-09-21T13:49:28.978Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4108lb # [ 45.366935] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4109lb # [ 45.381906] niks3-server[995]: 2026/09/21 13:49:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4110worker1 # [ 45.388110] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.031Z level=INFO msg="Uploading 1 narinfos"4111lb # [ 45.397430] niks3-server[995]: 2026/09/21 13:49:29 INFO Signed narinfos id=2 count=14112lb # [ 45.399474] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4113lb # [ 45.406366] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4114worker1 # [ 45.415501] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.059Z level=INFO msg="Upload complete. (296ms)"4115lb # [ 45.414786] niks3-server[995]: 2026/09/21 13:49:29 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4116lb # [ 45.418194] niks3-server[995]: 2026/09/21 13:49:29 INFO Completed upload id=24117lb # [ 45.420884] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4118worker1 # [ 45.434777] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14119worker1 # [ 45.449310] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04120worker1 # [ 45.453084] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14121worker1 # [ 45.457256] nix-daemon[860]: reaped child process 898, status = succeeded4122worker1 # [ 45.481590] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924123worker1 # [ 45.494856] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924124worker1 # [ 45.508705] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.152Z level=INFO msg="All 1 paths already cached"4125client # copying 1 paths to 'lb:50051'...4126worker1 # [ 45.521402] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924127client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4128worker1 # [ 45.553723] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4129worker1 # [ 45.761689] nix-daemon[860]: accepted connection from pid 926, user nix-grpc-daemon (trusted)4130worker1 # [ 45.781670] nix-daemon[860]: reaped child process 929, status = succeeded4131worker1 # [ 45.955946] nix-daemon[860]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4132worker1 # [ 45.974314] nix-daemon[860]: reaped child process 934, status = succeeded4133lb # [ 45.986774] niks3-server[995]: 2026/09/21 13:49:29 INFO Received uploads request method=POST path=/api/pending_closures4134worker1 # [ 45.997762] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.641Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4135worker1 # [ 45.999257] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.641Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4136lb # [ 46.017996] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4137worker1 # [ 46.031265] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.675Z level=INFO msg="Uploading 1 narinfos"4138lb # [ 46.030491] niks3-server[995]: 2026/09/21 13:49:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4139lb # [ 46.037145] niks3-server[995]: 2026/09/21 13:49:29 INFO Signed narinfos id=3 count=14140lb # [ 46.043601] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4141lb # [ 46.053635] niks3-server[995]: 2026/09/21 13:49:29 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4142lb # [ 46.063660] niks3-server[995]: 2026/09/21 13:49:29 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4143lb # [ 46.068336] niks3-server[995]: 2026/09/21 13:49:29 INFO Completed upload id=34144worker1 # [ 46.069148] nix-grpc-daemon[875]: time=2026-09-21T13:49:29.713Z level=INFO msg="Upload complete. (499ms)"4145worker1 # [ 46.077089] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=6244146worker1 # [ 46.090546] nix-grpc-daemon[777]: ts=2026-09-21T13:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924147worker1 # [ 46.097083] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4148worker1 # [ 46.102056] nix-daemon[860]: reaped child process 924, status = succeeded4149client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4150worker1 # [ 46.113288] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4151client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4152client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4153client # farm-b> LOG-farm-b-t14154worker1 # [ 47.756907] nix-daemon[860]: reaped child process 938, status = succeeded4155worker1 # [ 47.928136] nix-daemon[860]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4156worker1 # [ 47.945309] nix-daemon[860]: reaped child process 957, status = succeeded4157lb # [ 47.959381] niks3-server[995]: 2026/09/21 13:49:31 INFO Received uploads request method=POST path=/api/pending_closures4158worker1 # [ 47.970282] nix-grpc-daemon[875]: time=2026-09-21T13:49:31.613Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4159worker1 # [ 47.971752] nix-grpc-daemon[875]: time=2026-09-21T13:49:31.613Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4160lb # [ 47.996880] niks3-server[995]: 2026/09/21 13:49:31 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4161worker1 # [ 48.017024] nix-grpc-daemon[875]: time=2026-09-21T13:49:31.660Z level=INFO msg="Uploading 1 narinfos"4162lb # [ 48.008797] niks3-server[995]: 2026/09/21 13:49:31 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4163lb # [ 48.022329] niks3-server[995]: 2026/09/21 13:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4164lb # [ 48.031630] niks3-server[995]: 2026/09/21 13:49:31 INFO Signed narinfos id=4 count=14165lb # [ 48.039148] niks3-server[995]: 2026/09/21 13:49:31 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4166worker1 # [ 48.045859] nix-grpc-daemon[875]: time=2026-09-21T13:49:31.690Z level=INFO msg="Upload complete. (319ms)"4167lb # [ 48.045798] niks3-server[995]: 2026/09/21 13:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4168lb # [ 48.052761] niks3-server[995]: 2026/09/21 13:49:31 INFO Completed upload id=44169lb # [ 48.055593] niks3-server[995]: 2026/09/21 13:49:31 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4170worker1 # [ 48.061163] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14171worker1 # [ 48.076711] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04172worker1 # [ 48.080300] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14173worker1 # [ 48.083863] nix-daemon[860]: reaped child process 936, status = succeeded4174worker1 # [ 48.121177] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924175worker1 # [ 48.137159] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924176worker1 # [ 48.153168] nix-grpc-daemon[875]: time=2026-09-21T13:49:31.797Z level=INFO msg="All 2 paths already cached"4177client # copying 1 paths to 'lb:50051'...4178client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4179worker1 # [ 48.163822] nix-grpc-daemon[777]: ts=2026-09-21T13:49:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924180worker1 # [ 48.182758] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4181worker1 # [ 48.388160] nix-daemon[860]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4182worker1 # [ 48.406521] nix-daemon[860]: reaped child process 966, status = succeeded4183worker1 # [ 48.578740] nix-daemon[860]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4184worker1 # [ 48.595954] nix-daemon[860]: reaped child process 971, status = succeeded4185lb # [ 48.609562] niks3-server[995]: 2026/09/21 13:49:32 INFO Received uploads request method=POST path=/api/pending_closures4186worker1 # [ 48.619702] nix-grpc-daemon[875]: time=2026-09-21T13:49:32.263Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4187worker1 # [ 48.621797] nix-grpc-daemon[875]: time=2026-09-21T13:49:32.265Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4188lb # [ 48.640284] niks3-server[995]: 2026/09/21 13:49:32 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4189worker1 # [ 48.653802] nix-grpc-daemon[875]: time=2026-09-21T13:49:32.297Z level=INFO msg="Uploading 1 narinfos"4190lb # [ 48.657198] niks3-server[995]: 2026/09/21 13:49:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4191lb # [ 48.666844] niks3-server[995]: 2026/09/21 13:49:32 INFO Signed narinfos id=5 count=14192worker1 # [ 48.680167] nix-grpc-daemon[875]: time=2026-09-21T13:49:32.324Z level=INFO msg="Upload complete. (479ms)"4193lb # [ 48.679947] niks3-server[995]: 2026/09/21 13:49:32 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4194lb # [ 48.685100] niks3-server[995]: 2026/09/21 13:49:32 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4195lb # [ 48.690410] niks3-server[995]: 2026/09/21 13:49:32 INFO Completed upload id=54196worker1 # [ 48.688121] nix-grpc-daemon[777]: ts=2026-09-21T13:49:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=7524197lb # [ 48.693469] niks3-server[995]: 2026/09/21 13:49:32 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4198worker1 # [ 48.696879] nix-grpc-daemon[777]: ts=2026-09-21T13:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924199worker1 # [ 48.705487] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4200worker1 # [ 48.710625] nix-daemon[860]: reaped child process 961, status = succeeded4201client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4202worker1 # [ 48.719837] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4203client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4204client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4205client # farm-top> LOG-farm-top-t14206worker1 # [ 49.969888] nix-daemon[860]: reaped child process 975, status = succeeded4207worker1 # [ 50.147179] nix-daemon[860]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4208worker1 # [ 50.166110] nix-daemon[860]: reaped child process 994, status = succeeded4209lb # [ 50.177732] niks3-server[995]: 2026/09/21 13:49:33 INFO Received uploads request method=POST path=/api/pending_closures4210worker1 # [ 50.190270] nix-grpc-daemon[875]: time=2026-09-21T13:49:33.832Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4211worker1 # [ 50.191800] nix-grpc-daemon[875]: time=2026-09-21T13:49:33.832Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4212lb # [ 50.208719] niks3-server[995]: 2026/09/21 13:49:33 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4213lb # [ 50.224383] niks3-server[995]: 2026/09/21 13:49:33 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4214worker1 # [ 50.237956] nix-grpc-daemon[875]: time=2026-09-21T13:49:33.881Z level=INFO msg="Uploading 1 narinfos"4215lb # [ 50.237937] niks3-server[995]: 2026/09/21 13:49:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4216lb # [ 50.244740] niks3-server[995]: 2026/09/21 13:49:33 INFO Signed narinfos id=6 count=14217lb # [ 50.249977] niks3-server[995]: 2026/09/21 13:49:33 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4218lb # [ 50.258679] niks3-server[995]: 2026/09/21 13:49:33 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4219lb # [ 50.265312] niks3-server[995]: 2026/09/21 13:49:33 INFO Completed upload id=64220lb # [ 50.267602] niks3-server[995]: 2026/09/21 13:49:33 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4221worker1 # [ 50.265626] nix-grpc-daemon[875]: time=2026-09-21T13:49:33.909Z level=INFO msg="Upload complete. (334ms)"4222lb # [ 50.280831] envoy[912]: conn peer=[2001:db8:1::1]:51010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=26704223worker1 # [ 50.281213] nix-grpc-daemon[777]: ts=2026-09-21T13:49:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14224lb # [ 50.285257] envoy[912]: conn peer=[2001:db8:1::1]:54304 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=744225lb # [ 50.289922] envoy[912]: conn peer=[2001:db8:1::1]:54312 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114226worker1 # [ 50.294440] nix-daemon[860]: reaped child process 973, status = succeeded4227lb # [ 50.293377] envoy[912]: conn peer=[2001:db8:1::1]:54322 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104228lb # [ 50.297251] envoy[912]: conn peer=[2001:db8:1::1]:54330 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=4096 ms=74814229client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.74 seconds)4230client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4231client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4232client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4233client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.04 seconds)4234client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4235client # copying 3 paths...4236client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4237client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4238client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4239client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4240(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.41 seconds)4241subtest: repeat is answered Cached by the scheduler without building4242worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4243worker1 # finding garbage collector roots...4244worker1 # deleting specified paths...4245worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4246worker1 # deleting unused links...4247worker1 # note: hard linking is currently saving 0.0 KiB4248worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4249worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4250worker2 # finding garbage collector roots...4251worker2 # deleting specified paths...4252worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4253worker2 # deleting unused links...4254worker2 # note: hard linking is currently saving 0.0 KiB4255worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4256client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24257worker2 # [ 51.904774] nix-grpc-daemon[779]: ts=2026-09-21T13:49:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543544258client # don't know how to build these paths:4259client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4260worker1 # [ 51.863714] nix-grpc-daemon[777]: ts=2026-09-21T13:49:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924261worker1 # [ 51.876222] nix-grpc-daemon[875]: time=2026-09-21T13:49:35.519Z level=INFO msg="All 2 paths already cached"4262worker1 # [ 51.893319] nix-grpc-daemon[777]: ts=2026-09-21T13:49:35Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14263client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4264client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4265client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4266lb: must succeed: curl -sf localhost:9901/clusters4267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4268lb: must succeed: curl -sf localhost:9901/clusters4269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4270worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4271worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4272(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.47 seconds)4273subtest: inputs already in the cache are not sent to the scheduler4274lb: must succeed: curl -sf localhost:9901/clusters4275lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4276lb: must succeed: curl -sf localhost:9901/clusters4277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4278worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4279worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4280client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24281worker2 # [ 52.990107] nix-grpc-daemon[779]: ts=2026-09-21T13:49:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543544282client # don't know how to build these paths:4283client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4284worker2 # [ 52.999670] nix-grpc-daemon[779]: ts=2026-09-21T13:49:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544285worker1 # [ 52.979707] nix-grpc-daemon[777]: ts=2026-09-21T13:49:36Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04286worker1 # [ 52.985061] nix-grpc-daemon[777]: ts=2026-09-21T13:49:36Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14287worker1 # [ 53.025333] nix-grpc-daemon[777]: ts=2026-09-21T13:49:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924288worker1 # [ 53.041068] nix-grpc-daemon[777]: ts=2026-09-21T13:49:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924289worker1 # [ 53.057424] nix-grpc-daemon[875]: time=2026-09-21T13:49:36.700Z level=INFO msg="All 2 paths already cached"4290client # copying 1 paths to 'lb:50051'...4291client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4292worker1 # [ 53.068142] nix-grpc-daemon[777]: ts=2026-09-21T13:49:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924293worker1 # [ 53.084974] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4294worker1 # [ 53.295514] nix-daemon[860]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)4295worker1 # [ 53.313285] nix-daemon[860]: reaped child process 1026, status = succeeded4296worker1 # [ 53.487641] nix-daemon[860]: accepted connection from pid 1028, user nix-grpc-daemon (trusted)4297worker1 # [ 53.502847] nix-daemon[860]: reaped child process 1031, status = succeeded4298lb # [ 53.518707] niks3-server[995]: 2026/09/21 13:49:37 INFO Received uploads request method=POST path=/api/pending_closures4299worker1 # [ 53.531915] nix-grpc-daemon[875]: time=2026-09-21T13:49:37.175Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4300worker1 # [ 53.533534] nix-grpc-daemon[875]: time=2026-09-21T13:49:37.175Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4301lb # [ 53.548971] niks3-server[995]: 2026/09/21 13:49:37 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4302worker1 # [ 53.563353] nix-grpc-daemon[875]: time=2026-09-21T13:49:37.207Z level=INFO msg="Uploading 1 narinfos"4303lb # [ 53.563162] niks3-server[995]: 2026/09/21 13:49:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4304lb # [ 53.572550] niks3-server[995]: 2026/09/21 13:49:37 INFO Signed narinfos id=7 count=14305lb # [ 53.585318] niks3-server[995]: 2026/09/21 13:49:37 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4306lb # [ 53.590910] niks3-server[995]: 2026/09/21 13:49:37 INFO Completed upload id=74307worker1 # [ 53.591712] nix-grpc-daemon[875]: time=2026-09-21T13:49:37.235Z level=INFO msg="Upload complete. (489ms)"4308lb # [ 53.593613] niks3-server[995]: 2026/09/21 13:49:37 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4309lb # [ 53.598303] niks3-server[995]: 2026/09/21 13:49:37 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4310worker1 # [ 53.598878] nix-grpc-daemon[777]: ts=2026-09-21T13:49:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=7524311worker1 # [ 53.613520] nix-grpc-daemon[777]: ts=2026-09-21T13:49:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924312worker1 # [ 53.619267] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4313worker1 # [ 53.624630] nix-daemon[860]: reaped child process 1021, status = succeeded4314client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4315worker1 # [ 53.635689] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4316client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4317client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4318client # farm-top> LOG-farm-top-t1b4319worker1 # [ 54.476964] nix-daemon[860]: reaped child process 1035, status = succeeded4320worker1 # [ 54.651807] nix-daemon[860]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4321worker1 # [ 54.670468] nix-daemon[860]: reaped child process 1054, status = succeeded4322lb # [ 54.682605] niks3-server[995]: 2026/09/21 13:49:38 INFO Received uploads request method=POST path=/api/pending_closures4323worker1 # [ 54.696050] nix-grpc-daemon[875]: time=2026-09-21T13:49:38.337Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4324worker1 # [ 54.697580] nix-grpc-daemon[875]: time=2026-09-21T13:49:38.337Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4325lb # [ 54.715551] niks3-server[995]: 2026/09/21 13:49:38 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4326lb # [ 54.729944] niks3-server[995]: 2026/09/21 13:49:38 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4327lb # [ 54.744133] niks3-server[995]: 2026/09/21 13:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4328worker1 # [ 54.749685] nix-grpc-daemon[875]: time=2026-09-21T13:49:38.393Z level=INFO msg="Uploading 1 narinfos"4329lb # [ 54.753503] niks3-server[995]: 2026/09/21 13:49:38 INFO Signed narinfos id=8 count=14330lb # [ 54.759081] niks3-server[995]: 2026/09/21 13:49:38 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4331lb # [ 54.766434] niks3-server[995]: 2026/09/21 13:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4332lb # [ 54.773865] niks3-server[995]: 2026/09/21 13:49:38 INFO Completed upload id=84333worker1 # [ 54.773898] nix-grpc-daemon[875]: time=2026-09-21T13:49:38.418Z level=INFO msg="Upload complete. (326ms)"4334lb # [ 54.776620] niks3-server[995]: 2026/09/21 13:49:38 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4335worker1 # [ 54.789011] nix-grpc-daemon[777]: ts=2026-09-21T13:49:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14336worker1 # [ 54.801711] nix-daemon[860]: reaped child process 1033, status = succeeded4337client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.16 seconds)4338client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4339client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4340lb: must succeed: curl -sf localhost:9901/clusters4341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4342lb: must succeed: curl -sf localhost:9901/clusters4343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4344worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4345worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4346(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.79 seconds)4347subtest: two clients wanting the same drv build it once4348client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4349client # [ 55.390523] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4350client # Running as unit: dedup1.service; invocation ID: f85cbb144ec34067b9c8cf77b35562cd4351client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4352client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4353client # [ 55.658922] nix[879]: don't know how to build these paths:4354client # [ 55.659948] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4355worker1 # [ 55.610294] nix-grpc-daemon[777]: ts=2026-09-21T13:49:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565924356worker1 # [ 55.625547] nix-grpc-daemon[777]: ts=2026-09-21T13:49:39Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04357worker1 # [ 55.631563] nix-grpc-daemon[777]: ts=2026-09-21T13:49:39Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14358worker2 # [ 55.728919] nix-grpc-daemon[779]: ts=2026-09-21T13:49:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544359worker2 # [ 55.747853] nix-grpc-daemon[779]: ts=2026-09-21T13:49:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544360client # [ 55.756174] nix[879]: copying 1 paths to 'lb:50051'...4361client # [ 55.761933] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4362worker2 # [ 55.767789] nix-grpc-daemon[779]: ts=2026-09-21T13:49:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543544363worker2 # [ 55.784326] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4364worker2 # [ 55.814248] nix-grpc-daemon[779]: ts=2026-09-21T13:49:39Z level=info event=niks3_push_started pid=8754365worker2 # [ 56.079134] nix-grpc-daemon[875]: time=2026-09-21T13:49:39.721Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4366worker2 # [ 56.268617] nix-daemon[848]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4367worker2 # [ 56.285904] nix-daemon[848]: reaped child process 891, status = succeeded4368worker2 # [ 56.460946] nix-daemon[848]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4369worker2 # [ 56.476951] nix-daemon[848]: reaped child process 896, status = succeeded4370lb # [ 56.437548] niks3-server[995]: 2026/09/21 13:49:40 INFO Received uploads request method=POST path=/api/pending_closures4371worker2 # [ 56.501749] nix-grpc-daemon[875]: time=2026-09-21T13:49:40.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4372worker2 # [ 56.505009] nix-grpc-daemon[875]: time=2026-09-21T13:49:40.147Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4373client # [ 56.507005] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4374lb # [ 56.481001] niks3-server[995]: 2026/09/21 13:49:40 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4375client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4376worker2 # [ 56.545329] nix-grpc-daemon[875]: time=2026-09-21T13:49:40.189Z level=INFO msg="Uploading 1 narinfos"4377lb # [ 56.494271] niks3-server[995]: 2026/09/21 13:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4378lb # [ 56.505019] niks3-server[995]: 2026/09/21 13:49:40 INFO Signed narinfos id=9 count=14379lb # [ 56.509888] niks3-server[995]: 2026/09/21 13:49:40 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4380worker2 # [ 56.572198] nix-grpc-daemon[875]: time=2026-09-21T13:49:40.216Z level=INFO msg="Upload complete. (485ms)"4381lb # [ 56.519068] niks3-server[995]: 2026/09/21 13:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4382lb # [ 56.524531] niks3-server[995]: 2026/09/21 13:49:40 INFO Completed upload id=94383lb # [ 56.527823] niks3-server[995]: 2026/09/21 13:49:40 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4384worker2 # [ 56.580122] nix-grpc-daemon[779]: ts=2026-09-21T13:49:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=0 paths=1 nar_bytes_in=4324385worker2 # [ 56.594196] nix-grpc-daemon[779]: ts=2026-09-21T13:49:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544386worker2 # [ 56.600318] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4387worker2 # [ 56.605398] nix-daemon[848]: reaped child process 873, status = succeeded4388client # Running as unit: dedup2.service; invocation ID: 59c17d5cff004813a805ee4d08de05b24389client # [ 56.609888] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4390worker2 # [ 56.619583] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4391client # [ 56.862131] nix[895]: don't know how to build these paths:4392worker1 # [ 56.815343] nix-grpc-daemon[777]: ts=2026-09-21T13:49:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565924393client # [ 56.866300] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4394worker1 # [ 56.830656] nix-grpc-daemon[777]: ts=2026-09-21T13:49:40Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04395worker2 # [ 56.887828] nix-grpc-daemon[779]: ts=2026-09-21T13:49:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544396client # [ 56.888684] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4397client: must succeed: sleep 24398client: (finished: must succeed: sleep 2, in 2.07 seconds)4399client: waiting for success: ! systemctl is-active dedup1 dedup24400worker2 # [ 60.267608] nix-daemon[848]: reaped child process 900, status = succeeded4401lb # [ 60.279855] envoy[912]: conn peer=[2001:db8:1::1]:56178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=624402lb # [ 60.284701] envoy[912]: conn peer=[2001:db8:1::1]:56194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1783 ms=18624403worker2 # [ 60.446939] nix-daemon[848]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4404worker2 # [ 60.463891] nix-daemon[848]: reaped child process 933, status = succeeded4405lb # [ 60.423128] niks3-server[995]: 2026/09/21 13:49:44 INFO Received uploads request method=POST path=/api/pending_closures4406worker2 # [ 60.487759] nix-grpc-daemon[875]: time=2026-09-21T13:49:44.130Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4407worker2 # [ 60.489846] nix-grpc-daemon[875]: time=2026-09-21T13:49:44.130Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4408lb # [ 60.458660] niks3-server[995]: 2026/09/21 13:49:44 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4409lb # [ 60.477497] niks3-server[995]: 2026/09/21 13:49:44 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4410lb # [ 60.489089] niks3-server[995]: 2026/09/21 13:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4411worker2 # [ 60.550876] nix-grpc-daemon[875]: time=2026-09-21T13:49:44.193Z level=INFO msg="Uploading 1 narinfos"4412lb # [ 60.498913] niks3-server[995]: 2026/09/21 13:49:44 INFO Signed narinfos id=10 count=14413lb # [ 60.506071] niks3-server[995]: 2026/09/21 13:49:44 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4414lb # [ 60.519318] niks3-server[995]: 2026/09/21 13:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4415lb # [ 60.526620] niks3-server[995]: 2026/09/21 13:49:44 INFO Completed upload id=104416worker2 # [ 60.579742] nix-grpc-daemon[875]: time=2026-09-21T13:49:44.224Z level=INFO msg="Upload complete. (347ms)"4417lb # [ 60.529957] niks3-server[995]: 2026/09/21 13:49:44 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4418worker2 # [ 60.593151] nix-grpc-daemon[779]: ts=2026-09-21T13:49:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14419worker2 # [ 60.610537] nix-grpc-daemon[779]: ts=2026-09-21T13:49:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14420worker2 # [ 60.617928] nix-daemon[848]: reaped child process 898, status = succeeded4421client # [ 60.629836] systemd[1]: dedup2.service: Deactivated successfully.4422client # [ 60.636628] systemd[1]: dedup2.service: Consumed 213ms CPU time over 4.127s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4423client # [ 60.679505] systemd[1]: dedup1.service: Deactivated successfully.4424client # [ 60.682282] systemd[1]: dedup1.service: Consumed 172ms CPU time over 5.291s wall clock time, 41.3M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4425client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4426client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4427client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4428worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4429worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4430worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4431worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4432worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4433worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4434worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4435worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4436(finished: subtest: two clients wanting the same drv build it once, in 6.29 seconds)4437subtest: build hook: nix-daemon with builders = grpc://lb4438client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4439client # [ 61.844746] systemd[1]: Started Nix Daemon.4440client # [ 62.058464] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4441worker1 # [ 62.264889] nix-grpc-daemon[777]: ts=2026-09-21T13:49:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565924442worker2 # [ 62.322056] nix-grpc-daemon[779]: ts=2026-09-21T13:49:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544443worker1 # [ 62.291327] nix-grpc-daemon[777]: ts=2026-09-21T13:49:45Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04444worker1 # [ 62.299013] nix-grpc-daemon[777]: ts=2026-09-21T13:49:45Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14445worker1 # [ 62.338502] nix-grpc-daemon[777]: ts=2026-09-21T13:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924446worker1 # [ 62.358877] nix-grpc-daemon[777]: ts=2026-09-21T13:49:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924447worker1 # [ 62.379214] nix-grpc-daemon[777]: ts=2026-09-21T13:49:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924448worker1 # [ 62.395544] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4449worker1 # [ 62.640643] nix-daemon[860]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4450worker1 # [ 62.661094] nix-daemon[860]: reaped child process 1093, status = succeeded4451worker1 # [ 62.899323] nix-daemon[860]: accepted connection from pid 1095, user nix-grpc-daemon (trusted)4452worker1 # [ 62.921339] nix-daemon[860]: reaped child process 1098, status = succeeded4453lb # [ 62.938210] niks3-server[995]: 2026/09/21 13:49:46 INFO Received uploads request method=POST path=/api/pending_closures4454worker1 # [ 62.955242] nix-grpc-daemon[875]: time=2026-09-21T13:49:46.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4455worker1 # [ 62.956915] nix-grpc-daemon[875]: time=2026-09-21T13:49:46.598Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4456lb # [ 62.990768] niks3-server[995]: 2026/09/21 13:49:46 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4457worker1 # [ 63.012236] nix-grpc-daemon[875]: time=2026-09-21T13:49:46.656Z level=INFO msg="Uploading 1 narinfos"4458lb # [ 63.004868] niks3-server[995]: 2026/09/21 13:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4459lb # [ 63.017223] niks3-server[995]: 2026/09/21 13:49:46 INFO Signed narinfos id=11 count=14460lb # [ 63.019954] niks3-server[995]: 2026/09/21 13:49:46 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4461lb # [ 63.029468] niks3-server[995]: 2026/09/21 13:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4462worker1 # [ 63.037305] nix-grpc-daemon[875]: time=2026-09-21T13:49:46.681Z level=INFO msg="Upload complete. (623ms)"4463lb # [ 63.038263] niks3-server[995]: 2026/09/21 13:49:46 INFO Completed upload id=114464lb # [ 63.041579] niks3-server[995]: 2026/09/21 13:49:46 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4465worker1 # [ 63.048346] nix-grpc-daemon[777]: ts=2026-09-21T13:49:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=5124466worker1 # [ 63.066852] nix-grpc-daemon[777]: ts=2026-09-21T13:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924467worker1 # [ 63.075260] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4468worker1 # [ 63.081717] nix-daemon[860]: reaped child process 1088, status = succeeded4469worker1 # [ 63.096217] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4470worker1 # [ 64.150509] nix-daemon[860]: reaped child process 1102, status = succeeded4471worker1 # [ 64.324363] nix-daemon[860]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4472worker1 # [ 64.340293] nix-daemon[860]: reaped child process 1121, status = succeeded4473lb # [ 64.358139] niks3-server[995]: 2026/09/21 13:49:47 INFO Received uploads request method=POST path=/api/pending_closures4474worker1 # [ 64.368400] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.010Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4475worker1 # [ 64.369883] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.010Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4476lb # [ 64.394771] niks3-server[995]: 2026/09/21 13:49:48 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4477lb # [ 64.409280] niks3-server[995]: 2026/09/21 13:49:48 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4478lb # [ 64.419668] niks3-server[995]: 2026/09/21 13:49:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4479worker1 # [ 64.427343] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.070Z level=INFO msg="Uploading 1 narinfos"4480lb # [ 64.426117] niks3-server[995]: 2026/09/21 13:49:48 INFO Signed narinfos id=12 count=14481lb # [ 64.431796] niks3-server[995]: 2026/09/21 13:49:48 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4482lb # [ 64.444435] niks3-server[995]: 2026/09/21 13:49:48 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4483lb # [ 64.451097] niks3-server[995]: 2026/09/21 13:49:48 INFO Completed upload id=124484worker1 # [ 64.451277] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.095Z level=INFO msg="Upload complete. (338ms)"4485lb # [ 64.454416] niks3-server[995]: 2026/09/21 13:49:48 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4486worker1 # [ 64.465184] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14487worker2 # [ 64.522112] nix-grpc-daemon[779]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544488worker1 # [ 64.482958] nix-daemon[860]: reaped child process 1100, status = succeeded4489worker2 # [ 64.539521] nix-grpc-daemon[779]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4490worker1 # [ 64.533529] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565924491worker1 # [ 64.540568] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_out=1284492worker2 # [ 64.865272] nix-grpc-daemon[779]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:543544493worker1 # [ 64.822467] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924494worker1 # [ 64.832508] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.476Z level=INFO msg="All 1 paths already cached"4495worker1 # [ 64.850662] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04496worker1 # [ 64.858288] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14497worker1 # [ 64.867114] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924498worker1 # [ 64.879018] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924499worker1 # [ 64.893781] nix-grpc-daemon[875]: time=2026-09-21T13:49:48.537Z level=INFO msg="All 1 paths already cached"4500worker1 # [ 64.903133] nix-grpc-daemon[777]: ts=2026-09-21T13:49:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924501worker1 # [ 64.917665] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4502worker1 # [ 65.139180] nix-daemon[860]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4503worker1 # [ 65.159731] nix-daemon[860]: reaped child process 1130, status = succeeded4504worker1 # [ 65.333927] nix-daemon[860]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4505worker1 # [ 65.350220] nix-daemon[860]: reaped child process 1135, status = succeeded4506lb # [ 65.361942] niks3-server[995]: 2026/09/21 13:49:48 INFO Received uploads request method=POST path=/api/pending_closures4507worker1 # [ 65.374048] nix-grpc-daemon[875]: time=2026-09-21T13:49:49.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4508worker1 # [ 65.375503] nix-grpc-daemon[875]: time=2026-09-21T13:49:49.017Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4509lb # [ 65.389290] niks3-server[995]: 2026/09/21 13:49:49 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4510worker1 # [ 65.403787] nix-grpc-daemon[875]: time=2026-09-21T13:49:49.047Z level=INFO msg="Uploading 1 narinfos"4511lb # [ 65.403949] niks3-server[995]: 2026/09/21 13:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4512lb # [ 65.414021] niks3-server[995]: 2026/09/21 13:49:49 INFO Signed narinfos id=13 count=14513lb # [ 65.423689] niks3-server[995]: 2026/09/21 13:49:49 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4514lb # [ 65.429202] niks3-server[995]: 2026/09/21 13:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4515lb # [ 65.434125] niks3-server[995]: 2026/09/21 13:49:49 INFO Completed upload id=134516worker1 # [ 65.434223] nix-grpc-daemon[875]: time=2026-09-21T13:49:49.078Z level=INFO msg="Upload complete. (499ms)"4517lb # [ 65.436261] niks3-server[995]: 2026/09/21 13:49:49 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4518worker1 # [ 65.440278] nix-grpc-daemon[777]: ts=2026-09-21T13:49:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=6244519worker1 # [ 65.452583] nix-grpc-daemon[777]: ts=2026-09-21T13:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924520worker1 # [ 65.460277] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4521worker1 # [ 65.462843] nix-daemon[860]: reaped child process 1125, status = succeeded4522worker1 # [ 65.473066] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4523worker1 # [ 66.920961] nix-daemon[860]: reaped child process 1139, status = succeeded4524worker1 # [ 67.097134] nix-daemon[860]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)4525worker1 # [ 67.115370] nix-daemon[860]: reaped child process 1158, status = succeeded4526lb # [ 67.127214] niks3-server[995]: 2026/09/21 13:49:50 INFO Received uploads request method=POST path=/api/pending_closures4527worker1 # [ 67.140201] nix-grpc-daemon[875]: time=2026-09-21T13:49:50.781Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4528worker1 # [ 67.141732] nix-grpc-daemon[875]: time=2026-09-21T13:49:50.781Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4529lb # [ 67.163809] niks3-server[995]: 2026/09/21 13:49:50 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4530lb # [ 67.174352] niks3-server[995]: 2026/09/21 13:49:50 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4531lb # [ 67.186717] niks3-server[995]: 2026/09/21 13:49:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4532worker1 # [ 67.193369] nix-grpc-daemon[875]: time=2026-09-21T13:49:50.837Z level=INFO msg="Uploading 1 narinfos"4533lb # [ 67.194912] niks3-server[995]: 2026/09/21 13:49:50 INFO Signed narinfos id=14 count=14534lb # [ 67.201021] niks3-server[995]: 2026/09/21 13:49:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4535lb # [ 67.210932] niks3-server[995]: 2026/09/21 13:49:50 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4536lb # [ 67.218149] niks3-server[995]: 2026/09/21 13:49:50 INFO Completed upload id=144537worker1 # [ 67.218126] nix-grpc-daemon[875]: time=2026-09-21T13:49:50.862Z level=INFO msg="Upload complete. (333ms)"4538lb # [ 67.221246] niks3-server[995]: 2026/09/21 13:49:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4539worker1 # [ 67.230600] nix-grpc-daemon[777]: ts=2026-09-21T13:49:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14540worker2 # [ 67.289692] nix-grpc-daemon[779]: ts=2026-09-21T13:49:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544541worker1 # [ 67.249702] nix-daemon[860]: reaped child process 1137, status = succeeded4542worker2 # [ 67.302604] nix-grpc-daemon[779]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4543worker2 # [ 67.344079] nix-grpc-daemon[779]: ts=2026-09-21T13:49:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:543544544worker2 # [ 67.350628] nix-grpc-daemon[779]: ts=2026-09-21T13:49:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=0 paths=1 nar_bytes_out=1764545worker1 # [ 67.626468] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565924546worker1 # [ 67.634248] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924547worker1 # [ 67.641648] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.284Z level=INFO msg="All 2 paths already cached"4548worker1 # [ 67.653764] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04549worker1 # [ 67.661166] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14550worker1 # [ 67.669903] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924551worker1 # [ 67.681161] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924552worker1 # [ 67.695325] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.339Z level=INFO msg="All 2 paths already cached"4553worker1 # [ 67.704972] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924554worker1 # [ 67.719874] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4555worker1 # [ 67.921780] nix-daemon[860]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)4556worker1 # [ 67.940475] nix-daemon[860]: reaped child process 1167, status = succeeded4557worker1 # [ 68.113543] nix-daemon[860]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)4558worker1 # [ 68.130931] nix-daemon[860]: reaped child process 1172, status = succeeded4559lb # [ 68.143268] niks3-server[995]: 2026/09/21 13:49:51 INFO Received uploads request method=POST path=/api/pending_closures4560worker1 # [ 68.154481] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.798Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4561worker1 # [ 68.155922] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.798Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4562lb # [ 68.171149] niks3-server[995]: 2026/09/21 13:49:51 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4563lb # [ 68.186010] niks3-server[995]: 2026/09/21 13:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4564worker1 # [ 68.192230] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.836Z level=INFO msg="Uploading 1 narinfos"4565lb # [ 68.198857] niks3-server[995]: 2026/09/21 13:49:51 INFO Signed narinfos id=15 count=14566lb # [ 68.205617] niks3-server[995]: 2026/09/21 13:49:51 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4567lb # [ 68.215170] niks3-server[995]: 2026/09/21 13:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4568worker1 # [ 68.221435] nix-grpc-daemon[875]: time=2026-09-21T13:49:51.865Z level=INFO msg="Upload complete. (486ms)"4569lb # [ 68.221430] niks3-server[995]: 2026/09/21 13:49:51 INFO Completed upload id=154570lb # [ 68.224703] niks3-server[995]: 2026/09/21 13:49:51 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4571worker1 # [ 68.228124] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_in=7524572worker1 # [ 68.242550] nix-grpc-daemon[777]: ts=2026-09-21T13:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924573worker1 # [ 68.248858] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4574worker1 # [ 68.254455] nix-daemon[860]: reaped child process 1162, status = succeeded4575worker1 # [ 68.268761] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4576worker1 # [ 69.294709] nix-daemon[860]: reaped child process 1176, status = succeeded4577worker1 # [ 69.455622] nix-daemon[860]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)4578worker1 # [ 69.475128] nix-daemon[860]: reaped child process 1195, status = succeeded4579lb # [ 69.487428] niks3-server[995]: 2026/09/21 13:49:53 INFO Received uploads request method=POST path=/api/pending_closures4580worker1 # [ 69.528594] nix-grpc-daemon[875]: time=2026-09-21T13:49:53.171Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4581worker1 # [ 69.530777] nix-grpc-daemon[875]: time=2026-09-21T13:49:53.171Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4582lb # [ 69.558642] niks3-server[995]: 2026/09/21 13:49:53 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4583lb # [ 69.574616] niks3-server[995]: 2026/09/21 13:49:53 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4584lb # [ 69.587564] niks3-server[995]: 2026/09/21 13:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4585worker1 # [ 69.593567] nix-grpc-daemon[875]: time=2026-09-21T13:49:53.237Z level=INFO msg="Uploading 1 narinfos"4586lb # [ 69.597596] niks3-server[995]: 2026/09/21 13:49:53 INFO Signed narinfos id=16 count=14587lb # [ 69.603022] niks3-server[995]: 2026/09/21 13:49:53 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4588lb # [ 69.613659] niks3-server[995]: 2026/09/21 13:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4589lb # [ 69.619050] niks3-server[995]: 2026/09/21 13:49:53 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4590lb # [ 69.622840] niks3-server[995]: 2026/09/21 13:49:53 INFO Completed upload id=164591worker1 # [ 69.623600] nix-grpc-daemon[875]: time=2026-09-21T13:49:53.267Z level=INFO msg="Upload complete. (346ms)"4592worker1 # [ 69.635388] nix-grpc-daemon[777]: ts=2026-09-21T13:49:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14593worker2 # [ 69.691131] nix-grpc-daemon[779]: ts=2026-09-21T13:49:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544594worker2 # [ 69.705348] nix-grpc-daemon[779]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4595worker1 # [ 69.657633] nix-daemon[860]: reaped child process 1174, status = succeeded4596worker1 # [ 69.693307] nix-grpc-daemon[777]: ts=2026-09-21T13:49:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565924597worker1 # [ 69.699944] nix-grpc-daemon[777]: ts=2026-09-21T13:49:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_out=2324598client # [ 69.840753] nix-daemon[937]: reaped child process 939, status = succeeded4599client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.25 seconds)4600client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4601client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4602client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4603client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4604client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4605client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4606(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.38 seconds)4607subtest: an input only one worker has still reaches the builder4608client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4609client # this derivation will be built:4610client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4611lb # [ 70.285148] envoy[912]: conn peer=[2001:db8:1::1]:56222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=37424612client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4613client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4614client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4615client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4616worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4617worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4618worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4619worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4620client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24621lb # [ 70.290122] envoy[912]: conn peer=[2001:db8:1::1]:56208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1453 ms=49994622lb # [ 70.290245] envoy[912]: conn peer=[2001:db8:1::1]:42086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=334623lb # [ 70.290344] envoy[912]: conn peer=[2001:db8:1::1]:42078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3437 tx=1869 ms=22994624lb # [ 70.290444] envoy[912]: conn peer=[2001:db8:1::1]:42102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=2068 ms=25454625lb # [ 70.290607] envoy[912]: conn peer=[2001:db8:1::1]:42118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=724626lb # [ 70.290752] envoy[912]: conn peer=[2001:db8:1::1]:42132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2278 ms=20904627lb # [ 70.290854] envoy[912]: conn peer=[2001:db8:1::1]:42138 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=294628client # [ 71.127610] nix-daemon[937]: accepted connection from pid 1019, user root (trusted)4629client # this derivation will be built:4630client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4631client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4632worker2 # [ 71.352794] nix-grpc-daemon[779]: ts=2026-09-21T13:49:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:543544633worker1 # [ 71.307528] nix-grpc-daemon[777]: ts=2026-09-21T13:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924634worker1 # [ 71.510838] nix-daemon[860]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)4635worker1 # [ 71.529428] nix-daemon[860]: reaped child process 1205, status = succeeded4636lb # [ 71.542604] niks3-server[995]: 2026/09/21 13:49:55 INFO Received uploads request method=POST path=/api/pending_closures4637worker1 # [ 71.552297] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.195Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4638worker1 # [ 71.553857] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.195Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4639lb # [ 71.576390] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4640lb # [ 71.588320] niks3-server[995]: 2026/09/21 13:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4641worker1 # [ 71.596360] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.240Z level=INFO msg="Uploading 1 narinfos"4642lb # [ 71.597940] niks3-server[995]: 2026/09/21 13:49:55 INFO Signed narinfos id=17 count=14643lb # [ 71.603427] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4644lb # [ 71.611876] niks3-server[995]: 2026/09/21 13:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4645worker1 # [ 71.618445] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.262Z level=INFO msg="Upload complete. (305ms)"4646lb # [ 71.620104] niks3-server[995]: 2026/09/21 13:49:55 INFO Completed upload id=174647lb # [ 71.623894] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4648worker1 # [ 71.641575] nix-grpc-daemon[777]: ts=2026-09-21T13:49:55Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04649worker1 # [ 71.647904] nix-grpc-daemon[777]: ts=2026-09-21T13:49:55Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14650worker2 # [ 71.743337] nix-grpc-daemon[779]: ts=2026-09-21T13:49:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544651worker2 # [ 71.759540] nix-grpc-daemon[779]: ts=2026-09-21T13:49:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544652client # copying 2 paths to 'lb:50051'...4653client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4654worker2 # [ 71.779576] nix-grpc-daemon[779]: ts=2026-09-21T13:49:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543544655client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4656worker2 # [ 71.795582] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4657worker2 # [ 72.016444] nix-daemon[848]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4658worker2 # [ 72.034567] nix-daemon[848]: reaped child process 955, status = succeeded4659worker2 # [ 72.206059] nix-daemon[848]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4660worker2 # [ 72.221526] nix-daemon[848]: reaped child process 960, status = succeeded4661lb # [ 72.179528] niks3-server[995]: 2026/09/21 13:49:55 INFO Received uploads request method=POST path=/api/pending_closures4662lb # [ 72.189353] niks3-server[995]: 2026/09/21 13:49:55 INFO Received uploads request method=POST path=/api/pending_closures4663worker2 # [ 72.252476] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.895Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4664worker2 # [ 72.254692] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.895Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4665worker2 # [ 72.256481] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.895Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4666lb # [ 72.217371] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4667lb # [ 72.236396] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4668lb # [ 72.247145] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4669lb # [ 72.257938] niks3-server[995]: 2026/09/21 13:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4670lb # [ 72.265372] niks3-server[995]: 2026/09/21 13:49:55 INFO Signed narinfos id=19 count=24671worker2 # [ 72.320479] nix-grpc-daemon[875]: time=2026-09-21T13:49:55.963Z level=INFO msg="Uploading 3 narinfos"4672lb # [ 72.270946] niks3-server[995]: 2026/09/21 13:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4673lb # [ 72.279059] niks3-server[995]: 2026/09/21 13:49:55 INFO Signed narinfos id=18 count=14674lb # [ 72.284333] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4675lb # [ 72.302888] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4676lb # [ 72.311467] niks3-server[995]: 2026/09/21 13:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4677lb # [ 72.315490] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4678lb # [ 72.319458] niks3-server[995]: 2026/09/21 13:49:55 INFO Completed upload id=184679lb # [ 72.321756] niks3-server[995]: 2026/09/21 13:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4680worker2 # [ 72.375300] nix-grpc-daemon[875]: time=2026-09-21T13:49:56.019Z level=INFO msg="Upload complete. (559ms)"4681lb # [ 72.324651] niks3-server[995]: 2026/09/21 13:49:55 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4682lb # [ 72.328038] niks3-server[995]: 2026/09/21 13:49:55 INFO Completed upload id=194683worker2 # [ 72.389207] nix-grpc-daemon[779]: ts=2026-09-21T13:49:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=0 paths=2 nar_bytes_in=9844684worker2 # [ 72.406599] nix-grpc-daemon[779]: ts=2026-09-21T13:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544685worker2 # [ 72.417154] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4686worker2 # [ 72.426962] nix-daemon[848]: reaped child process 950, status = succeeded4687worker2 # [ 72.437465] nix-grpc-daemon[779]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4688client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4689worker2 # [ 72.454846] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4690client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4691client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4692worker2 # [ 73.284937] nix-daemon[848]: reaped child process 965, status = succeeded4693worker2 # [ 73.447567] nix-daemon[848]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4694worker2 # [ 73.465541] nix-daemon[848]: reaped child process 984, status = succeeded4695lb # [ 73.425817] niks3-server[995]: 2026/09/21 13:49:57 INFO Received uploads request method=POST path=/api/pending_closures4696worker2 # [ 73.489761] nix-grpc-daemon[875]: time=2026-09-21T13:49:57.133Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4697lb # [ 73.459953] niks3-server[995]: 2026/09/21 13:49:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4698worker2 # [ 73.521020] nix-grpc-daemon[875]: time=2026-09-21T13:49:57.165Z level=INFO msg="Uploading 1 narinfos"4699lb # [ 73.471059] niks3-server[995]: 2026/09/21 13:49:57 INFO Signed narinfos id=20 count=14700lb # [ 73.479062] niks3-server[995]: 2026/09/21 13:49:57 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4701lb # [ 73.487524] niks3-server[995]: 2026/09/21 13:49:57 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4702worker2 # [ 73.548234] nix-grpc-daemon[875]: time=2026-09-21T13:49:57.192Z level=INFO msg="Upload complete. (275ms)"4703lb # [ 73.496322] niks3-server[995]: 2026/09/21 13:49:57 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4704lb # [ 73.501967] niks3-server[995]: 2026/09/21 13:49:57 INFO Completed upload id=204705lb # [ 73.504607] niks3-server[995]: 2026/09/21 13:49:57 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4706client # copying 1 paths...4707worker2 # [ 73.560819] nix-grpc-daemon[779]: ts=2026-09-21T13:49:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14708client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4709worker2 # [ 73.574404] nix-grpc-daemon[779]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544710worker2 # [ 73.578702] nix-daemon[848]: reaped child process 962, status = succeeded4711worker1 # [ 73.545906] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565924712worker1 # [ 73.560879] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4713worker1 # [ 73.581174] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_out=1204714client # [ 73.755470] nix-daemon[937]: reaped child process 1022, status = succeeded4715client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.86 seconds)4716client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24717client # [ 73.975114] nix-daemon[937]: accepted connection from pid 1041, user root (trusted)4718client # this derivation will be built:4719client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4720worker1 # [ 74.137709] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565924721client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4722worker1 # [ 74.145674] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924723worker1 # [ 74.153242] nix-grpc-daemon[875]: time=2026-09-21T13:49:57.795Z level=INFO msg="All 1 paths already cached"4724worker1 # [ 74.164816] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04725worker1 # [ 74.172391] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14726worker1 # [ 74.181650] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924727worker1 # [ 74.192212] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565924728client # copying 1 paths to 'lb:50051'...4729client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4730worker1 # [ 74.212106] nix-grpc-daemon[777]: ts=2026-09-21T13:49:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565924731worker1 # [ 74.226875] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4732worker1 # [ 74.246780] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)4733worker1 # [ 74.312599] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)4734worker1 # [ 74.324600] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 3/5)4735worker1 # [ 74.504796] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 4/5)4736worker1 # [ 74.673785] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4737worker1 # [ 74.680789] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4738worker1 # [ 74.687957] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4739worker1 # [ 74.918646] nix-daemon[860]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)4740worker1 # [ 74.938376] nix-daemon[860]: reaped child process 1222, status = succeeded4741worker1 # [ 75.110893] nix-daemon[860]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)4742worker1 # [ 75.127364] nix-daemon[860]: reaped child process 1227, status = succeeded4743lb # [ 75.139217] niks3-server[995]: 2026/09/21 13:49:58 INFO Received uploads request method=POST path=/api/pending_closures4744worker1 # [ 75.150421] nix-grpc-daemon[875]: time=2026-09-21T13:49:58.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4745worker1 # [ 75.151950] nix-grpc-daemon[875]: time=2026-09-21T13:49:58.794Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4746lb # [ 75.167797] niks3-server[995]: 2026/09/21 13:49:58 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4747worker1 # [ 75.183419] nix-grpc-daemon[875]: time=2026-09-21T13:49:58.826Z level=INFO msg="Uploading 1 narinfos"4748lb # [ 75.182513] niks3-server[995]: 2026/09/21 13:49:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4749lb # [ 75.191079] niks3-server[995]: 2026/09/21 13:49:58 INFO Signed narinfos id=21 count=14750lb # [ 75.196929] niks3-server[995]: 2026/09/21 13:49:58 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4751lb # [ 75.204888] niks3-server[995]: 2026/09/21 13:49:58 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4752lb # [ 75.209813] niks3-server[995]: 2026/09/21 13:49:58 INFO Completed upload id=214753worker1 # [ 75.210565] nix-grpc-daemon[875]: time=2026-09-21T13:49:58.854Z level=INFO msg="Upload complete. (502ms)"4754lb # [ 75.212189] niks3-server[995]: 2026/09/21 13:49:58 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4755worker1 # [ 75.216243] nix-grpc-daemon[777]: ts=2026-09-21T13:49:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=1 paths=1 nar_bytes_in=5684756worker1 # [ 75.224553] nix-grpc-daemon[777]: ts=2026-09-21T13:49:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565924757worker1 # [ 75.237819] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4758worker1 # [ 75.240981] nix-daemon[860]: reaped child process 1212, status = succeeded4759client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4760worker1 # [ 75.252326] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4761client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4762client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4763worker1 # [ 75.880960] nix-daemon[860]: reaped child process 1231, status = succeeded4764worker1 # [ 76.045078] nix-daemon[860]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)4765worker1 # [ 76.064513] nix-daemon[860]: reaped child process 1250, status = succeeded4766lb # [ 76.074940] niks3-server[995]: 2026/09/21 13:49:59 INFO Received uploads request method=POST path=/api/pending_closures4767worker1 # [ 76.087599] nix-grpc-daemon[875]: time=2026-09-21T13:49:59.729Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4768worker1 # [ 76.113247] nix-grpc-daemon[875]: time=2026-09-21T13:49:59.757Z level=INFO msg="Uploading 1 narinfos"4769lb # [ 76.104707] niks3-server[995]: 2026/09/21 13:49:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4770lb # [ 76.120162] niks3-server[995]: 2026/09/21 13:49:59 INFO Signed narinfos id=22 count=14771lb # [ 76.127066] niks3-server[995]: 2026/09/21 13:49:59 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4772lb # [ 76.133828] niks3-server[995]: 2026/09/21 13:49:59 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4773worker1 # [ 76.142884] nix-grpc-daemon[875]: time=2026-09-21T13:49:59.787Z level=INFO msg="Upload complete. (282ms)"4774lb # [ 76.143696] niks3-server[995]: 2026/09/21 13:49:59 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4775lb # [ 76.148964] niks3-server[995]: 2026/09/21 13:49:59 INFO Completed upload id=224776lb # [ 76.151784] niks3-server[995]: 2026/09/21 13:49:59 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4777worker1 # [ 76.150449] nix-grpc-daemon[777]: ts=2026-09-21T13:49:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14778worker1 # [ 76.164100] nix-daemon[860]: reaped child process 1229, status = succeeded4779client # copying 1 paths...4780worker2 # [ 76.225449] nix-grpc-daemon[779]: ts=2026-09-21T13:49:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544781worker2 # [ 76.239011] nix-grpc-daemon[779]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4782client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4783worker1 # [ 76.231020] nix-grpc-daemon[777]: ts=2026-09-21T13:49:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565924784worker1 # [ 76.237816] nix-grpc-daemon[777]: ts=2026-09-21T13:49:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_out=1204785client # [ 76.375667] nix-daemon[937]: reaped child process 1044, status = succeeded4786client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.62 seconds)4787client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24788client # [ 76.600861] nix-daemon[937]: accepted connection from pid 1063, user root (trusted)4789client # this derivation will be built:4790client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4791worker1 # [ 76.763287] nix-grpc-daemon[777]: ts=2026-09-21T13:50:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565924792client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4793worker2 # [ 76.820540] nix-grpc-daemon[779]: ts=2026-09-21T13:50:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544794worker2 # [ 76.832939] nix-grpc-daemon[875]: time=2026-09-21T13:50:00.476Z level=INFO msg="All 1 paths already cached"4795worker1 # [ 76.798966] nix-grpc-daemon[777]: ts=2026-09-21T13:50:00Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04796worker1 # [ 76.805514] nix-grpc-daemon[777]: ts=2026-09-21T13:50:00Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14797worker2 # [ 76.897748] nix-grpc-daemon[779]: ts=2026-09-21T13:50:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544798worker2 # [ 76.913184] nix-grpc-daemon[779]: ts=2026-09-21T13:50:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543544799worker2 # [ 76.927725] nix-grpc-daemon[875]: time=2026-09-21T13:50:00.572Z level=INFO msg="All 1 paths already cached"4800client # copying 1 paths to 'lb:50051'...4801client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4802worker2 # [ 76.938385] nix-grpc-daemon[779]: ts=2026-09-21T13:50:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543544803worker2 # [ 76.954791] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4804worker2 # [ 77.158504] nix-daemon[848]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4805worker2 # [ 77.177441] nix-daemon[848]: reaped child process 992, status = succeeded4806worker2 # [ 77.352606] nix-daemon[848]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4807worker2 # [ 77.371134] nix-daemon[848]: reaped child process 997, status = succeeded4808lb # [ 77.333912] niks3-server[995]: 2026/09/21 13:50:00 INFO Received uploads request method=POST path=/api/pending_closures4809worker2 # [ 77.396727] nix-grpc-daemon[875]: time=2026-09-21T13:50:01.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4810worker2 # [ 77.398639] nix-grpc-daemon[875]: time=2026-09-21T13:50:01.042Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4811lb # [ 77.362377] niks3-server[995]: 2026/09/21 13:50:00 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4812worker2 # [ 77.429797] nix-grpc-daemon[875]: time=2026-09-21T13:50:01.073Z level=INFO msg="Uploading 1 narinfos"4813lb # [ 77.376131] niks3-server[995]: 2026/09/21 13:50:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4814lb # [ 77.388311] niks3-server[995]: 2026/09/21 13:50:01 INFO Signed narinfos id=23 count=14815lb # [ 77.395524] niks3-server[995]: 2026/09/21 13:50:01 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4816worker2 # [ 77.460094] nix-grpc-daemon[875]: time=2026-09-21T13:50:01.104Z level=INFO msg="Upload complete. (488ms)"4817lb # [ 77.407834] niks3-server[995]: 2026/09/21 13:50:01 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4818lb # [ 77.413461] niks3-server[995]: 2026/09/21 13:50:01 INFO Completed upload id=234819worker2 # [ 77.468743] nix-grpc-daemon[779]: ts=2026-09-21T13:50:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=0 paths=1 nar_bytes_in=5684820lb # [ 77.417280] niks3-server[995]: 2026/09/21 13:50:01 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4821worker2 # [ 77.482541] nix-grpc-daemon[779]: ts=2026-09-21T13:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543544822worker2 # [ 77.488574] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4823worker2 # [ 77.493420] nix-daemon[848]: reaped child process 987, status = succeeded4824client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4825worker2 # [ 77.506591] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4826client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4827client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4828worker2 # [ 78.737541] nix-daemon[848]: reaped child process 1001, status = succeeded4829worker2 # [ 78.908233] nix-daemon[848]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4830worker2 # [ 78.926372] nix-daemon[848]: reaped child process 1020, status = succeeded4831lb # [ 78.885727] niks3-server[995]: 2026/09/21 13:50:02 INFO Received uploads request method=POST path=/api/pending_closures4832worker2 # [ 78.950053] nix-grpc-daemon[875]: time=2026-09-21T13:50:02.594Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4833worker2 # [ 78.980056] nix-grpc-daemon[875]: time=2026-09-21T13:50:02.624Z level=INFO msg="Uploading 1 narinfos"4834lb # [ 78.919055] niks3-server[995]: 2026/09/21 13:50:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4835lb # [ 78.933813] niks3-server[995]: 2026/09/21 13:50:02 INFO Signed narinfos id=24 count=14836lb # [ 78.940459] niks3-server[995]: 2026/09/21 13:50:02 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4837lb # [ 78.951248] niks3-server[995]: 2026/09/21 13:50:02 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4838worker2 # [ 79.010480] nix-grpc-daemon[875]: time=2026-09-21T13:50:02.654Z level=INFO msg="Upload complete. (293ms)"4839lb # [ 78.957440] niks3-server[995]: 2026/09/21 13:50:02 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4840lb # [ 78.963536] niks3-server[995]: 2026/09/21 13:50:02 INFO Completed upload id=244841lb # [ 78.966895] niks3-server[995]: 2026/09/21 13:50:02 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4842worker2 # [ 79.020918] nix-grpc-daemon[779]: ts=2026-09-21T13:50:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54354 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14843client # copying 1 paths...4844worker2 # [ 79.033193] nix-grpc-daemon[779]: ts=2026-09-21T13:50:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:543544845client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4846worker2 # [ 79.037710] nix-daemon[848]: reaped child process 999, status = succeeded4847worker1 # [ 79.008682] nix-grpc-daemon[777]: ts=2026-09-21T13:50:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565924848worker1 # [ 79.022378] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4849worker1 # [ 79.039393] nix-grpc-daemon[777]: ts=2026-09-21T13:50:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56592 duration_s=0 paths=1 nar_bytes_out=1204850client # [ 79.218633] nix-daemon[937]: reaped child process 1066, status = succeeded4851client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.84 seconds)4852(finished: subtest: an input only one worker has still reaches the builder, in 9.25 seconds)4853subtest: upload whose reference the worker lacks is completed from the cache4854client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4855client # this derivation will be built:4856client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4857client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4858client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4859client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4860client # this derivation will be built:4861client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4862client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4863client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4864client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4865client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4866worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4867lb # [ 80.282246] envoy[912]: conn peer=[2001:db8:1::1]:60168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3872 tx=1843 ms=23424868lb # [ 80.287370] envoy[912]: conn peer=[2001:db8:1::1]:60174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1024869lb # [ 80.292275] envoy[912]: conn peer=[2001:db8:1::1]:60176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2013 ms=21174870lb # [ 80.297040] envoy[912]: conn peer=[2001:db8:1::1]:60186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=274871lb # [ 80.302473] envoy[912]: conn peer=[2001:db8:1::1]:60190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=1917 ms=23334872lb # [ 80.307303] envoy[912]: conn peer=[2001:db8:1::1]:60202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=944873worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)4874worker2 # [ 80.630433] systemd[1]: Stopping Nix worker-protocol over gRPC...4875worker2 # [ 80.791803] nix-grpc-daemon[779]: ts=2026-09-21T13:50:04Z level=info event=signal_exit4876worker1 # [ 82.000684] nix-grpc-daemon[777]: ts=2026-09-21T13:50:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565804877worker2 # [ 82.054756] nix-grpc-daemon[779]: ts=2026-09-21T13:50:05Z level=info event=scheduler_restarting4878worker1 # [ 82.005309] nix-grpc-daemon[777]: ts=2026-09-21T13:50:05Z level=info event=worker_gone addr=192.168.1.4:500514879worker2 # [ 82.082261] nix-grpc-daemon[779]: ts=2026-09-21T13:50:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4880worker2 # [ 82.106596] nix-daemon[848]: reaped child process 850, status = succeeded4881worker2 # [ 82.880524] nix-grpc-daemon[779]: E0000 00:00:1789998606.524706 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4882worker2 # [ 82.889431] nix-grpc-daemon[779]: E0000 00:00:1789998606.533663 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4883worker2 # [ 83.666907] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4884worker2 # [ 83.674056] systemd[1]: Stopped Nix worker-protocol over gRPC.4885worker2 # [ 83.680475] systemd[1]: nix-grpc-daemon.service: Consumed 1.789s CPU time over 1min 3.201s wall clock time, 51.4M memory peak, 51.5K incoming IP traffic, 144.7K outgoing IP traffic.4886worker2 # [ 83.693164] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4887worker2 # [ 83.695255] systemd[1]: Closed Nix worker-protocol over gRPC.4888worker2 # [ 83.699626] systemd[1]: nix-grpc-daemon.socket: Consumed 568us CPU time over 1min 12.399s wall clock time, 256K memory peak, 61.9K incoming IP traffic, 55.5K outgoing IP traffic.4889lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4890lb: must succeed: curl -sf localhost:9901/clusters4891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4892lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4893client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24894worker1 # [ 83.986899] nix-grpc-daemon[777]: ts=2026-09-21T13:50:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565924895worker1 # [ 83.996223] nix-grpc-daemon[777]: ts=2026-09-21T13:50:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565924896client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4897client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4898worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4899worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4900worker2 # [ 84.199528] systemd[1]: Listening on Nix worker-protocol over gRPC.4901worker1 # [ 84.225590] systemd[1]: Stopping Nix worker-protocol over gRPC...4902worker2 # [ 84.654298] systemd[1]: Starting scheduler-token.service...4903worker1 # [ 84.661657] nix-grpc-daemon[777]: ts=2026-09-21T13:50:08Z level=info event=signal_exit4904lb # [ 84.682787] niks3-server[995]: 2026/09/21 13:50:08 INFO lead: released remote=192.168.1.3:437404905worker2 # [ 84.765982] systemd[1]: scheduler-token.service: Deactivated successfully.4906worker2 # [ 84.772767] systemd[1]: Finished scheduler-token.service.4907worker2 # [ 84.774375] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 115ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4908worker2 # [ 84.782267] systemd[1]: Starting Nix worker-protocol over gRPC...4909worker2 # [ 84.969481] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4910worker2 # [ 84.972109] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4911worker2 # [ 84.975236] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14912worker2 # [ 84.983738] nix-grpc-daemon[1044]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4913worker2 # [ 84.993046] nix-grpc-daemon[1044]: E0000 00:00:1789998608.625990 1044 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4914worker2 # [ 84.997693] nix-grpc-daemon[1044]: E0000 00:00:1789998608.628012 1044 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4915worker2 # [ 85.000850] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4916worker2 # [ 85.005635] systemd[1]: Started Nix worker-protocol over gRPC.4917lb # [ 84.952567] niks3-server[995]: 2026/09/21 13:50:08 INFO lead: acquired remote=192.168.1.4:451664918worker2 # [ 85.008074] nix-grpc-daemon[1044]: E0000 00:00:1789998608.652487 1083 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4919worker2 # [ 85.036145] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=scheduler_take_over4920worker2 # [ 85.037327] nix-grpc-daemon[1044]: E0000 00:00:1789998608.670397 1083 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4921worker2 # [ 85.039543] nix-grpc-daemon[1044]: E0000 00:00:1789998608.670565 1083 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4922worker2 # [ 85.047514] nix-grpc-daemon[1044]: E0000 00:00:1789998608.691160 1083 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4923worker2 # [ 85.060271] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=scheduler_connected addr=lb:500514924worker2 # [ 85.062073] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4925worker2 # [ 85.263537] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=scheduler_connected addr=lb:500514926worker2 # [ 85.268719] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4927worker2 # [ 85.469766] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_connected addr=lb:500514928worker2 # [ 85.474887] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4929worker2 # [ 85.676056] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_connected addr=lb:500514930worker2 # [ 85.681072] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4931worker1 # [ 85.688851] nix-grpc-daemon[777]: ts=2026-09-21T13:50:09Z level=info event=scheduler_restarting4932worker1 # [ 85.692727] nix-grpc-daemon[777]: ts=2026-09-21T13:50:09Z level=info event=scheduler_restarting addr=lb:500514933worker1 # [ 85.692926] nix-grpc-daemon[777]: ts=2026-09-21T13:50:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565804934worker1 # [ 85.693039] nix-grpc-daemon[777]: ts=2026-09-21T13:50:09Z level=info event=worker_gone addr=192.168.1.3:500514935worker1 # [ 85.717644] nix-daemon[860]: reaped child process 862, status = succeeded4936worker1 # [ 85.749360] nix-grpc-daemon[777]: E0000 00:00:1789998609.393309 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4937worker1 # [ 85.778534] nix-grpc-daemon[777]: E0000 00:00:1789998609.422538 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4938worker2 # [ 85.882067] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_connected addr=lb:500514939worker2 # [ 85.887107] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4940worker1 # [ 85.909351] nix-grpc-daemon[777]: ts=2026-09-21T13:50:09Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4941worker2 # [ 86.090910] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=scheduler_connected addr=lb:500514942worker2 # [ 86.104131] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=oidc_jwks provider=mock keys=14943worker2 # [ 86.109208] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:477764944worker2 # [ 86.114479] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04945worker1 # [ 87.533575] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4946worker1 # [ 87.539817] systemd[1]: Stopped Nix worker-protocol over gRPC.4947worker1 # [ 87.547317] systemd[1]: nix-grpc-daemon.service: Consumed 3.856s CPU time over 1min 7.256s wall clock time, 66.4M memory peak, 100.8K incoming IP traffic, 210.2K outgoing IP traffic.4948worker1 # [ 87.561261] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4949worker1 # [ 87.563402] systemd[1]: Closed Nix worker-protocol over gRPC.4950worker1 # [ 87.568563] systemd[1]: nix-grpc-daemon.socket: Consumed 514us CPU time over 1min 17.093s wall clock time, 256K memory peak, 98.4K incoming IP traffic, 80.4K outgoing IP traffic.4951lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4952lb: must succeed: curl -sf localhost:9901/clusters4953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4954lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4955client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24956worker2 # [ 87.976974] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477924957worker2 # [ 87.993151] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4958worker2 # [ 88.023461] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 1/5)4959worker2 # [ 88.105676] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 2/5)4960worker2 # [ 88.308160] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 3/5)4961worker2 # [ 88.591637] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 588 ms (attempt 4/5)4962worker2 # [ 89.181697] nix-grpc-daemon[1044]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4963worker2 # [ 89.187881] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4964client # copying 2 paths...4965client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4966worker2 # [ 89.211505] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477924967client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4968worker2 # [ 89.230994] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4969worker2 # [ 89.260684] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:12Z level=info event=niks3_push_started pid=11044970worker2 # [ 89.468554] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.112Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4971worker2 # [ 89.642485] nix-daemon[848]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)4972worker2 # [ 89.659437] nix-daemon[848]: reaped child process 1120, status = succeeded4973lb # [ 89.619829] niks3-server[995]: 2026/09/21 13:50:13 INFO Received uploads request method=POST path=/api/pending_closures4974lb # [ 89.630279] niks3-server[995]: 2026/09/21 13:50:13 INFO Received uploads request method=POST path=/api/pending_closures4975worker2 # [ 89.695401] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.338Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4976worker2 # [ 89.698254] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.338Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4977worker2 # [ 89.700118] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.338Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4978lb # [ 89.683074] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4979lb # [ 89.697563] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4980lb # [ 89.711203] niks3-server[995]: 2026/09/21 13:50:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4981worker2 # [ 89.772120] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.415Z level=INFO msg="Uploading 3 narinfos"4982lb # [ 89.719609] niks3-server[995]: 2026/09/21 13:50:13 INFO Signed narinfos id=26 count=24983lb # [ 89.722382] niks3-server[995]: 2026/09/21 13:50:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4984lb # [ 89.728509] niks3-server[995]: 2026/09/21 13:50:13 INFO Signed narinfos id=25 count=14985lb # [ 89.730344] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4986lb # [ 89.738344] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4987lb # [ 89.745346] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4988lb # [ 89.755830] niks3-server[995]: 2026/09/21 13:50:13 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4989lb # [ 89.765138] niks3-server[995]: 2026/09/21 13:50:13 INFO Completed upload id=254990lb # [ 89.769794] niks3-server[995]: 2026/09/21 13:50:13 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4991worker2 # [ 89.826746] nix-grpc-daemon[1104]: time=2026-09-21T13:50:13.471Z level=INFO msg="Upload complete. (354ms)"4992lb # [ 89.775578] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4993lb # [ 89.783230] niks3-server[995]: 2026/09/21 13:50:13 INFO Completed upload id=264994worker2 # [ 89.836394] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=0 paths=2 nar_bytes_in=3124995lb # [ 89.790568] niks3-server[995]: 2026/09/21 13:50:13 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4996worker2 # [ 89.853779] nix-daemon[848]: reaped child process 1102, status = succeeded4997client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.16 seconds)4998worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4999worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5000worker1 # [ 89.938633] systemd[1]: Listening on Nix worker-protocol over gRPC.5001worker1 # [ 89.947819] systemd[1]: Starting Nix worker-protocol over gRPC...5002worker1 # [ 90.140443] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5003worker1 # [ 90.142573] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5004worker1 # [ 90.146750] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15005worker1 # [ 90.155709] nix-grpc-daemon[1276]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5006worker1 # [ 90.163745] nix-grpc-daemon[1276]: E0000 00:00:1789998613.797705 1276 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5007worker1 # [ 90.168068] nix-grpc-daemon[1276]: E0000 00:00:1789998613.799768 1276 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5008worker1 # [ 90.171622] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5009worker1 # [ 90.174082] systemd[1]: Started Nix worker-protocol over gRPC.5010lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5011lb: must succeed: curl -sf localhost:9901/clusters5012worker1 # [ 90.180107] nix-grpc-daemon[1276]: E0000 00:00:1789998613.820501 1315 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5013worker1 # [ 90.204091] nix-grpc-daemon[1276]: E0000 00:00:1789998613.838104 1315 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5014worker1 # [ 90.206397] nix-grpc-daemon[1276]: E0000 00:00:1789998613.838284 1315 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5015worker1 # [ 90.214215] nix-grpc-daemon[1276]: E0000 00:00:1789998613.857441 1315 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5016worker1 # [ 90.229381] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:13Z level=info event=scheduler_connected addr=lb:500515017worker2 # [ 90.285670] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:477765018worker2 # [ 90.290756] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05019lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5020lb # [ 90.285821] envoy[912]: conn peer=[2001:db8:1::4]:58516 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=61057 tx=3005 ms=612275021lb # [ 90.291749] envoy[912]: conn peer=[2001:db8:1::1]:48250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=325022lb # [ 90.295974] envoy[912]: conn peer=[2001:db8:1::3]:36018 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12582 tx=3463 ms=652155023lb # [ 90.299972] envoy[912]: conn peer=[2001:db8:1::1]:48260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=305 ms=18795024lb: must succeed: curl -sf localhost:9901/clusters5025lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5026lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5027(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.15 seconds)5028subtest: a niks3 restart does not cost the leader its role5029lb: must succeed: curl -sf localhost:9901/clusters5030lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5031lb: must succeed: curl -sf localhost:9901/clusters5032lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5033lb # [ 91.484424] niks3-server[995]: 2026/09/21 13:50:15 INFO Shutdown signal received, draining in-flight requests timeout=10s5034lb # [ 91.487212] systemd[1]: Stopping niks3 server...5035worker1 # [ 91.492778] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:15Z level=info event=scheduler_lock_lost status=2005036worker2 # [ 91.545919] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:15Z level=info event=scheduler_lock_lost status=2005037lb # [ 91.491259] niks3-server[995]: 2026/09/21 13:50:15 INFO lead: released remote=192.168.1.4:451665038lb # [ 95.161452] systemd[1]: niks3.service: Deactivated successfully.5039lb # [ 95.166318] systemd[1]: Stopped niks3 server.5040lb # [ 95.170479] systemd[1]: niks3.service: Consumed 533ms CPU time over 1min 9.082s wall clock time, 22M memory peak, 74K incoming IP traffic, 104.4K outgoing IP traffic.5041lb # [ 95.193476] systemd[1]: Starting niks3 server...5042lb # [ 95.304129] niks3-server[1130]: 2026/09/21 13:50:18 goose: no migrations to run. current version: 202609200000005043lb # [ 95.310224] niks3-server[1130]: 2026/09/21 13:50:18 OK 1_commit_pending_closure.sql (7.31ms)5044lb # [ 95.314001] niks3-server[1130]: 2026/09/21 13:50:18 OK 2_object_stats_trigger.sql (3.82ms)5045lb # [ 95.315230] niks3-server[1130]: 2026/09/21 13:50:18 goose: up to current file version: 25046lb # [ 95.316798] niks3-server[1130]: 2026/09/21 13:50:18 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key5047lb # [ 95.328730] niks3-server[1130]: 2026/09/21 13:50:18 INFO Read proxy enabled — serving cache objects from S35048lb # [ 95.333478] niks3-server[1130]: 2026/09/21 13:50:18 INFO Using socket-activated listener address=0.0.0.0:57515049lb # [ 95.334825] niks3-server[1130]: 2026/09/21 13:50:18 INFO systemd watchdog enabled interval=15s5050lb # [ 95.335922] niks3-server[1130]: 2026/09/21 13:50:18 INFO Starting HTTP server address=0.0.0.0:57515051lb # [ 95.338393] niks3-server[1130]: 2026/09/21 13:50:18 INFO lead: acquired remote=192.168.1.4:590025052lb # [ 95.339686] systemd[1]: Started niks3 server.5053lb: waiting for unit niks3.service5054lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5055lb: must succeed: curl -sf localhost:9901/clusters5056lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5057worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5058worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5059lb: must succeed: curl -sf localhost:9901/clusters5060lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5061lb: must succeed: curl -sf localhost:9901/clusters5062lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5063worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5064worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5065client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&25066worker1 # [ 95.985752] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443005067worker1 # [ 96.000237] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5068client # don't know how to build these paths:5069client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5070worker2 # [ 96.077312] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477925071worker2 # [ 96.123143] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05072worker2 # [ 96.130071] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15073worker2 # [ 96.178781] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925074worker2 # [ 96.195341] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477925075client # copying 1 paths to 'lb:50051'...5076client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5077worker2 # [ 96.213748] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477925078worker2 # [ 96.236741] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5079worker2 # [ 96.443234] nix-daemon[848]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5080worker2 # [ 96.465347] nix-daemon[848]: reaped child process 1142, status = succeeded5081worker2 # [ 96.639840] nix-daemon[848]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5082worker2 # [ 96.660363] nix-daemon[848]: reaped child process 1147, status = succeeded5083lb # [ 96.622084] niks3-server[1130]: 2026/09/21 13:50:20 INFO Received uploads request method=POST path=/api/pending_closures5084worker2 # [ 96.690739] nix-grpc-daemon[1104]: time=2026-09-21T13:50:20.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5085worker2 # [ 96.692323] nix-grpc-daemon[1104]: time=2026-09-21T13:50:20.334Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5086lb # [ 96.675628] niks3-server[1130]: 2026/09/21 13:50:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5087lb # [ 96.689744] niks3-server[1130]: 2026/09/21 13:50:20 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5088lb # [ 96.715652] niks3-server[1130]: 2026/09/21 13:50:20 INFO Signed narinfos id=27 count=15089worker2 # [ 96.773004] nix-grpc-daemon[1104]: time=2026-09-21T13:50:20.416Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 96.724373] niks3-server[1130]: 2026/09/21 13:50:20 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5091lb # [ 96.729561] niks3-server[1130]: 2026/09/21 13:50:20 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5092lb # [ 96.739620] niks3-server[1130]: 2026/09/21 13:50:20 INFO Completed upload id=275093worker2 # [ 96.793593] nix-grpc-daemon[1104]: time=2026-09-21T13:50:20.437Z level=INFO msg="Upload complete. (537ms)"5094lb # [ 96.746012] niks3-server[1130]: 2026/09/21 13:50:20 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5095worker2 # [ 96.805029] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=0 paths=1 nar_bytes_in=5445096worker2 # [ 96.821317] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925097worker2 # [ 96.834675] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5098worker2 # [ 96.839602] nix-daemon[848]: reaped child process 1137, status = succeeded5099client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5100client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5101worker2 # [ 96.853007] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5102client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5103client # farm-a> LOG-farm-a-after-niks3-restart5104worker2 # [ 98.318397] nix-daemon[848]: reaped child process 1151, status = succeeded5105worker2 # [ 98.620281] nix-daemon[848]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5106worker2 # [ 98.646418] nix-daemon[848]: reaped child process 1170, status = succeeded5107lb # [ 98.607795] niks3-server[1130]: 2026/09/21 13:50:22 INFO Received uploads request method=POST path=/api/pending_closures5108worker2 # [ 98.689785] nix-grpc-daemon[1104]: time=2026-09-21T13:50:22.333Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5109worker2 # [ 98.695114] nix-grpc-daemon[1104]: time=2026-09-21T13:50:22.336Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5110lb # [ 98.705783] niks3-server[1130]: 2026/09/21 13:50:22 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5111lb # [ 98.718644] niks3-server[1130]: 2026/09/21 13:50:22 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5112lb # [ 98.725956] niks3-server[1130]: 2026/09/21 13:50:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5113lb # [ 98.736791] niks3-server[1130]: 2026/09/21 13:50:22 INFO Signed narinfos id=28 count=15114worker2 # [ 98.789489] nix-grpc-daemon[1104]: time=2026-09-21T13:50:22.429Z level=INFO msg="Uploading 1 narinfos"5115lb # [ 98.741159] niks3-server[1130]: 2026/09/21 13:50:22 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5116lb # [ 98.755851] niks3-server[1130]: 2026/09/21 13:50:22 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5117lb # [ 98.761330] niks3-server[1130]: 2026/09/21 13:50:22 INFO Completed upload id=285118lb # [ 98.764448] niks3-server[1130]: 2026/09/21 13:50:22 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5119worker2 # [ 98.816524] nix-grpc-daemon[1104]: time=2026-09-21T13:50:22.460Z level=INFO msg="Upload complete. (548ms)"5120worker2 # [ 98.861661] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=2 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15121worker2 # [ 98.880394] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05122worker2 # [ 98.884800] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15123worker2 # [ 98.889714] nix-daemon[848]: reaped child process 1149, status = succeeded5124worker2 # [ 98.903934] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925125worker2 # [ 98.918428] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477925126worker2 # [ 98.933869] nix-grpc-daemon[1104]: time=2026-09-21T13:50:22.578Z level=INFO msg="All 1 paths already cached"5127client # copying 1 paths to 'lb:50051'...5128client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5129worker2 # [ 98.944779] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477925130worker2 # [ 98.959096] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5131worker2 # [ 99.160777] nix-daemon[848]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5132worker2 # [ 99.179702] nix-daemon[848]: reaped child process 1179, status = succeeded5133worker2 # [ 99.355721] nix-daemon[848]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5134worker2 # [ 99.371051] nix-daemon[848]: reaped child process 1184, status = succeeded5135lb # [ 99.332563] niks3-server[1130]: 2026/09/21 13:50:22 INFO Received uploads request method=POST path=/api/pending_closures5136worker2 # [ 99.395475] nix-grpc-daemon[1104]: time=2026-09-21T13:50:23.039Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5137worker2 # [ 99.397111] nix-grpc-daemon[1104]: time=2026-09-21T13:50:23.039Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5138lb # [ 99.361228] niks3-server[1130]: 2026/09/21 13:50:22 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5139lb # [ 99.372452] niks3-server[1130]: 2026/09/21 13:50:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5140worker2 # [ 99.432050] nix-grpc-daemon[1104]: time=2026-09-21T13:50:23.076Z level=INFO msg="Uploading 1 narinfos"5141lb # [ 99.382698] niks3-server[1130]: 2026/09/21 13:50:23 INFO Signed narinfos id=29 count=15142lb # [ 99.389143] niks3-server[1130]: 2026/09/21 13:50:23 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5143lb # [ 99.396593] niks3-server[1130]: 2026/09/21 13:50:23 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5144lb # [ 99.403044] niks3-server[1130]: 2026/09/21 13:50:23 INFO Completed upload id=295145worker2 # [ 99.456741] nix-grpc-daemon[1104]: time=2026-09-21T13:50:23.101Z level=INFO msg="Upload complete. (481ms)"5146lb # [ 99.405565] niks3-server[1130]: 2026/09/21 13:50:23 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5147worker2 # [ 99.464140] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=0 paths=1 nar_bytes_in=6565148worker2 # [ 99.475074] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925149worker2 # [ 99.483542] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5150worker2 # [ 99.485772] nix-daemon[848]: reaped child process 1174, status = succeeded5151client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5152worker2 # [ 99.496641] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5153client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5154client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5155client # farm-b> LOG-farm-b-after-niks3-restart5156worker2 # [ 100.333229] nix-daemon[848]: reaped child process 1188, status = succeeded5157worker2 # [ 100.505426] nix-daemon[848]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5158worker2 # [ 100.523991] nix-daemon[848]: reaped child process 1207, status = succeeded5159lb # [ 100.483660] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received uploads request method=POST path=/api/pending_closures5160worker2 # [ 100.547271] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5161worker2 # [ 100.548871] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.190Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5162lb # [ 100.512947] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5163lb # [ 100.525129] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5164lb # [ 100.539643] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5165worker2 # [ 100.599345] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.243Z level=INFO msg="Uploading 1 narinfos"5166lb # [ 100.552910] niks3-server[1130]: 2026/09/21 13:50:24 INFO Signed narinfos id=30 count=15167lb # [ 100.559946] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5168lb # [ 100.569738] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5169lb # [ 100.575037] niks3-server[1130]: 2026/09/21 13:50:24 INFO Completed upload id=305170worker2 # [ 100.628325] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.272Z level=INFO msg="Upload complete. (320ms)"5171lb # [ 100.578303] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5172worker2 # [ 100.645930] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15173worker2 # [ 100.660436] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05174worker2 # [ 100.663864] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15175worker2 # [ 100.667892] nix-daemon[848]: reaped child process 1186, status = succeeded5176worker2 # [ 100.690121] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925177worker2 # [ 100.704786] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477925178worker2 # [ 100.719847] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.364Z level=INFO msg="All 2 paths already cached"5179client # copying 1 paths to 'lb:50051'...5180client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5181worker2 # [ 100.729977] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477925182worker2 # [ 100.743561] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5183worker2 # [ 100.941769] nix-daemon[848]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5184worker2 # [ 100.961018] nix-daemon[848]: reaped child process 1216, status = succeeded5185worker2 # [ 101.132904] nix-daemon[848]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5186worker2 # [ 101.148360] nix-daemon[848]: reaped child process 1221, status = succeeded5187lb # [ 101.109276] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received uploads request method=POST path=/api/pending_closures5188worker2 # [ 101.172090] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.816Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5189worker2 # [ 101.173552] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.816Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5190lb # [ 101.141284] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5191lb # [ 101.152119] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5192worker2 # [ 101.213613] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.857Z level=INFO msg="Uploading 1 narinfos"5193lb # [ 101.163730] niks3-server[1130]: 2026/09/21 13:50:24 INFO Signed narinfos id=31 count=15194lb # [ 101.170964] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5195lb # [ 101.180245] niks3-server[1130]: 2026/09/21 13:50:24 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5196lb # [ 101.188627] niks3-server[1130]: 2026/09/21 13:50:24 INFO Completed upload id=315197worker2 # [ 101.242352] nix-grpc-daemon[1104]: time=2026-09-21T13:50:24.886Z level=INFO msg="Upload complete. (482ms)"5198lb # [ 101.192111] niks3-server[1130]: 2026/09/21 13:50:24 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5199worker2 # [ 101.252316] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=0 paths=1 nar_bytes_in=7845200worker2 # [ 101.263490] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477925201worker2 # [ 101.273171] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5202worker2 # [ 101.276086] nix-daemon[848]: reaped child process 1211, status = succeeded5203client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5204client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5205worker2 # [ 101.289834] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5206client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5207client # farm-top> LOG-farm-top-after-niks3-restart5208worker2 # [ 101.933116] nix-daemon[848]: reaped child process 1225, status = succeeded5209worker2 # [ 102.115429] nix-daemon[848]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5210worker2 # [ 102.134552] nix-daemon[848]: reaped child process 1244, status = succeeded5211lb # [ 102.093725] niks3-server[1130]: 2026/09/21 13:50:25 INFO Received uploads request method=POST path=/api/pending_closures5212worker2 # [ 102.157089] nix-grpc-daemon[1104]: time=2026-09-21T13:50:25.801Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5213worker2 # [ 102.160297] nix-grpc-daemon[1104]: time=2026-09-21T13:50:25.803Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5214lb # [ 102.127172] niks3-server[1130]: 2026/09/21 13:50:25 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5215lb # [ 102.140958] niks3-server[1130]: 2026/09/21 13:50:25 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5216lb # [ 102.153727] niks3-server[1130]: 2026/09/21 13:50:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5217worker2 # [ 102.212030] nix-grpc-daemon[1104]: time=2026-09-21T13:50:25.855Z level=INFO msg="Uploading 1 narinfos"5218lb # [ 102.162968] niks3-server[1130]: 2026/09/21 13:50:25 INFO Signed narinfos id=32 count=15219lb # [ 102.169006] niks3-server[1130]: 2026/09/21 13:50:25 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5220worker2 # [ 102.229680] nix-grpc-daemon[1104]: time=2026-09-21T13:50:25.874Z level=INFO msg="Upload complete. (330ms)"5221lb # [ 102.178364] niks3-server[1130]: 2026/09/21 13:50:25 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5222lb # [ 102.184399] niks3-server[1130]: 2026/09/21 13:50:25 INFO Completed upload id=325223lb # [ 102.187265] niks3-server[1130]: 2026/09/21 13:50:25 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5224worker2 # [ 102.241961] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47792 duration_s=0 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15225worker2 # [ 102.257580] nix-daemon[848]: reaped child process 1223, status = succeeded5226client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 6.55 seconds)5227client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath5228client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.25 seconds)5229(finished: subtest: a niks3 restart does not cost the leader its role, in 11.16 seconds)5230subtest: scheduler down: the other node takes the lock and keeps it5231lb: must succeed: curl -sf localhost:9901/clusters5232lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5233lb: must succeed: curl -sf localhost:9901/clusters5234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5235lb: must succeed: curl -sf localhost:9901/clusters5236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5237lb: must succeed: curl -sf localhost:9901/clusters5238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5239worker2 # [ 102.869282] systemd[1]: Stopping Nix worker-protocol over gRPC...5240worker2 # [ 103.001170] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:26Z level=info event=signal_exit5241lb # [ 103.713970] niks3-server[1130]: 2026/09/21 13:50:27 INFO lead: released remote=192.168.1.4:590025242worker1 # [ 104.715544] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_restarting addr=lb:500515243worker2 # [ 104.771424] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=scheduler_restarting5244worker2 # [ 104.771946] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=scheduler_restarting addr=lb:500515245worker2 # [ 104.780770] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477765246worker2 # [ 104.786102] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=worker_gone addr=192.168.1.4:500515247worker2 # [ 104.791055] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477765248worker2 # [ 104.796114] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=worker_gone addr=192.168.1.3:500515249worker2 # [ 104.822415] nix-daemon[848]: reaped child process 1096, status = succeeded5250worker1 # [ 104.915260] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_connected addr=lb:500515251worker1 # [ 104.920456] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5252worker2 # [ 104.989655] nix-grpc-daemon[1044]: ts=2026-09-21T13:50:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5253worker2 # [ 105.049510] nix-grpc-daemon[1044]: E0000 00:00:1789998628.693490 1089 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5254worker2 # [ 105.056187] nix-grpc-daemon[1044]: E0000 00:00:1789998628.693794 1089 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5255worker1 # [ 105.121213] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_connected addr=lb:500515256worker1 # [ 105.126425] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5257worker1 # [ 105.327324] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_connected addr=lb:500515258worker1 # [ 105.332614] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259lb # [ 105.334041] niks3-server[1130]: 2026/09/21 13:50:28 INFO lead: acquired remote=192.168.1.3:476205260worker1 # [ 105.338260] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:28Z level=info event=scheduler_take_over5261worker1 # [ 105.533451] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=info event=scheduler_connected addr=lb:500515262worker1 # [ 105.538718] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5263worker2 # [ 105.615258] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5264worker2 # [ 105.623461] systemd[1]: Stopped Nix worker-protocol over gRPC.5265worker2 # [ 105.627237] systemd[1]: nix-grpc-daemon.service: Consumed 1.687s CPU time over 20.828s wall clock time, 36.4M memory peak, 46.7K incoming IP traffic, 81.4K outgoing IP traffic.5266worker2 # [ 105.642936] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5267worker2 # [ 105.647326] systemd[1]: Closed Nix worker-protocol over gRPC.5268worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5269worker2 # [ 105.649648] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 21.452s wall clock time, 256K memory peak, 41.1K incoming IP traffic, 31K outgoing IP traffic.5270worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5271worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5272worker1 # [ 105.739693] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 105.747751] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:430825274worker1 # [ 105.750610] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05275worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5276client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25277worker1 # [ 106.017771] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443005278client # don't know how to build these paths:5279client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5280worker1 # [ 106.030302] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443005281worker1 # [ 106.052462] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 1/5)5282worker1 # [ 106.116962] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 2/5)5283worker1 # [ 106.286456] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5)5284worker1 # [ 106.688843] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 4/5)5285worker1 # [ 106.845118] nix-grpc-daemon[1276]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5286worker1 # [ 106.851137] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5287worker1 # [ 106.882325] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05288worker1 # [ 106.889520] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15289worker1 # [ 106.930033] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005290worker1 # [ 106.946133] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443005291client # copying 1 paths to 'lb:50051'...5292worker1 # [ 106.960106] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443005293client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5294worker1 # [ 106.974535] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5295worker1 # [ 107.007069] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:30Z level=info event=niks3_push_started pid=13495296worker1 # [ 107.219687] nix-grpc-daemon[1349]: time=2026-09-21T13:50:30.863Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5297worker1 # [ 107.391264] nix-daemon[860]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)5298worker1 # [ 107.410672] nix-daemon[860]: reaped child process 1365, status = succeeded5299worker1 # [ 107.580537] nix-daemon[860]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5300worker1 # [ 107.595233] nix-daemon[860]: reaped child process 1370, status = succeeded5301lb # [ 107.611361] niks3-server[1130]: 2026/09/21 13:50:31 INFO Received uploads request method=POST path=/api/pending_closures5302worker1 # [ 107.624066] nix-grpc-daemon[1349]: time=2026-09-21T13:50:31.267Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5303worker1 # [ 107.625528] nix-grpc-daemon[1349]: time=2026-09-21T13:50:31.267Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5304lb # [ 107.647812] niks3-server[1130]: 2026/09/21 13:50:31 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5305worker1 # [ 107.664974] nix-grpc-daemon[1349]: time=2026-09-21T13:50:31.308Z level=INFO msg="Uploading 1 narinfos"5306lb # [ 107.663629] niks3-server[1130]: 2026/09/21 13:50:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5307lb # [ 107.675287] niks3-server[1130]: 2026/09/21 13:50:31 INFO Signed narinfos id=33 count=15308lb # [ 107.682252] niks3-server[1130]: 2026/09/21 13:50:31 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5309lb # [ 107.697403] niks3-server[1130]: 2026/09/21 13:50:31 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5310lb # [ 107.704192] niks3-server[1130]: 2026/09/21 13:50:31 INFO Completed upload id=335311worker1 # [ 107.705150] nix-grpc-daemon[1349]: time=2026-09-21T13:50:31.349Z level=INFO msg="Upload complete. (481ms)"5312lb # [ 107.708868] niks3-server[1130]: 2026/09/21 13:50:31 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5313worker1 # [ 107.716080] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=0 paths=1 nar_bytes_in=5205314worker1 # [ 107.730590] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005315worker1 # [ 107.737488] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5316worker1 # [ 107.742470] nix-daemon[860]: reaped child process 1347, status = succeeded5317client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5318worker1 # [ 107.754036] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5319client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5320client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5321client # farm-a> LOG-farm-a-on-standby5322worker1 # [ 108.798485] nix-daemon[860]: reaped child process 1374, status = succeeded5323worker1 # [ 108.982504] nix-daemon[860]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5324worker1 # [ 108.998464] nix-daemon[860]: reaped child process 1393, status = succeeded5325lb # [ 109.010815] niks3-server[1130]: 2026/09/21 13:50:32 INFO Received uploads request method=POST path=/api/pending_closures5326worker1 # [ 109.024387] nix-grpc-daemon[1349]: time=2026-09-21T13:50:32.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5327worker1 # [ 109.025873] nix-grpc-daemon[1349]: time=2026-09-21T13:50:32.666Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5328lb # [ 109.049729] niks3-server[1130]: 2026/09/21 13:50:32 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5329lb # [ 109.066224] niks3-server[1130]: 2026/09/21 13:50:32 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5330lb # [ 109.075307] niks3-server[1130]: 2026/09/21 13:50:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5331worker1 # [ 109.083700] nix-grpc-daemon[1349]: time=2026-09-21T13:50:32.726Z level=INFO msg="Uploading 1 narinfos"5332lb # [ 109.081691] niks3-server[1130]: 2026/09/21 13:50:32 INFO Signed narinfos id=34 count=15333lb # [ 109.086588] niks3-server[1130]: 2026/09/21 13:50:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5334lb # [ 109.097366] niks3-server[1130]: 2026/09/21 13:50:32 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5335worker1 # [ 109.103732] nix-grpc-daemon[1349]: time=2026-09-21T13:50:32.747Z level=INFO msg="Upload complete. (336ms)"5336lb # [ 109.104548] niks3-server[1130]: 2026/09/21 13:50:32 INFO Completed upload id=345337lb # [ 109.107344] niks3-server[1130]: 2026/09/21 13:50:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5338worker1 # [ 109.124336] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15339worker1 # [ 109.137679] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05340worker1 # [ 109.141314] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15341worker1 # [ 109.144991] nix-daemon[860]: reaped child process 1372, status = succeeded5342worker1 # [ 109.168720] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005343worker1 # [ 109.184275] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443005344worker1 # [ 109.197912] nix-grpc-daemon[1349]: time=2026-09-21T13:50:32.842Z level=INFO msg="All 1 paths already cached"5345client # copying 1 paths to 'lb:50051'...5346client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5347worker1 # [ 109.207454] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443005348worker1 # [ 109.220457] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5349worker1 # [ 109.420715] nix-daemon[860]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5350worker1 # [ 109.436905] nix-daemon[860]: reaped child process 1402, status = succeeded5351worker1 # [ 109.609104] nix-daemon[860]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5352worker1 # [ 109.624161] nix-daemon[860]: reaped child process 1407, status = succeeded5353lb # [ 109.638165] niks3-server[1130]: 2026/09/21 13:50:33 INFO Received uploads request method=POST path=/api/pending_closures5354worker1 # [ 109.647608] nix-grpc-daemon[1349]: time=2026-09-21T13:50:33.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5355worker1 # [ 109.649245] nix-grpc-daemon[1349]: time=2026-09-21T13:50:33.291Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5356lb # [ 109.662196] niks3-server[1130]: 2026/09/21 13:50:33 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5357lb # [ 109.676546] niks3-server[1130]: 2026/09/21 13:50:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5358worker1 # [ 109.682512] nix-grpc-daemon[1349]: time=2026-09-21T13:50:33.326Z level=INFO msg="Uploading 1 narinfos"5359lb # [ 109.688081] niks3-server[1130]: 2026/09/21 13:50:33 INFO Signed narinfos id=35 count=15360lb # [ 109.691102] niks3-server[1130]: 2026/09/21 13:50:33 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5361lb # [ 109.700554] niks3-server[1130]: 2026/09/21 13:50:33 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5362lb # [ 109.707289] niks3-server[1130]: 2026/09/21 13:50:33 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5363lb # [ 109.711117] niks3-server[1130]: 2026/09/21 13:50:33 INFO Completed upload id=355364worker1 # [ 109.711950] nix-grpc-daemon[1349]: time=2026-09-21T13:50:33.356Z level=INFO msg="Upload complete. (475ms)"5365worker1 # [ 109.719074] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=0 paths=1 nar_bytes_in=6405366worker1 # [ 109.731183] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005367worker1 # [ 109.737225] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5368worker1 # [ 109.741928] nix-daemon[860]: reaped child process 1397, status = succeeded5369client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5370client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5371worker1 # [ 109.754869] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5372lb # [ 110.284874] envoy[912]: conn peer=[2001:db8:1::1]:60750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9779 tx=4105 ms=62345373client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5374client # farm-b> LOG-farm-b-on-standby5375worker1 # [ 110.801543] nix-daemon[860]: reaped child process 1411, status = succeeded5376lb # [ 110.289984] envoy[912]: conn peer=[2001:db8:1::4]:48432 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=199185377worker1 # [ 110.980952] nix-daemon[860]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5378worker1 # [ 110.997356] nix-daemon[860]: reaped child process 1430, status = succeeded5379lb # [ 111.010323] niks3-server[1130]: 2026/09/21 13:50:34 INFO Received uploads request method=POST path=/api/pending_closures5380worker1 # [ 111.020357] nix-grpc-daemon[1349]: time=2026-09-21T13:50:34.663Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5381worker1 # [ 111.021808] nix-grpc-daemon[1349]: time=2026-09-21T13:50:34.663Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5382lb # [ 111.038024] niks3-server[1130]: 2026/09/21 13:50:34 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5383lb # [ 111.052724] niks3-server[1130]: 2026/09/21 13:50:34 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5384lb # [ 111.066318] niks3-server[1130]: 2026/09/21 13:50:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5385worker1 # [ 111.071993] nix-grpc-daemon[1349]: time=2026-09-21T13:50:34.715Z level=INFO msg="Uploading 1 narinfos"5386lb # [ 111.076632] niks3-server[1130]: 2026/09/21 13:50:34 INFO Signed narinfos id=36 count=15387lb # [ 111.079334] niks3-server[1130]: 2026/09/21 13:50:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5388lb # [ 111.088112] niks3-server[1130]: 2026/09/21 13:50:34 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5389lb # [ 111.096432] niks3-server[1130]: 2026/09/21 13:50:34 INFO Completed upload id=365390worker1 # [ 111.096842] nix-grpc-daemon[1349]: time=2026-09-21T13:50:34.741Z level=INFO msg="Upload complete. (332ms)"5391lb # [ 111.099393] niks3-server[1130]: 2026/09/21 13:50:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5392worker1 # [ 111.112742] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15393worker1 # [ 111.125320] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05394worker1 # [ 111.129103] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15395worker1 # [ 111.132721] nix-daemon[860]: reaped child process 1409, status = succeeded5396worker1 # [ 111.156903] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005397worker1 # [ 111.171006] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443005398worker1 # [ 111.186575] nix-grpc-daemon[1349]: time=2026-09-21T13:50:34.830Z level=INFO msg="All 2 paths already cached"5399client # copying 1 paths to 'lb:50051'...5400client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5401worker1 # [ 111.197375] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443005402worker1 # [ 111.210890] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5403worker1 # [ 111.413215] nix-daemon[860]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5404worker1 # [ 111.431700] nix-daemon[860]: reaped child process 1439, status = succeeded5405worker1 # [ 111.601328] nix-daemon[860]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5406worker1 # [ 111.619372] nix-daemon[860]: reaped child process 1444, status = succeeded5407lb # [ 111.630123] niks3-server[1130]: 2026/09/21 13:50:35 INFO Received uploads request method=POST path=/api/pending_closures5408worker1 # [ 111.640053] nix-grpc-daemon[1349]: time=2026-09-21T13:50:35.284Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5409worker1 # [ 111.641967] nix-grpc-daemon[1349]: time=2026-09-21T13:50:35.285Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5410lb # [ 111.662110] niks3-server[1130]: 2026/09/21 13:50:35 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5411lb # [ 111.673577] niks3-server[1130]: 2026/09/21 13:50:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5412worker1 # [ 111.680842] nix-grpc-daemon[1349]: time=2026-09-21T13:50:35.324Z level=INFO msg="Uploading 1 narinfos"5413lb # [ 111.684825] niks3-server[1130]: 2026/09/21 13:50:35 INFO Signed narinfos id=37 count=15414lb # [ 111.691990] niks3-server[1130]: 2026/09/21 13:50:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5415lb # [ 111.700361] niks3-server[1130]: 2026/09/21 13:50:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5416lb # [ 111.708220] niks3-server[1130]: 2026/09/21 13:50:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5417lb # [ 111.713134] niks3-server[1130]: 2026/09/21 13:50:35 INFO Completed upload id=375418worker1 # [ 111.713460] nix-grpc-daemon[1349]: time=2026-09-21T13:50:35.357Z level=INFO msg="Upload complete. (487ms)"5419worker1 # [ 111.724095] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=0 paths=1 nar_bytes_in=7685420worker1 # [ 111.735365] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005421worker1 # [ 111.741579] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5422worker1 # [ 111.746671] nix-daemon[860]: reaped child process 1434, status = succeeded5423client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5424worker1 # [ 111.757942] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5425client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5426client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5427client # farm-top> LOG-farm-top-on-standby5428worker1 # [ 112.806448] nix-daemon[860]: reaped child process 1448, status = succeeded5429worker1 # [ 112.987666] nix-daemon[860]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5430worker1 # [ 113.002944] nix-daemon[860]: reaped child process 1467, status = succeeded5431lb # [ 113.018446] niks3-server[1130]: 2026/09/21 13:50:36 INFO Received uploads request method=POST path=/api/pending_closures5432worker1 # [ 113.029640] nix-grpc-daemon[1349]: time=2026-09-21T13:50:36.672Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5433worker1 # [ 113.031095] nix-grpc-daemon[1349]: time=2026-09-21T13:50:36.672Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5434lb # [ 113.051111] niks3-server[1130]: 2026/09/21 13:50:36 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5435lb # [ 113.063155] niks3-server[1130]: 2026/09/21 13:50:36 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5436lb # [ 113.073639] niks3-server[1130]: 2026/09/21 13:50:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5437worker1 # [ 113.080169] nix-grpc-daemon[1349]: time=2026-09-21T13:50:36.724Z level=INFO msg="Uploading 1 narinfos"5438lb # [ 113.079350] niks3-server[1130]: 2026/09/21 13:50:36 INFO Signed narinfos id=38 count=15439lb # [ 113.084164] niks3-server[1130]: 2026/09/21 13:50:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5440lb # [ 113.095036] niks3-server[1130]: 2026/09/21 13:50:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5441worker1 # [ 113.103063] nix-grpc-daemon[1349]: time=2026-09-21T13:50:36.747Z level=INFO msg="Upload complete. (330ms)"5442lb # [ 113.102872] niks3-server[1130]: 2026/09/21 13:50:36 INFO Completed upload id=385443lb # [ 113.107132] niks3-server[1130]: 2026/09/21 13:50:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5444worker1 # [ 113.116178] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15445worker1 # [ 113.130807] nix-daemon[860]: reaped child process 1446, status = succeeded5446client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.42 seconds)5447client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5448client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5449worker2 # [ 113.540848] systemd[1]: Listening on Nix worker-protocol over gRPC.5450worker2 # [ 113.546636] systemd[1]: Starting scheduler-token.service...5451worker2 # [ 113.640949] systemd[1]: scheduler-token.service: Deactivated successfully.5452worker2 # [ 113.644644] systemd[1]: Finished scheduler-token.service.5453worker2 # [ 113.645707] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5454worker2 # [ 113.652433] systemd[1]: Starting Nix worker-protocol over gRPC...5455worker2 # [ 113.829617] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5456worker2 # [ 113.832806] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5457worker2 # [ 113.837067] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15458worker2 # [ 113.844080] nix-grpc-daemon[1265]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5459worker2 # [ 113.845526] nix-grpc-daemon[1265]: E0000 00:00:1789998637.488069 1265 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5460worker2 # [ 113.862201] nix-grpc-daemon[1265]: E0000 00:00:1789998637.494462 1265 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5461lb: must succeed: curl -sf localhost:9901/clusters5462worker2 # [ 113.866554] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5463worker2 # [ 113.871986] systemd[1]: Started Nix worker-protocol over gRPC.5464worker2 # [ 113.874614] nix-grpc-daemon[1265]: E0000 00:00:1789998637.518913 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5465worker2 # [ 113.904153] nix-grpc-daemon[1265]: E0000 00:00:1789998637.536554 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5466worker2 # [ 113.906409] nix-grpc-daemon[1265]: E0000 00:00:1789998637.536712 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5467worker2 # [ 113.912376] nix-grpc-daemon[1265]: E0000 00:00:1789998637.555955 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5468worker2 # [ 113.925122] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:37Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 113.880141] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:37Z level=info event=oidc_jwks provider=mock keys=15470worker1 # [ 113.886287] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:430825471worker1 # [ 113.891636] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5473worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5474worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5475lb: must succeed: curl -sf localhost:9901/clusters5476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5477lb: must succeed: curl -sf localhost:9901/clusters5478lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5479worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5480worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5481(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.68 seconds)5482subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5483client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5484client # [ 114.304386] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5485client # Running as unit: rs.service; invocation ID: e5d755bda58d4ae4b7752ef4ab84a0675486client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5487worker2 # [ 114.560159] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:391465488worker2 # [ 114.575284] nix-daemon[848]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5489client # [ 114.586426] nix[1188]: don't know how to build these paths:5490client # [ 114.589316] nix[1188]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5491worker1 # [ 114.553095] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05492worker1 # [ 114.553421] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15493worker1 # [ 114.596799] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005494worker1 # [ 114.612455] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443005495client # [ 114.672200] nix[1188]: copying 1 paths to 'lb:50051'...5496worker1 # [ 114.626247] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443005497client # [ 114.676162] nix[1188]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5498worker1 # [ 114.641112] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5499worker1 # [ 114.844852] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)5500worker1 # [ 114.864162] nix-daemon[860]: reaped child process 1483, status = succeeded5501worker1 # [ 115.033090] nix-daemon[860]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)5502worker1 # [ 115.050349] nix-daemon[860]: reaped child process 1488, status = succeeded5503lb # [ 115.062322] niks3-server[1130]: 2026/09/21 13:50:38 INFO Received uploads request method=POST path=/api/pending_closures5504worker1 # [ 115.072558] nix-grpc-daemon[1349]: time=2026-09-21T13:50:38.716Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5505worker1 # [ 115.073999] nix-grpc-daemon[1349]: time=2026-09-21T13:50:38.716Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5506lb # [ 115.100804] niks3-server[1130]: 2026/09/21 13:50:38 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5507worker1 # [ 115.114327] nix-grpc-daemon[1349]: time=2026-09-21T13:50:38.758Z level=INFO msg="Uploading 1 narinfos"5508lb # [ 115.113728] niks3-server[1130]: 2026/09/21 13:50:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5509lb # [ 115.124563] niks3-server[1130]: 2026/09/21 13:50:38 INFO Signed narinfos id=39 count=15510lb # [ 115.127157] niks3-server[1130]: 2026/09/21 13:50:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5511lb # [ 115.140571] niks3-server[1130]: 2026/09/21 13:50:38 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5512lb # [ 115.146078] niks3-server[1130]: 2026/09/21 13:50:38 INFO Completed upload id=395513worker1 # [ 115.146882] nix-grpc-daemon[1349]: time=2026-09-21T13:50:38.791Z level=INFO msg="Upload complete. (489ms)"5514lb # [ 115.148462] niks3-server[1130]: 2026/09/21 13:50:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5515worker1 # [ 115.156175] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=0 paths=1 nar_bytes_in=4165516worker1 # [ 115.165552] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443005517worker1 # [ 115.176560] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5518worker1 # [ 115.181881] nix-daemon[860]: reaped child process 1478, status = succeeded5519client # [ 115.238322] nix[1188]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5520worker1 # [ 115.195423] nix-daemon[860]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5521lb: must succeed: curl -sf localhost:9901/clusters5522lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5523lb: must succeed: curl -sf localhost:9901/clusters5524lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5525lb: must succeed: curl -sf localhost:9901/clusters5526lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5527lb: must succeed: curl -sf localhost:9901/clusters5528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5529worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5530worker2 # [ 117.274087] systemd-journald[276]: Received client request to rotate journal, rotating.5531worker2 # [ 117.293900] systemd-journald[276]: Vacuuming done, freed 0B of archived journals from /var/log/journal/1480047a5416486c8d89e2a74b0a1b2a.5532worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5533worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5534worker1 # [ 117.300265] systemd[1]: Stopping Nix worker-protocol over gRPC...5535worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5536client: must succeed: sleep 5.05537worker1 # [ 118.180533] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:41Z level=info event=draining5538lb # [ 120.289018] envoy[912]: conn peer=[2001:db8:1::1]:46706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=4149 ms=71115539client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5540worker1 # [ 122.621710] nix-daemon[860]: reaped child process 1492, status = succeeded5541worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5542worker1 # [ 122.779934] nix-daemon[860]: accepted connection from pid 1533, user nix-grpc-daemon (trusted)5543worker1 # [ 122.794955] nix-daemon[860]: reaped child process 1536, status = succeeded5544lb # [ 122.810907] niks3-server[1130]: 2026/09/21 13:50:46 INFO Received uploads request method=POST path=/api/pending_closures5545worker1 # [ 122.823225] nix-grpc-daemon[1349]: time=2026-09-21T13:50:46.464Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5546worker1 # [ 122.824847] nix-grpc-daemon[1349]: time=2026-09-21T13:50:46.464Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5547lb # [ 122.847161] niks3-server[1130]: 2026/09/21 13:50:46 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5548lb # [ 122.860496] niks3-server[1130]: 2026/09/21 13:50:46 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5549lb # [ 122.876096] niks3-server[1130]: 2026/09/21 13:50:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5550worker1 # [ 122.881301] nix-grpc-daemon[1349]: time=2026-09-21T13:50:46.525Z level=INFO msg="Uploading 1 narinfos"5551lb # [ 122.885847] niks3-server[1130]: 2026/09/21 13:50:46 INFO Signed narinfos id=40 count=15552lb # [ 122.890763] niks3-server[1130]: 2026/09/21 13:50:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5553lb # [ 122.898841] niks3-server[1130]: 2026/09/21 13:50:46 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5554lb # [ 122.907141] niks3-server[1130]: 2026/09/21 13:50:46 INFO Completed upload id=405555worker1 # [ 122.907989] nix-grpc-daemon[1349]: time=2026-09-21T13:50:46.552Z level=INFO msg="Upload complete. (297ms)"5556lb # [ 122.910794] niks3-server[1130]: 2026/09/21 13:50:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5557worker1 # [ 122.924990] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44300 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15558worker1 # [ 122.938150] nix-daemon[860]: reaped child process 1490, status = succeeded5559client # [ 123.008513] systemd[1]: rs.service: Deactivated successfully.5560client # [ 123.013684] systemd[1]: rs.service: Consumed 173ms CPU time over 8.707s wall clock time, 39.2M memory peak, 7K incoming IP traffic, 8.4K outgoing IP traffic.5561worker1 # [ 123.181937] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:46Z level=info event=signal_exit5562lb # [ 123.736321] niks3-server[1130]: 2026/09/21 13:50:47 INFO lead: released remote=192.168.1.3:476205563lb # [ 123.811770] niks3-server[1130]: 2026/09/21 13:50:47 INFO lead: acquired remote=192.168.1.4:362665564worker2 # [ 123.870502] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:47Z level=info event=scheduler_take_over5565worker2 # [ 124.785050] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:48Z level=info event=scheduler_restarting addr=lb:500515566worker1 # [ 124.737239] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=scheduler_restarting5567worker1 # [ 124.742145] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=scheduler_restarting addr=lb:500515568worker1 # [ 124.747324] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430825569worker1 # [ 124.752850] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=worker_gone addr=192.168.1.4:500515570worker1 # [ 124.757369] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430825571worker1 # [ 124.762462] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=worker_gone addr=192.168.1.3:500515572worker1 # [ 124.787170] nix-daemon[860]: reaped child process 1330, status = succeeded5573worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5574client: waiting for success: ! systemctl is-active rs5575worker2 # [ 124.987754] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:48Z level=info event=scheduler_connected addr=lb:500515576client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5577client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5578worker2 # [ 125.004142] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:48Z level=info event=oidc_jwks provider=mock keys=15579worker1 # [ 124.956418] nix-grpc-daemon[1276]: ts=2026-09-21T13:50:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5580worker2 # [ 125.008940] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:521905581worker2 # [ 125.014200] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05582client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5583client: must fail: journalctl -u rs | grep -q 'reconnecting'5584client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5585lb: must succeed: curl -sf localhost:9901/clusters5586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5587worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5588worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5589worker1 # [ 125.595660] nix-grpc-daemon[1276]: E0000 00:00:1789998649.239522 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5590worker1 # [ 125.602279] nix-grpc-daemon[1276]: E0000 00:00:1789998649.239777 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5591worker1 # [ 126.031648] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5592worker1 # [ 126.050537] systemd[1]: Stopped Nix worker-protocol over gRPC.5593worker1 # [ 126.055010] systemd[1]: nix-grpc-daemon.service: Consumed 1.820s CPU time over 36.079s wall clock time, 36.5M memory peak, 46K incoming IP traffic, 84.1K outgoing IP traffic.5594worker1 # [ 126.063187] systemd[1]: Starting Nix worker-protocol over gRPC...5595lb: must succeed: curl -sf localhost:9901/clusters5596worker1 # [ 126.260690] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5597lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5598worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5599worker1 # [ 126.262695] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5600worker1 # [ 126.266831] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15601worker1 # [ 126.274015] nix-grpc-daemon[1543]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5602worker1 # [ 126.275435] nix-grpc-daemon[1543]: E0000 00:00:1789998649.917559 1543 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5603worker1 # [ 126.282951] nix-grpc-daemon[1543]: E0000 00:00:1789998649.923731 1543 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5604worker1 # [ 126.286736] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5605worker1 # [ 126.289232] systemd[1]: Started Nix worker-protocol over gRPC.5606worker1 # [ 126.296054] nix-grpc-daemon[1543]: E0000 00:00:1789998649.936973 1585 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5607worker1 # [ 126.320056] nix-grpc-daemon[1543]: E0000 00:00:1789998649.954666 1585 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5608worker1 # [ 126.322331] nix-grpc-daemon[1543]: E0000 00:00:1789998649.954826 1585 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5609worker1 # [ 126.330787] nix-grpc-daemon[1543]: E0000 00:00:1789998649.974061 1585 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5610worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5611worker1 # [ 126.343132] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:49Z level=info event=scheduler_connected addr=lb:500515612worker2 # [ 126.397950] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:521905613worker2 # [ 126.401328] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5616worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5617worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5618(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.31 seconds)5619subtest: scheduler restart mid-build: build finishes, no double build5620client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5621client # [ 127.612763] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5622client # Running as unit: mid.service; invocation ID: cccd39c951514ec8b2adddf1e802593c5623client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5624worker1 # [ 127.825226] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440265625worker1 # [ 127.840669] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5626client # [ 127.903501] nix[1224]: don't know how to build these paths:5627client # [ 127.905567] nix[1224]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5628worker2 # [ 127.923341] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:51Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05629worker2 # [ 127.923633] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:51Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15630worker2 # [ 127.966861] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391465631worker2 # [ 127.986355] nix-grpc-daemon[1265]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)5632worker2 # [ 128.033846] nix-grpc-daemon[1265]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)5633worker2 # [ 128.195276] nix-grpc-daemon[1265]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 3/5)5634worker2 # [ 128.239326] nix-grpc-daemon[1265]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 678 ms (attempt 4/5)5635worker2 # [ 128.929018] nix-grpc-daemon[1265]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5636worker2 # [ 128.933614] nix-grpc-daemon[1265]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5637worker2 # [ 128.946389] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391465638client # [ 128.957407] nix[1224]: copying 1 paths to 'lb:50051'...5639worker2 # [ 128.963832] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:391465640client # [ 128.961938] nix[1224]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5641worker2 # [ 128.980617] nix-daemon[848]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5642worker2 # [ 129.005511] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:52Z level=info event=niks3_push_started pid=13945643worker2 # [ 129.204302] nix-grpc-daemon[1394]: time=2026-09-21T13:50:52.848Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5644worker2 # [ 129.384613] nix-daemon[848]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5645worker2 # [ 129.401307] nix-daemon[848]: reaped child process 1410, status = succeeded5646worker2 # [ 129.569995] nix-daemon[848]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5647worker2 # [ 129.586190] nix-daemon[848]: reaped child process 1415, status = succeeded5648lb # [ 129.549802] niks3-server[1130]: 2026/09/21 13:50:53 INFO Received uploads request method=POST path=/api/pending_closures5649worker2 # [ 129.616068] nix-grpc-daemon[1394]: time=2026-09-21T13:50:53.259Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5650worker2 # [ 129.617557] nix-grpc-daemon[1394]: time=2026-09-21T13:50:53.259Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5651lb # [ 129.585664] niks3-server[1130]: 2026/09/21 13:50:53 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5652lb # [ 129.598622] niks3-server[1130]: 2026/09/21 13:50:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5653worker2 # [ 129.658318] nix-grpc-daemon[1394]: time=2026-09-21T13:50:53.302Z level=INFO msg="Uploading 1 narinfos"5654lb # [ 129.610353] niks3-server[1130]: 2026/09/21 13:50:53 INFO Signed narinfos id=41 count=15655lb # [ 129.616712] niks3-server[1130]: 2026/09/21 13:50:53 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5656lb # [ 129.629897] niks3-server[1130]: 2026/09/21 13:50:53 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5657lb # [ 129.639096] niks3-server[1130]: 2026/09/21 13:50:53 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5658lb # [ 129.671205] niks3-server[1130]: 2026/09/21 13:50:53 INFO Completed upload id=415659worker2 # [ 129.725202] nix-grpc-daemon[1394]: time=2026-09-21T13:50:53.369Z level=INFO msg="Upload complete. (516ms)"5660worker2 # [ 129.736307] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39146 duration_s=0 paths=1 nar_bytes_in=4245661worker2 # [ 129.756271] nix-grpc-daemon[1265]: ts=2026-09-21T13:50:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391465662worker2 # [ 129.761955] nix-daemon[848]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5663worker2 # [ 129.769220] nix-daemon[848]: reaped child process 1392, status = succeeded5664client # [ 129.772571] nix[1224]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5665worker2 # [ 129.782993] nix-daemon[848]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5666lb # [ 130.292219] envoy[912]: conn peer=[2001:db8:1::1]:59508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1404 ms=84295667lb # [ 130.297009] envoy[912]: conn peer=[2001:db8:1::3]:41242 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2008 tx=752 ms=347165668lb: must succeed: curl -sf localhost:9901/clusters5669lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5670lb: must succeed: curl -sf localhost:9901/clusters5671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5672worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5673worker2 # [ 131.669592] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1265 (nix-grpc-daemon) on client request.5674worker2 # [ 131.671219] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5675lb # [ 131.637991] niks3-server[1130]: 2026/09/21 13:50:55 INFO lead: released remote=192.168.1.4:362665676worker2 # [ 131.701439] systemd[1]: Starting scheduler-token.service...5677client # [ 131.694368] nix[1224]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5678worker1 # [ 131.647219] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5679client # [ 131.700576] nix[1224]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5680worker2 # [ 131.727443] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5681worker2 # [ 131.729602] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5682worker2 # [ 131.731089] systemd[1]: nix-grpc-daemon.service: Consumed 541ms CPU time over 18.066s wall clock time, 34.5M memory peak, 15.8K incoming IP traffic, 27.3K outgoing IP traffic.5683worker2 # [ 131.743103] nix-daemon[848]: reaped child process 1417, status = succeeded5684worker2 # [ 131.745430] nix-daemon[848]: reaped child process 1324, status = succeeded5685worker2 # [ 131.755867] nix-daemon[1419]: unexpected Nix daemon error: error: interrupted by the user5686worker2 # [ 131.760252] nix-daemon[848]: reaped child process 1419, status = failed with exit code 15687worker2 # [ 131.809405] systemd[1]: scheduler-token.service: Deactivated successfully.5688worker2 # [ 131.810497] systemd[1]: Finished scheduler-token.service.5689worker2 # [ 131.811493] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 105ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5690worker2 # [ 131.813658] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5691worker2 # [ 131.816542] systemd[1]: Starting scheduler-token.service...5692worker2 # [ 131.885633] systemd[1]: scheduler-token.service: Deactivated successfully.5693worker2 # [ 131.887673] systemd[1]: Finished scheduler-token.service.5694worker2 # [ 131.890566] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 71ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5695worker2 # [ 131.894141] systemd[1]: Starting Nix worker-protocol over gRPC...5696worker1 # [ 131.847343] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:55Z level=info event=scheduler_connected addr=lb:500515697worker2 # [ 132.058093] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5698worker2 # [ 132.060887] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5699worker2 # [ 132.064867] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15700worker2 # [ 132.072076] nix-grpc-daemon[1462]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5701worker2 # [ 132.087989] nix-grpc-daemon[1462]: E0000 00:00:1789998655.715838 1462 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5702lb # [ 132.036626] niks3-server[1130]: 2026/09/21 13:50:55 INFO lead: acquired remote=192.168.1.4:583565703worker2 # [ 132.093254] nix-grpc-daemon[1462]: E0000 00:00:1789998655.719442 1462 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5704worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)5705worker2 # [ 132.097793] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5706client: must succeed: sleep 5.05707worker2 # [ 132.099712] systemd[1]: Started Nix worker-protocol over gRPC.5708worker1 # [ 132.042877] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5709worker2 # [ 132.108169] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=scheduler_take_over5710worker2 # [ 132.109319] nix-grpc-daemon[1462]: E0000 00:00:1789998655.747011 1502 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5711worker2 # [ 132.136859] nix-grpc-daemon[1462]: E0000 00:00:1789998655.765866 1502 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5712worker2 # [ 132.139554] nix-grpc-daemon[1462]: E0000 00:00:1789998655.766005 1502 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5713worker2 # [ 132.142207] nix-grpc-daemon[1462]: E0000 00:00:1789998655.780693 1502 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5714worker2 # [ 132.149705] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=scheduler_connected addr=lb:500515715worker2 # [ 132.155312] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=oidc_jwks provider=mock keys=15716worker2 # [ 132.156812] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:448985717worker2 # [ 132.158480] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05718worker2 # [ 132.207597] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05719worker2 # [ 132.211010] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15720worker2 # [ 132.257833] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449105721worker2 # [ 132.270099] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5722worker2 # [ 132.283589] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5723worker2 # [ 132.297241] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5724client # [ 132.290004] nix[1224]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5725worker1 # [ 132.243920] nix-grpc-daemon[1543]: ts=2026-09-21T13:50:55Z level=info event=scheduler_connected addr=lb:500515726worker2 # [ 132.301936] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:448985727worker2 # [ 132.305598] nix-grpc-daemon[1462]: ts=2026-09-21T13:50:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05728client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5729client: waiting for success: ! systemctl is-active mid5730worker2 # [ 137.522620] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:01Z level=info event=niks3_push_started pid=15455731worker2 # [ 137.540799] nix-daemon[848]: reaped child process 1520, status = succeeded5732worker2 # [ 137.718373] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.362Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5733worker2 # [ 137.889937] nix-daemon[848]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)5734worker2 # [ 137.906579] nix-daemon[848]: reaped child process 1561, status = succeeded5735lb # [ 137.866474] niks3-server[1130]: 2026/09/21 13:51:01 INFO Received uploads request method=POST path=/api/pending_closures5736worker2 # [ 137.931417] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.574Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5737worker2 # [ 137.934148] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.574Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5738lb # [ 137.907001] niks3-server[1130]: 2026/09/21 13:51:01 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5739lb # [ 137.922329] niks3-server[1130]: 2026/09/21 13:51:01 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5740worker2 # [ 137.994844] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.638Z level=INFO msg="Uploading 1 narinfos"5741lb # [ 137.933123] niks3-server[1130]: 2026/09/21 13:51:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5742lb # [ 137.945526] niks3-server[1130]: 2026/09/21 13:51:01 INFO Signed narinfos id=42 count=15743lb # [ 137.960222] niks3-server[1130]: 2026/09/21 13:51:01 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5744worker2 # [ 138.019733] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.664Z level=INFO msg="Upload complete. (297ms)"5745lb # [ 137.973802] niks3-server[1130]: 2026/09/21 13:51:01 INFO Completed upload id=425746worker2 # [ 138.036249] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44910 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15747lb # [ 137.991916] niks3-server[1130]: 2026/09/21 13:51:01 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5748worker2 # [ 138.046999] nix-grpc-daemon[1545]: time=2026-09-21T13:51:01.691Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5749worker2 # [ 138.051888] nix-daemon[848]: reaped child process 1518, status = succeeded5750client # [ 138.069737] systemd[1]: mid.service: Deactivated successfully.5751client # [ 138.073819] systemd[1]: mid.service: Consumed 187ms CPU time over 10.460s wall clock time, 43.2M memory peak, 8.8K incoming IP traffic, 10.4K outgoing IP traffic.5752client: (finished: waiting for success: ! systemctl is-active mid, in 1.13 seconds)5753client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5754client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5755(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.10 seconds)5756subtest: interrupting the client stops the build on the worker5757client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5758client # [ 138.687140] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5759client # Running as unit: intr.service; invocation ID: c40ab8a3301f4538a271c7753969ee6d5760client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5761worker2 # [ 138.930116] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:449105762client # [ 138.931376] nix[1260]: don't know how to build these paths:5763client # [ 138.933319] nix[1260]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5764worker2 # [ 138.951217] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:02Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05765worker2 # [ 138.957055] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:02Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15766worker1 # [ 138.945858] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265767worker1 # [ 138.963194] nix-grpc-daemon[1543]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)5768worker1 # [ 139.062666] nix-grpc-daemon[1543]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 2/5)5769worker1 # [ 139.134818] nix-grpc-daemon[1543]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 3/5)5770worker1 # [ 139.156313] nix-grpc-daemon[1543]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 524 ms (attempt 4/5)5771worker1 # [ 139.688541] nix-grpc-daemon[1543]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5772worker1 # [ 139.694266] nix-grpc-daemon[1543]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5773worker1 # [ 139.713230] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440265774client # [ 139.775631] nix[1260]: copying 1 paths to 'lb:50051'...5775worker1 # [ 139.728633] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440265776client # [ 139.776066] nix[1260]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5777worker1 # [ 139.743845] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5778worker1 # [ 139.771489] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:03Z level=info event=niks3_push_started pid=16355779worker1 # [ 140.002122] nix-grpc-daemon[1635]: time=2026-09-21T13:51:03.645Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5780worker1 # [ 140.179879] nix-daemon[860]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)5781worker1 # [ 140.196192] nix-daemon[860]: reaped child process 1655, status = succeeded5782lb # [ 140.293024] envoy[912]: conn peer=[2001:db8:1::4]:40856 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1918 tx=311 ms=177825783lb # [ 140.293528] envoy[912]: conn peer=[2001:db8:1::1]:53090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1854 ms=101715784worker1 # [ 140.370756] nix-daemon[860]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5785worker1 # [ 140.387159] nix-daemon[860]: reaped child process 1660, status = succeeded5786lb # [ 140.402446] niks3-server[1130]: 2026/09/21 13:51:04 INFO Received uploads request method=POST path=/api/pending_closures5787worker1 # [ 140.422328] nix-grpc-daemon[1635]: time=2026-09-21T13:51:04.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5788worker1 # [ 140.423810] nix-grpc-daemon[1635]: time=2026-09-21T13:51:04.065Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5789lb # [ 140.448943] niks3-server[1130]: 2026/09/21 13:51:04 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5790worker1 # [ 140.467714] nix-grpc-daemon[1635]: time=2026-09-21T13:51:04.111Z level=INFO msg="Uploading 1 narinfos"5791lb # [ 140.460935] niks3-server[1130]: 2026/09/21 13:51:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5792lb # [ 140.473239] niks3-server[1130]: 2026/09/21 13:51:04 INFO Signed narinfos id=43 count=15793lb # [ 140.479283] niks3-server[1130]: 2026/09/21 13:51:04 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5794lb # [ 140.486324] niks3-server[1130]: 2026/09/21 13:51:04 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5795lb # [ 140.499107] niks3-server[1130]: 2026/09/21 13:51:04 INFO Completed upload id=435796worker1 # [ 140.500112] nix-grpc-daemon[1635]: time=2026-09-21T13:51:04.144Z level=INFO msg="Upload complete. (494ms)"5797lb # [ 140.509212] niks3-server[1130]: 2026/09/21 13:51:04 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5798worker1 # [ 140.511020] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=4245799worker1 # [ 140.525093] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265800worker1 # [ 140.533551] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5801worker1 # [ 140.536358] nix-daemon[860]: reaped child process 1633, status = succeeded5802client # [ 140.593956] nix[1260]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5803worker1 # [ 140.549766] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5804client: must succeed: systemctl kill -s INT intr5805client # [ 142.403808] systemd[1]: intr.service: Sent signal SIGINT to main process 1260 (nix) on client request.5806worker1 # [ 142.365223] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:06Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5807client # [ 142.412589] nix[1260]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5808client # [ 142.424143] nix[1260]: error: interrupted by the user5809client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5810client # [ 142.460739] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5811client # [ 142.466460] systemd[1]: intr.service: Failed with result 'exit-code'.5812client # [ 142.469274] systemd[1]: intr.service: Consumed 160ms CPU time over 3.777s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5813worker1 # [ 142.579478] nix-daemon[860]: reaped child process 1662, status = succeeded5814worker1 # [ 142.585284] nix-daemon[1664]: unexpected Nix daemon error: error: interrupted by the user5815worker1 # [ 142.590193] nix-daemon[860]: reaped child process 1664, status = failed with exit code 15816(finished: subtest: interrupting the client stops the build on the worker, in 5.09 seconds)5817subtest: a deploy mid-build drains: the build finishes, the new generation takes over5818client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5819client # [ 143.803333] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5820client # Running as unit: sw.service; invocation ID: 103031c328f54394acd35c953f6ae9ae5821client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.10 seconds)5822worker2 # [ 144.061536] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:449105823client # [ 144.062826] nix[1280]: don't know how to build these paths:5824client # [ 144.063871] nix[1280]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5825worker2 # [ 144.083046] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:07Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05826worker2 # [ 144.090234] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:07Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15827worker2 # [ 144.129663] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449105828worker2 # [ 144.146224] nix-grpc-daemon[1462]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 1/5)5829worker2 # [ 144.230480] nix-grpc-daemon[1462]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)5830worker2 # [ 144.381203] nix-grpc-daemon[1462]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 3/5)5831worker2 # [ 144.446666] nix-grpc-daemon[1462]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 702 ms (attempt 4/5)5832worker2 # [ 145.158319] nix-grpc-daemon[1462]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5833worker2 # [ 145.160809] nix-grpc-daemon[1462]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5834worker2 # [ 145.177767] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449105835client # [ 145.183071] nix[1280]: copying 1 paths to 'lb:50051'...5836worker2 # [ 145.188947] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449105837client # [ 145.188261] nix[1280]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5838worker2 # [ 145.202668] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5839worker2 # [ 145.388712] nix-daemon[848]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)5840worker2 # [ 145.405391] nix-daemon[848]: reaped child process 1608, status = succeeded5841worker2 # [ 145.580341] nix-daemon[848]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)5842worker2 # [ 145.595575] nix-daemon[848]: reaped child process 1613, status = succeeded5843lb # [ 145.558099] niks3-server[1130]: 2026/09/21 13:51:09 INFO Received uploads request method=POST path=/api/pending_closures5844worker2 # [ 145.626996] nix-grpc-daemon[1545]: time=2026-09-21T13:51:09.270Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5845worker2 # [ 145.628526] nix-grpc-daemon[1545]: time=2026-09-21T13:51:09.270Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5846lb # [ 145.593379] niks3-server[1130]: 2026/09/21 13:51:09 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5847lb # [ 145.604569] niks3-server[1130]: 2026/09/21 13:51:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5848worker2 # [ 145.664952] nix-grpc-daemon[1545]: time=2026-09-21T13:51:09.309Z level=INFO msg="Uploading 1 narinfos"5849lb # [ 145.615573] niks3-server[1130]: 2026/09/21 13:51:09 INFO Signed narinfos id=44 count=15850lb # [ 145.621489] niks3-server[1130]: 2026/09/21 13:51:09 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5851lb # [ 145.629443] niks3-server[1130]: 2026/09/21 13:51:09 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5852lb # [ 145.636369] niks3-server[1130]: 2026/09/21 13:51:09 INFO Completed upload id=445853worker2 # [ 145.690039] nix-grpc-daemon[1545]: time=2026-09-21T13:51:09.334Z level=INFO msg="Upload complete. (476ms)"5854lb # [ 145.639160] niks3-server[1130]: 2026/09/21 13:51:09 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5855worker2 # [ 145.701055] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44910 duration_s=0 paths=1 nar_bytes_in=4165856worker2 # [ 145.710801] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449105857worker2 # [ 145.719842] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5858worker2 # [ 145.722348] nix-daemon[848]: reaped child process 1603, status = succeeded5859client # [ 145.726078] nix[1280]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5860worker2 # [ 145.735982] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5861worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25862worker2 # Checking switch inhibitors... done5863worker2 # [ 148.476727] nixos[1649]: switching to system configuration /nix/store/5r602b45pssx56b8iya1nrpq8v9x41is-nixos-system-worker2-test5864worker2 # [ 148.481547] systemd[1]: Stopped target Local File Systems.5865worker2 # [ 148.484654] systemd[1]: Stopped target Remote File Systems.5866worker2 # activating the configuration...5867worker2 # setting up /etc...5868worker2 # [ 149.275261] systemd[1]: Reload requested from client PID 1649 ('.switch-to-conf') (unit backdoor.service)...5869worker2 # [ 149.277679] systemd[1]: Reloading...5870worker2 # [ 150.164170] systemd[1]: Reloading finished in 883 ms.5871worker2 # restarting sysinit-reactivation.target5872worker2 # [ 150.248448] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5873lb # [ 150.289104] envoy[912]: conn peer=[2001:db8:1::1]:48380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=826 ms=35115874worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5875worker2 # [ 150.401406] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5876worker2 # [ 150.405217] systemd[1]: Reached target Reactivate sysinit units.5877worker2 # [ 150.412513] systemd[1]: Reloading D-Bus System Message Bus...5878worker2 # [ 150.417754] dbus-broker-launch[503]: Caught SIGHUP, trigger reload.5879worker2 # [ 150.420455] systemd[1]: Reloading Nix worker-protocol over gRPC...5880worker2 # [ 150.435592] dbus-broker-launch[503]: Looking up NSS user entry for 'systemd-timesync'...5881worker2 # [ 150.440359] dbus-broker-launch[503]: NSS returned no entry for 'systemd-timesync'5882worker2 # [ 150.441609] dbus-broker-launch[503]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5883worker2 # [ 150.464095] systemd[1]: Reloaded Nix worker-protocol over gRPC.5884worker2 # [ 150.489175] systemd[1]: Reloaded D-Bus System Message Bus.5885worker2 # [ 150.498904] systemd[1]: Reached target Local File Systems.5886worker2 # [ 150.499720] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5887worker2 # [ 150.506278] systemd[1]: Starting Create SUID/SGID Wrappers...5888worker2 # [ 150.516125] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5889worker2 # [ 150.517395] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675890worker2 # [ 150.519477] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5891worker2 # [ 150.528109] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5892worker2 # [ 150.529388] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5893worker2 # [ 150.530651] systemd[1]: Reached target Remote File Systems.5894worker2 # [ 150.531418] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5895worker2 # [ 150.540228] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5896worker2 # [ 151.003017] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5897worker2 # [ 151.005437] systemd[1]: Finished Create SUID/SGID Wrappers.5898worker2 # [ 151.008506] systemd[1]: Starting linger-users.service...5899worker2 # [ 151.060885] systemd[1]: linger-users.service: Deactivated successfully.5900worker2 # [ 151.061864] systemd[1]: Finished linger-users.service.5901worker2 # [ 151.084257] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:14Z level=info event=draining5902worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5903worker2 # [ 151.314787] nixos[1649]: finished switching to system configuration /nix/store/5r602b45pssx56b8iya1nrpq8v9x41is-nixos-system-worker2-test5904worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.66 seconds)5905worker2: must succeed: pgrep -f 'read -t [9]0 x'5906worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5907client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25908client # don't know how to build these paths:5909client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5910worker1 # [ 151.600422] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440265911worker1 # [ 151.607833] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440265912worker2 # [ 151.683077] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:15Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05913worker2 # [ 151.685122] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:15Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15914worker1 # [ 151.681722] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265915worker1 # [ 151.696565] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440265916client # copying 1 paths to 'lb:50051'...5917worker1 # [ 151.712437] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440265918client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5919worker1 # [ 151.736613] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5920worker1 # [ 151.943919] nix-daemon[860]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)5921worker1 # [ 151.959906] nix-daemon[860]: reaped child process 1722, status = succeeded5922worker1 # [ 152.136899] nix-daemon[860]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)5923worker1 # [ 152.152920] nix-daemon[860]: reaped child process 1727, status = succeeded5924lb # [ 152.166128] niks3-server[1130]: 2026/09/21 13:51:15 INFO Received uploads request method=POST path=/api/pending_closures5925worker1 # [ 152.175438] nix-grpc-daemon[1635]: time=2026-09-21T13:51:15.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5926worker1 # [ 152.177059] nix-grpc-daemon[1635]: time=2026-09-21T13:51:15.819Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5927lb # [ 152.194197] niks3-server[1130]: 2026/09/21 13:51:15 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5928lb # [ 152.206903] niks3-server[1130]: 2026/09/21 13:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5929worker1 # [ 152.213844] nix-grpc-daemon[1635]: time=2026-09-21T13:51:15.857Z level=INFO msg="Uploading 1 narinfos"5930lb # [ 152.217593] niks3-server[1130]: 2026/09/21 13:51:15 INFO Signed narinfos id=45 count=15931lb # [ 152.223490] niks3-server[1130]: 2026/09/21 13:51:15 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5932lb # [ 152.230577] niks3-server[1130]: 2026/09/21 13:51:15 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5933lb # [ 152.236313] niks3-server[1130]: 2026/09/21 13:51:15 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5934lb # [ 152.240817] niks3-server[1130]: 2026/09/21 13:51:15 INFO Completed upload id=455935worker1 # [ 152.241602] nix-grpc-daemon[1635]: time=2026-09-21T13:51:15.885Z level=INFO msg="Upload complete. (488ms)"5936worker1 # [ 152.248829] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=5285937worker1 # [ 152.262610] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265938worker1 # [ 152.268972] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5939worker1 # [ 152.273755] nix-daemon[860]: reaped child process 1717, status = succeeded5940client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5941worker1 # [ 152.284809] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5942client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5943client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5944client # farm-a> LOG-farm-a-during-drain5945worker1 # [ 153.336298] nix-daemon[860]: reaped child process 1731, status = succeeded5946worker1 # [ 153.517942] nix-daemon[860]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)5947worker1 # [ 153.534629] nix-daemon[860]: reaped child process 1750, status = succeeded5948lb # [ 153.546460] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received uploads request method=POST path=/api/pending_closures5949worker1 # [ 153.560167] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5950worker1 # [ 153.561794] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.201Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5951lb # [ 153.584925] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5952lb # [ 153.599079] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5953lb # [ 153.607522] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5954worker1 # [ 153.614822] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.257Z level=INFO msg="Uploading 1 narinfos"5955lb # [ 153.613720] niks3-server[1130]: 2026/09/21 13:51:17 INFO Signed narinfos id=46 count=15956lb # [ 153.618144] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5957lb # [ 153.628089] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5958lb # [ 153.639601] niks3-server[1130]: 2026/09/21 13:51:17 INFO Completed upload id=465959worker1 # [ 153.640415] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.284Z level=INFO msg="Upload complete. (340ms)"5960lb # [ 153.642855] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5961worker2 # [ 153.712038] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:17Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05962worker1 # [ 153.653098] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15963worker2 # [ 153.713856] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:17Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15964worker1 # [ 153.671760] nix-daemon[860]: reaped child process 1729, status = succeeded5965worker1 # [ 153.705407] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265966worker1 # [ 153.720318] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440265967worker1 # [ 153.734292] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.378Z level=INFO msg="All 1 paths already cached"5968client # copying 1 paths to 'lb:50051'...5969worker1 # [ 153.743078] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440265970client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5971worker1 # [ 153.768453] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5972worker1 # [ 153.970603] nix-daemon[860]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)5973worker1 # [ 153.988388] nix-daemon[860]: reaped child process 1757, status = succeeded5974worker1 # [ 154.161973] nix-daemon[860]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)5975worker1 # [ 154.177737] nix-daemon[860]: reaped child process 1762, status = succeeded5976lb # [ 154.187956] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received uploads request method=POST path=/api/pending_closures5977worker1 # [ 154.198310] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5978worker1 # [ 154.199888] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.842Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5979lb # [ 154.223795] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5980lb # [ 154.235720] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5981worker1 # [ 154.244108] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.887Z level=INFO msg="Uploading 1 narinfos"5982lb # [ 154.243432] niks3-server[1130]: 2026/09/21 13:51:17 INFO Signed narinfos id=47 count=15983lb # [ 154.250155] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5984lb # [ 154.263428] niks3-server[1130]: 2026/09/21 13:51:17 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5985worker1 # [ 154.270797] nix-grpc-daemon[1635]: time=2026-09-21T13:51:17.914Z level=INFO msg="Upload complete. (485ms)"5986lb # [ 154.270668] niks3-server[1130]: 2026/09/21 13:51:17 INFO Completed upload id=475987lb # [ 154.274978] niks3-server[1130]: 2026/09/21 13:51:17 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5988worker1 # [ 154.280249] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=6405989worker1 # [ 154.289103] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440265990worker1 # [ 154.303313] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5991worker1 # [ 154.305862] nix-daemon[860]: reaped child process 1752, status = succeeded5992client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5993client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5994worker1 # [ 154.320267] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5995client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5996client # farm-b> LOG-farm-b-during-drain5997worker1 # [ 155.565949] nix-daemon[860]: reaped child process 1766, status = succeeded5998worker1 # [ 155.747224] nix-daemon[860]: accepted connection from pid 1782, user nix-grpc-daemon (trusted)5999worker1 # [ 155.764960] nix-daemon[860]: reaped child process 1785, status = succeeded6000lb # [ 155.778781] niks3-server[1130]: 2026/09/21 13:51:19 INFO Received uploads request method=POST path=/api/pending_closures6001worker1 # [ 155.788665] nix-grpc-daemon[1635]: time=2026-09-21T13:51:19.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6002worker1 # [ 155.790936] nix-grpc-daemon[1635]: time=2026-09-21T13:51:19.434Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6003lb # [ 155.818511] niks3-server[1130]: 2026/09/21 13:51:19 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6004worker1 # [ 155.832691] nix-grpc-daemon[1635]: time=2026-09-21T13:51:19.476Z level=INFO msg="Uploading 1 narinfos"6005lb # [ 155.833640] niks3-server[1130]: 2026/09/21 13:51:19 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6006lb # [ 155.842658] niks3-server[1130]: 2026/09/21 13:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6007lb # [ 155.848671] niks3-server[1130]: 2026/09/21 13:51:19 INFO Signed narinfos id=48 count=16008lb # [ 155.854714] niks3-server[1130]: 2026/09/21 13:51:19 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6009worker1 # [ 155.858965] nix-grpc-daemon[1635]: time=2026-09-21T13:51:19.503Z level=INFO msg="Upload complete. (326ms)"6010lb # [ 155.858648] niks3-server[1130]: 2026/09/21 13:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6011lb # [ 155.862454] niks3-server[1130]: 2026/09/21 13:51:19 INFO Completed upload id=486012lb # [ 155.865354] niks3-server[1130]: 2026/09/21 13:51:19 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6013worker2 # [ 155.926910] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:19Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06014worker1 # [ 155.869069] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16015worker2 # [ 155.928973] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:19Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16016worker1 # [ 155.887329] nix-daemon[860]: reaped child process 1764, status = succeeded6017worker1 # [ 155.925431] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266018worker1 # [ 155.940778] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266019worker1 # [ 155.955270] nix-grpc-daemon[1635]: time=2026-09-21T13:51:19.599Z level=INFO msg="All 2 paths already cached"6020client # copying 1 paths to 'lb:50051'...6021worker1 # [ 155.966105] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266022client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6023worker1 # [ 155.990937] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6024worker1 # [ 156.193388] nix-daemon[860]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6025worker1 # [ 156.210282] nix-daemon[860]: reaped child process 1792, status = succeeded6026worker1 # [ 156.379140] nix-daemon[860]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6027worker1 # [ 156.394800] nix-daemon[860]: reaped child process 1797, status = succeeded6028lb # [ 156.405459] niks3-server[1130]: 2026/09/21 13:51:20 INFO Received uploads request method=POST path=/api/pending_closures6029worker1 # [ 156.415310] nix-grpc-daemon[1635]: time=2026-09-21T13:51:20.059Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6030worker1 # [ 156.416911] nix-grpc-daemon[1635]: time=2026-09-21T13:51:20.059Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6031lb # [ 156.432466] niks3-server[1130]: 2026/09/21 13:51:20 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6032worker1 # [ 156.444562] nix-grpc-daemon[1635]: time=2026-09-21T13:51:20.088Z level=INFO msg="Uploading 1 narinfos"6033lb # [ 156.447850] niks3-server[1130]: 2026/09/21 13:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6034lb # [ 156.458236] niks3-server[1130]: 2026/09/21 13:51:20 INFO Signed narinfos id=49 count=16035lb # [ 156.463170] niks3-server[1130]: 2026/09/21 13:51:20 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6036worker1 # [ 156.472913] nix-grpc-daemon[1635]: time=2026-09-21T13:51:20.117Z level=INFO msg="Upload complete. (464ms)"6037lb # [ 156.472819] niks3-server[1130]: 2026/09/21 13:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6038lb # [ 156.477663] niks3-server[1130]: 2026/09/21 13:51:20 INFO Completed upload id=496039worker1 # [ 156.480118] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=7686040lb # [ 156.480951] niks3-server[1130]: 2026/09/21 13:51:20 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6041worker1 # [ 156.490289] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266042worker1 # [ 156.495649] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6043worker1 # [ 156.500292] nix-daemon[860]: reaped child process 1787, status = succeeded6044client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6045worker1 # [ 156.511944] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6046client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6047client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6048client # farm-top> LOG-farm-top-during-drain6049worker1 # [ 157.948439] nix-daemon[860]: reaped child process 1801, status = succeeded6050worker1 # [ 158.124150] nix-daemon[860]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6051worker1 # [ 158.140848] nix-daemon[860]: reaped child process 1820, status = succeeded6052lb # [ 158.152907] niks3-server[1130]: 2026/09/21 13:51:21 INFO Received uploads request method=POST path=/api/pending_closures6053worker1 # [ 158.164439] nix-grpc-daemon[1635]: time=2026-09-21T13:51:21.807Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6054worker1 # [ 158.166183] nix-grpc-daemon[1635]: time=2026-09-21T13:51:21.807Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6055lb # [ 158.194950] niks3-server[1130]: 2026/09/21 13:51:21 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6056lb # [ 158.208309] niks3-server[1130]: 2026/09/21 13:51:21 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6057lb # [ 158.217703] niks3-server[1130]: 2026/09/21 13:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6058worker1 # [ 158.227227] nix-grpc-daemon[1635]: time=2026-09-21T13:51:21.870Z level=INFO msg="Uploading 1 narinfos"6059lb # [ 158.229407] niks3-server[1130]: 2026/09/21 13:51:21 INFO Signed narinfos id=50 count=16060lb # [ 158.235865] niks3-server[1130]: 2026/09/21 13:51:21 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6061lb # [ 158.242775] niks3-server[1130]: 2026/09/21 13:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6062lb # [ 158.252227] niks3-server[1130]: 2026/09/21 13:51:21 INFO Completed upload id=506063lb # [ 158.255295] niks3-server[1130]: 2026/09/21 13:51:21 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6064worker1 # [ 158.253055] nix-grpc-daemon[1635]: time=2026-09-21T13:51:21.897Z level=INFO msg="Upload complete. (331ms)"6065worker1 # [ 158.265311] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16066worker1 # [ 158.283229] nix-daemon[860]: reaped child process 1799, status = succeeded6067client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.98 seconds)6068client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6069client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6070worker2: must succeed: pkill -f 'read -t [9]0 x'6071worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6072client: waiting for success: ! systemctl is-active sw6073worker2 # [ 158.782739] nix-daemon[848]: reaped child process 1617, status = succeeded6074worker2 # [ 158.955446] nix-daemon[848]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6075worker2 # [ 158.972234] nix-daemon[848]: reaped child process 1798, status = succeeded6076lb # [ 158.930093] niks3-server[1130]: 2026/09/21 13:51:22 INFO Received uploads request method=POST path=/api/pending_closures6077worker2 # [ 158.993966] nix-grpc-daemon[1545]: time=2026-09-21T13:51:22.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6078worker2 # [ 158.995639] nix-grpc-daemon[1545]: time=2026-09-21T13:51:22.637Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6079lb # [ 158.964601] niks3-server[1130]: 2026/09/21 13:51:22 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6080lb # [ 158.978169] niks3-server[1130]: 2026/09/21 13:51:22 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6081lb # [ 158.986410] niks3-server[1130]: 2026/09/21 13:51:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6082lb # [ 158.992487] niks3-server[1130]: 2026/09/21 13:51:22 INFO Signed narinfos id=51 count=16083worker2 # [ 159.047358] nix-grpc-daemon[1545]: time=2026-09-21T13:51:22.690Z level=INFO msg="Uploading 1 narinfos"6084lb # [ 158.998923] niks3-server[1130]: 2026/09/21 13:51:22 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6085lb # [ 159.007375] niks3-server[1130]: 2026/09/21 13:51:22 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6086lb # [ 159.015583] niks3-server[1130]: 2026/09/21 13:51:22 INFO Completed upload id=516087worker2 # [ 159.069367] nix-grpc-daemon[1545]: time=2026-09-21T13:51:22.713Z level=INFO msg="Upload complete. (311ms)"6088lb # [ 159.018510] niks3-server[1130]: 2026/09/21 13:51:22 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6089worker2 # [ 159.085064] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44910 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16090worker2 # [ 159.100529] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:22Z level=info event=signal_exit6091worker2 # [ 159.104632] nix-daemon[848]: reaped child process 1615, status = succeeded6092client # [ 159.117010] systemd[1]: sw.service: Deactivated successfully.6093client # [ 159.121305] systemd[1]: sw.service: Consumed 178ms CPU time over 15.317s wall clock time, 39.2M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.6094lb # [ 159.265382] niks3-server[1130]: 2026/09/21 13:51:22 INFO lead: released remote=192.168.1.4:583566095client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)6096client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6097client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6098worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6099worker2 # [ 159.930583] systemd[1]: Starting scheduler-token.service...6100worker2 # [ 160.019947] systemd[1]: scheduler-token.service: Deactivated successfully.6101worker2 # [ 160.023808] systemd[1]: Finished scheduler-token.service.6102worker2 # [ 160.025424] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6103worker2 # [ 160.318889] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=scheduler_restarting6104worker1 # [ 160.267792] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:23Z level=info event=scheduler_restarting addr=lb:500516105worker2 # [ 160.329832] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448986106worker2 # [ 160.334806] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=worker_gone addr=192.168.1.3:500516107worker2 # [ 160.338986] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=scheduler_restarting addr=lb:500516108worker2 # [ 160.343720] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448986109lb # [ 160.292699] envoy[912]: conn peer=[2001:db8:1::1]:42112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9681 tx=4071 ms=66806110worker2 # [ 160.348652] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:23Z level=info event=worker_gone addr=192.168.1.4:500516111lb # [ 160.292965] envoy[912]: conn peer=[2001:db8:1::1]:48624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1470 ms=150326112worker2 # [ 160.370826] nix-daemon[848]: reaped child process 1516, status = succeeded6113worker1 # [ 160.467806] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_connected addr=lb:500516114worker1 # [ 160.473201] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6115worker2 # [ 160.547247] nix-grpc-daemon[1462]: ts=2026-09-21T13:51:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6116worker1 # [ 160.673808] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_connected addr=lb:500516117worker1 # [ 160.678552] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6118worker1 # [ 160.879273] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_connected addr=lb:500516119worker1 # [ 160.884525] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6120worker1 # [ 161.085166] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_connected addr=lb:500516121worker1 # [ 161.089909] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6122worker2 # [ 161.153239] nix-grpc-daemon[1462]: E0000 00:00:1789998684.797209 1507 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6123worker2 # [ 161.153548] nix-grpc-daemon[1462]: E0000 00:00:1789998684.797612 1507 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6124worker1 # [ 161.292494] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_connected addr=lb:500516125lb # [ 161.291771] niks3-server[1130]: 2026/09/21 13:51:24 INFO lead: acquired remote=192.168.1.3:605606126worker1 # [ 161.297532] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_take_over6127worker1 # [ 161.301572] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6128worker1 # [ 161.498313] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_connected addr=lb:500516129worker1 # [ 161.503533] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6130worker1 # [ 161.704099] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_connected addr=lb:500516131worker1 # [ 161.709144] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6132worker1 # [ 161.909977] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_connected addr=lb:500516133worker1 # [ 161.915053] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6134worker2 # [ 162.136295] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6135worker2 # [ 162.140429] systemd[1]: nix-grpc-daemon.service: Consumed 822ms CPU time over 30.414s wall clock time, 35M memory peak, 25.4K incoming IP traffic, 32.2K outgoing IP traffic.6136worker2 # [ 162.161696] systemd[1]: Starting Nix worker-protocol over gRPC...6137worker1 # [ 162.115652] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=scheduler_connected addr=lb:500516138worker1 # [ 162.133978] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:417346139worker1 # [ 162.139309] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06140worker2 # [ 162.357417] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6141worker2 # [ 162.360047] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6142worker2 # [ 162.364277] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16143worker2 # [ 162.371417] nix-grpc-daemon[1815]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6144worker2 # [ 162.379441] nix-grpc-daemon[1815]: E0000 00:00:1789998686.014989 1815 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6145worker2 # [ 162.384112] nix-grpc-daemon[1815]: E0000 00:00:1789998686.018584 1815 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6146worker2 # [ 162.387436] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6147worker2 # [ 162.391370] systemd[1]: Started Nix worker-protocol over gRPC.6148worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.55 seconds)6149lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6150lb: must succeed: curl -sf localhost:9901/clusters6151worker2 # [ 162.400115] nix-grpc-daemon[1815]: E0000 00:00:1789998686.038684 1855 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6152worker2 # [ 162.424044] nix-grpc-daemon[1815]: E0000 00:00:1789998686.056341 1855 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6153worker2 # [ 162.426334] nix-grpc-daemon[1815]: E0000 00:00:1789998686.056487 1855 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6154worker2 # [ 162.432148] nix-grpc-daemon[1815]: E0000 00:00:1789998686.075620 1855 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6155worker2 # [ 162.448248] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:26Z level=info event=scheduler_connected addr=lb:500516156worker1 # [ 162.405208] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:26Z level=info event=oidc_jwks provider=mock keys=16157worker1 # [ 162.410606] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:417346158worker1 # [ 162.415648] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6160lb: must succeed: curl -sf localhost:9901/clusters6161lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6162lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.16 seconds)6163lb: must succeed: curl -sf localhost:9901/clusters6164lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6165worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6166worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6167(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.96 seconds)6168subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6169client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16170worker2 # [ 163.963619] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:513206171worker2 # [ 163.979098] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6172worker2 # [ 163.999572] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513206173worker2 # [ 164.014730] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)6174worker2 # [ 164.028778] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)6175worker2 # [ 164.077761] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 3/5)6176worker2 # [ 164.239328] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 458 ms (attempt 4/5)6177worker2 # [ 164.705056] nix-grpc-daemon[1815]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6178worker2 # [ 164.711371] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6179worker1 # [ 164.707931] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06180worker1 # [ 164.713234] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16181worker1 # [ 164.719011] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06182worker1 # [ 164.724351] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06183worker1 # [ 164.754201] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266184worker1 # [ 164.768848] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266185worker1 # [ 164.782164] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266186worker1 # [ 164.797115] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6187worker1 # [ 164.995280] nix-daemon[860]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6188worker1 # [ 165.013889] nix-daemon[860]: reaped child process 1832, status = succeeded6189worker1 # [ 165.184625] nix-daemon[860]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6190worker1 # [ 165.200804] nix-daemon[860]: reaped child process 1837, status = succeeded6191lb # [ 165.214642] niks3-server[1130]: 2026/09/21 13:51:28 INFO Received uploads request method=POST path=/api/pending_closures6192worker1 # [ 165.228003] nix-grpc-daemon[1635]: time=2026-09-21T13:51:28.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6193worker1 # [ 165.229618] nix-grpc-daemon[1635]: time=2026-09-21T13:51:28.871Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6194lb # [ 165.251358] niks3-server[1130]: 2026/09/21 13:51:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6195worker1 # [ 165.262439] nix-grpc-daemon[1635]: time=2026-09-21T13:51:28.906Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 165.268695] niks3-server[1130]: 2026/09/21 13:51:28 INFO Signed narinfos id=52 count=16197lb # [ 165.283308] niks3-server[1130]: 2026/09/21 13:51:28 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6198worker1 # [ 165.300527] nix-grpc-daemon[1635]: time=2026-09-21T13:51:28.944Z level=INFO msg="Upload complete. (486ms)"6199lb # [ 165.300360] niks3-server[1130]: 2026/09/21 13:51:28 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6200lb # [ 165.306364] niks3-server[1130]: 2026/09/21 13:51:28 INFO Completed upload id=526201lb # [ 165.308932] niks3-server[1130]: 2026/09/21 13:51:28 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6202worker1 # [ 165.308334] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=5126203worker1 # [ 165.320268] nix-grpc-daemon[1635]: time=2026-09-21T13:51:28.950Z level=WARN msg="Failed to register uploaded object" key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls error="server returned 404: object is not pending upload\n"6204worker1 # [ 165.327463] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266205worker1 # [ 165.334607] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6206worker1 # [ 165.336808] nix-daemon[860]: reaped child process 1827, status = succeeded6207worker1 # [ 165.350471] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6208worker1 # [ 166.596150] nix-daemon[860]: reaped child process 1841, status = succeeded6209worker1 # [ 166.767184] nix-daemon[860]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6210worker1 # [ 166.782305] nix-daemon[860]: reaped child process 1860, status = succeeded6211lb # [ 166.798443] niks3-server[1130]: 2026/09/21 13:51:30 INFO Received uploads request method=POST path=/api/pending_closures6212worker1 # [ 166.808442] nix-grpc-daemon[1635]: time=2026-09-21T13:51:30.452Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6213worker1 # [ 166.810081] nix-grpc-daemon[1635]: time=2026-09-21T13:51:30.452Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6214lb # [ 166.829230] niks3-server[1130]: 2026/09/21 13:51:30 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6215worker1 # [ 166.860479] nix-grpc-daemon[1635]: time=2026-09-21T13:51:30.504Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 166.851786] niks3-server[1130]: 2026/09/21 13:51:30 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6217lb # [ 166.863344] niks3-server[1130]: 2026/09/21 13:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6218lb # [ 166.880222] niks3-server[1130]: 2026/09/21 13:51:30 INFO Signed narinfos id=53 count=16219worker1 # [ 166.879549] nix-grpc-daemon[1635]: time=2026-09-21T13:51:30.523Z level=INFO msg="Upload complete. (319ms)"6220lb # [ 166.882647] niks3-server[1130]: 2026/09/21 13:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6221lb # [ 166.890463] niks3-server[1130]: 2026/09/21 13:51:30 INFO Completed upload id=536222lb # [ 166.898745] niks3-server[1130]: 2026/09/21 13:51:30 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6223worker1 # [ 166.898923] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16224lb # [ 166.902118] niks3-server[1130]: 2026/09/21 13:51:30 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6225worker1 # [ 166.913778] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16226worker1 # [ 166.918138] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06227worker1 # [ 166.921532] nix-daemon[860]: reaped child process 1839, status = succeeded6228worker1 # [ 166.941632] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266229worker1 # [ 166.955777] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266230worker1 # [ 166.969564] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266231worker1 # [ 166.982331] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6232worker1 # [ 167.179693] nix-daemon[860]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6233worker1 # [ 167.196241] nix-daemon[860]: reaped child process 1869, status = succeeded6234worker1 # [ 167.368431] nix-daemon[860]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6235worker1 # [ 167.385372] nix-daemon[860]: reaped child process 1874, status = succeeded6236lb # [ 167.396282] niks3-server[1130]: 2026/09/21 13:51:31 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 167.409431] nix-grpc-daemon[1635]: time=2026-09-21T13:51:31.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6238worker1 # [ 167.410911] nix-grpc-daemon[1635]: time=2026-09-21T13:51:31.053Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6239worker1 # [ 167.435578] nix-grpc-daemon[1635]: time=2026-09-21T13:51:31.079Z level=INFO msg="Uploading 1 narinfos"6240lb # [ 167.434652] niks3-server[1130]: 2026/09/21 13:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6241lb # [ 167.445991] niks3-server[1130]: 2026/09/21 13:51:31 INFO Signed narinfos id=54 count=16242lb # [ 167.452984] niks3-server[1130]: 2026/09/21 13:51:31 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6243lb # [ 167.459667] niks3-server[1130]: 2026/09/21 13:51:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6244lb # [ 167.470907] niks3-server[1130]: 2026/09/21 13:51:31 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6245lb # [ 167.476664] niks3-server[1130]: 2026/09/21 13:51:31 INFO Completed upload id=546246worker1 # [ 167.477469] nix-grpc-daemon[1635]: time=2026-09-21T13:51:31.121Z level=INFO msg="Upload complete. (481ms)"6247lb # [ 167.479672] niks3-server[1130]: 2026/09/21 13:51:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6248worker1 # [ 167.484341] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=5126249worker1 # [ 167.499347] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266250worker1 # [ 167.506307] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6251worker1 # [ 167.511155] nix-daemon[860]: reaped child process 1864, status = succeeded6252worker1 # [ 167.524268] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6253worker1 # [ 168.948362] nix-daemon[860]: reaped child process 1878, status = succeeded6254worker1 # [ 169.107135] nix-daemon[860]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6255worker1 # [ 169.126323] nix-daemon[860]: reaped child process 1897, status = succeeded6256lb # [ 169.138216] niks3-server[1130]: 2026/09/21 13:51:32 INFO Received uploads request method=POST path=/api/pending_closures6257worker1 # [ 169.152381] nix-grpc-daemon[1635]: time=2026-09-21T13:51:32.793Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6258worker1 # [ 169.154012] nix-grpc-daemon[1635]: time=2026-09-21T13:51:32.793Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6259lb # [ 169.179761] niks3-server[1130]: 2026/09/21 13:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6260worker1 # [ 169.190987] nix-grpc-daemon[1635]: time=2026-09-21T13:51:32.834Z level=INFO msg="Uploading 1 narinfos"6261lb # [ 169.195323] niks3-server[1130]: 2026/09/21 13:51:32 INFO Signed narinfos id=55 count=16262lb # [ 169.204458] niks3-server[1130]: 2026/09/21 13:51:32 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6263lb # [ 169.220423] niks3-server[1130]: 2026/09/21 13:51:32 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6264lb # [ 169.225583] niks3-server[1130]: 2026/09/21 13:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6265lb # [ 169.229802] niks3-server[1130]: 2026/09/21 13:51:32 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6266lb # [ 169.234389] niks3-server[1130]: 2026/09/21 13:51:32 INFO Completed upload id=556267worker1 # [ 169.235062] nix-grpc-daemon[1635]: time=2026-09-21T13:51:32.879Z level=INFO msg="Upload complete. (300ms)"6268lb # [ 169.237259] niks3-server[1130]: 2026/09/21 13:51:32 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6269worker1 # [ 169.249051] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16270worker1 # [ 169.264285] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16271worker1 # [ 169.274150] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266272worker1 # [ 169.278925] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06273worker1 # [ 169.282488] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266274worker1 # [ 169.287542] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266275worker1 # [ 169.300252] nix-daemon[860]: reaped child process 1876, status = succeeded6276worker1 # [ 169.302013] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6277worker1 # [ 169.483706] nix-daemon[860]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6278worker1 # [ 169.500944] nix-daemon[860]: reaped child process 1906, status = succeeded6279worker1 # [ 169.672322] nix-daemon[860]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6280worker1 # [ 169.690533] nix-daemon[860]: reaped child process 1911, status = succeeded6281lb # [ 169.702669] niks3-server[1130]: 2026/09/21 13:51:33 INFO Received uploads request method=POST path=/api/pending_closures6282worker1 # [ 169.712686] nix-grpc-daemon[1635]: time=2026-09-21T13:51:33.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6283worker1 # [ 169.714298] nix-grpc-daemon[1635]: time=2026-09-21T13:51:33.356Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6284lb # [ 169.733525] niks3-server[1130]: 2026/09/21 13:51:33 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6285worker1 # [ 169.742723] nix-grpc-daemon[1635]: time=2026-09-21T13:51:33.386Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 169.748263] niks3-server[1130]: 2026/09/21 13:51:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6287worker1 # [ 169.763538] nix-grpc-daemon[1635]: time=2026-09-21T13:51:33.407Z level=INFO msg="Upload complete. (455ms)"6288lb # [ 169.758092] niks3-server[1130]: 2026/09/21 13:51:33 INFO Signed narinfos id=56 count=16289lb # [ 169.766037] niks3-server[1130]: 2026/09/21 13:51:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6290lb # [ 169.770992] niks3-server[1130]: 2026/09/21 13:51:33 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6291worker1 # [ 169.769312] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=5126292lb # [ 169.775902] niks3-server[1130]: 2026/09/21 13:51:33 INFO Completed upload id=566293worker1 # [ 169.777937] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266294lb # [ 169.778604] niks3-server[1130]: 2026/09/21 13:51:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6295worker1 # [ 169.787152] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6296worker1 # [ 169.791245] nix-daemon[860]: reaped child process 1901, status = succeeded6297worker1 # [ 169.801340] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6298lb # [ 170.297450] envoy[912]: conn peer=[2001:db8:1::4]:55118 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1520 tx=344 ms=283876299worker1 # [ 171.245670] nix-daemon[860]: reaped child process 1915, status = succeeded6300worker1 # [ 171.428544] nix-daemon[860]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6301worker1 # [ 171.444594] nix-daemon[860]: reaped child process 1934, status = succeeded6302lb # [ 171.459001] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received uploads request method=POST path=/api/pending_closures6303worker1 # [ 171.471395] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6304worker1 # [ 171.473456] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.114Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6305lb # [ 171.500958] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6306worker1 # [ 171.514704] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.158Z level=INFO msg="Uploading 1 narinfos"6307lb # [ 171.513808] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6308lb # [ 171.524241] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6309lb # [ 171.527827] niks3-server[1130]: 2026/09/21 13:51:35 INFO Signed narinfos id=57 count=16310lb # [ 171.533544] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6311lb # [ 171.542166] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6312worker1 # [ 171.547325] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.191Z level=INFO msg="Upload complete. (334ms)"6313lb # [ 171.547153] niks3-server[1130]: 2026/09/21 13:51:35 INFO Completed upload id=576314lb # [ 171.550773] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6315worker1 # [ 171.564952] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16316worker1 # [ 171.580795] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16317worker1 # [ 171.587234] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266318worker1 # [ 171.591241] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06319worker1 # [ 171.595708] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266320worker1 # [ 171.599089] nix-daemon[860]: reaped child process 1913, status = succeeded6321worker1 # [ 171.602003] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.245Z level=INFO msg="All 1 paths already cached"6322worker1 # [ 171.605859] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266323worker1 # [ 171.616814] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6324worker1 # [ 171.805516] nix-daemon[860]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6325worker1 # [ 171.823995] nix-daemon[860]: reaped child process 1943, status = succeeded6326worker1 # [ 171.995293] nix-daemon[860]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6327worker1 # [ 172.011893] nix-daemon[860]: reaped child process 1948, status = succeeded6328lb # [ 172.025445] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received uploads request method=POST path=/api/pending_closures6329worker1 # [ 172.035705] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.679Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6330worker1 # [ 172.037307] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.679Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6331lb # [ 172.051983] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6332worker1 # [ 172.066499] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.710Z level=INFO msg="Uploading 1 narinfos"6333lb # [ 172.066352] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6334lb # [ 172.077476] niks3-server[1130]: 2026/09/21 13:51:35 INFO Signed narinfos id=58 count=16335lb # [ 172.085337] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6336lb # [ 172.096787] niks3-server[1130]: 2026/09/21 13:51:35 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6337lb # [ 172.101846] niks3-server[1130]: 2026/09/21 13:51:35 INFO Completed upload id=586338lb # [ 172.106177] niks3-server[1130]: 2026/09/21 13:51:35 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6339worker1 # [ 172.101495] nix-grpc-daemon[1635]: time=2026-09-21T13:51:35.745Z level=INFO msg="Upload complete. (473ms)"6340worker1 # [ 172.111670] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=6246341worker1 # [ 172.122569] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266342worker1 # [ 172.130216] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6343worker1 # [ 172.134790] nix-daemon[860]: reaped child process 1938, status = succeeded6344worker1 # [ 172.147360] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6345worker1 # [ 172.991450] nix-daemon[860]: reaped child process 1952, status = succeeded6346worker1 # [ 173.171983] nix-daemon[860]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6347worker1 # [ 173.188268] nix-daemon[860]: reaped child process 1971, status = succeeded6348lb # [ 173.202988] niks3-server[1130]: 2026/09/21 13:51:36 INFO Received uploads request method=POST path=/api/pending_closures6349worker1 # [ 173.216402] nix-grpc-daemon[1635]: time=2026-09-21T13:51:36.857Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6350worker1 # [ 173.217873] nix-grpc-daemon[1635]: time=2026-09-21T13:51:36.857Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6351lb # [ 173.238915] niks3-server[1130]: 2026/09/21 13:51:36 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6352lb # [ 173.255358] niks3-server[1130]: 2026/09/21 13:51:36 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6353lb # [ 173.264383] niks3-server[1130]: 2026/09/21 13:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6354worker1 # [ 173.270342] nix-grpc-daemon[1635]: time=2026-09-21T13:51:36.914Z level=INFO msg="Uploading 1 narinfos"6355lb # [ 173.273259] niks3-server[1130]: 2026/09/21 13:51:36 INFO Signed narinfos id=59 count=16356lb # [ 173.275799] niks3-server[1130]: 2026/09/21 13:51:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6357worker1 # [ 173.289688] nix-grpc-daemon[1635]: time=2026-09-21T13:51:36.933Z level=INFO msg="Upload complete. (333ms)"6358lb # [ 173.290393] niks3-server[1130]: 2026/09/21 13:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6359lb # [ 173.296582] niks3-server[1130]: 2026/09/21 13:51:36 INFO Completed upload id=596360lb # [ 173.299771] niks3-server[1130]: 2026/09/21 13:51:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6361worker1 # [ 173.299020] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16362worker1 # [ 173.311813] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16363worker1 # [ 173.315569] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266364worker1 # [ 173.319492] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06365worker1 # [ 173.323266] nix-daemon[860]: reaped child process 1950, status = succeeded6366worker1 # [ 173.326609] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266367worker1 # [ 173.336183] nix-grpc-daemon[1635]: time=2026-09-21T13:51:36.980Z level=INFO msg="All 1 paths already cached"6368worker1 # [ 173.342272] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266369worker1 # [ 173.349738] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6370worker1 # [ 173.526903] nix-daemon[860]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6371worker1 # [ 173.544168] nix-daemon[860]: reaped child process 1980, status = succeeded6372worker1 # [ 173.717757] nix-daemon[860]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6373worker1 # [ 173.734266] nix-daemon[860]: reaped child process 1985, status = succeeded6374lb # [ 173.745019] niks3-server[1130]: 2026/09/21 13:51:37 INFO Received uploads request method=POST path=/api/pending_closures6375worker1 # [ 173.754542] nix-grpc-daemon[1635]: time=2026-09-21T13:51:37.398Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6376worker1 # [ 173.756010] nix-grpc-daemon[1635]: time=2026-09-21T13:51:37.398Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6377lb # [ 173.772676] niks3-server[1130]: 2026/09/21 13:51:37 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6378lb # [ 173.783858] niks3-server[1130]: 2026/09/21 13:51:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6379worker1 # [ 173.790537] nix-grpc-daemon[1635]: time=2026-09-21T13:51:37.434Z level=INFO msg="Uploading 1 narinfos"6380lb # [ 173.794372] niks3-server[1130]: 2026/09/21 13:51:37 INFO Signed narinfos id=60 count=16381lb # [ 173.800973] niks3-server[1130]: 2026/09/21 13:51:37 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6382worker1 # [ 173.809701] nix-grpc-daemon[1635]: time=2026-09-21T13:51:37.453Z level=INFO msg="Upload complete. (451ms)"6383lb # [ 173.809700] niks3-server[1130]: 2026/09/21 13:51:37 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6384lb # [ 173.813951] niks3-server[1130]: 2026/09/21 13:51:37 INFO Completed upload id=606385lb # [ 173.815941] niks3-server[1130]: 2026/09/21 13:51:37 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6386worker1 # [ 173.816145] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=6246387worker1 # [ 173.823732] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266388worker1 # [ 173.834033] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6389worker1 # [ 173.837915] nix-daemon[860]: reaped child process 1975, status = succeeded6390worker1 # [ 173.848183] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6391worker1 # [ 174.869761] nix-daemon[860]: reaped child process 1989, status = succeeded6392worker1 # [ 175.027374] nix-daemon[860]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)6393worker1 # [ 175.045730] nix-daemon[860]: reaped child process 2008, status = succeeded6394lb # [ 175.057489] niks3-server[1130]: 2026/09/21 13:51:38 INFO Received uploads request method=POST path=/api/pending_closures6395worker1 # [ 175.068265] nix-grpc-daemon[1635]: time=2026-09-21T13:51:38.711Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6396worker1 # [ 175.069739] nix-grpc-daemon[1635]: time=2026-09-21T13:51:38.711Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6397lb # [ 175.092491] niks3-server[1130]: 2026/09/21 13:51:38 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6398lb # [ 175.107447] niks3-server[1130]: 2026/09/21 13:51:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6399worker1 # [ 175.113598] nix-grpc-daemon[1635]: time=2026-09-21T13:51:38.757Z level=INFO msg="Uploading 1 narinfos"6400lb # [ 175.118223] niks3-server[1130]: 2026/09/21 13:51:38 INFO Signed narinfos id=61 count=16401lb # [ 175.124224] niks3-server[1130]: 2026/09/21 13:51:38 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6402lb # [ 175.130119] niks3-server[1130]: 2026/09/21 13:51:38 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6403worker1 # [ 175.138262] nix-grpc-daemon[1635]: time=2026-09-21T13:51:38.782Z level=INFO msg="Upload complete. (280ms)"6404lb # [ 175.137223] niks3-server[1130]: 2026/09/21 13:51:38 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6405lb # [ 175.141137] niks3-server[1130]: 2026/09/21 13:51:38 INFO Completed upload id=616406lb # [ 175.143787] niks3-server[1130]: 2026/09/21 13:51:38 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6407worker1 # [ 175.152129] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16408worker1 # [ 175.164726] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16409worker1 # [ 175.171167] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266410worker1 # [ 175.176437] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06411worker1 # [ 175.181809] nix-daemon[860]: reaped child process 1987, status = succeeded6412worker1 # [ 175.183583] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266413worker1 # [ 175.191079] nix-grpc-daemon[1635]: time=2026-09-21T13:51:38.835Z level=INFO msg="All 1 paths already cached"6414worker1 # [ 175.197993] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266415worker1 # [ 175.208543] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6416worker1 # [ 175.391637] nix-daemon[860]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6417worker1 # [ 175.407185] nix-daemon[860]: reaped child process 2017, status = succeeded6418worker1 # [ 175.583329] nix-daemon[860]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6419worker1 # [ 175.598036] nix-daemon[860]: reaped child process 2022, status = succeeded6420lb # [ 175.613135] niks3-server[1130]: 2026/09/21 13:51:39 INFO Received uploads request method=POST path=/api/pending_closures6421worker1 # [ 175.622713] nix-grpc-daemon[1635]: time=2026-09-21T13:51:39.266Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6422worker1 # [ 175.624254] nix-grpc-daemon[1635]: time=2026-09-21T13:51:39.266Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6423lb # [ 175.642383] niks3-server[1130]: 2026/09/21 13:51:39 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6424worker1 # [ 175.652712] nix-grpc-daemon[1635]: time=2026-09-21T13:51:39.296Z level=INFO msg="Uploading 1 narinfos"6425lb # [ 175.656479] niks3-server[1130]: 2026/09/21 13:51:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6426lb # [ 175.670691] niks3-server[1130]: 2026/09/21 13:51:39 INFO Signed narinfos id=62 count=16427worker1 # [ 175.675548] nix-grpc-daemon[1635]: time=2026-09-21T13:51:39.319Z level=INFO msg="Upload complete. (457ms)"6428lb # [ 175.675540] niks3-server[1130]: 2026/09/21 13:51:39 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6429lb # [ 175.680622] niks3-server[1130]: 2026/09/21 13:51:39 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6430worker1 # [ 175.684152] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=6246431lb # [ 175.685204] niks3-server[1130]: 2026/09/21 13:51:39 INFO Completed upload id=626432lb # [ 175.687530] niks3-server[1130]: 2026/09/21 13:51:39 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6433worker1 # [ 175.692993] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266434worker1 # [ 175.699318] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6435worker1 # [ 175.704169] nix-daemon[860]: reaped child process 2012, status = succeeded6436worker1 # [ 175.716229] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6437worker1 # [ 176.959294] nix-daemon[860]: reaped child process 2026, status = succeeded6438worker1 # [ 177.132657] nix-daemon[860]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6439worker1 # [ 177.149854] nix-daemon[860]: reaped child process 2045, status = succeeded6440lb # [ 177.166059] niks3-server[1130]: 2026/09/21 13:51:40 INFO Received uploads request method=POST path=/api/pending_closures6441worker1 # [ 177.176637] nix-grpc-daemon[1635]: time=2026-09-21T13:51:40.818Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6442worker1 # [ 177.178830] nix-grpc-daemon[1635]: time=2026-09-21T13:51:40.818Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6443lb # [ 177.201168] niks3-server[1130]: 2026/09/21 13:51:40 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6444lb # [ 177.216950] niks3-server[1130]: 2026/09/21 13:51:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6445worker1 # [ 177.223437] nix-grpc-daemon[1635]: time=2026-09-21T13:51:40.867Z level=INFO msg="Uploading 1 narinfos"6446lb # [ 177.224255] niks3-server[1130]: 2026/09/21 13:51:40 INFO Signed narinfos id=63 count=16447lb # [ 177.232889] niks3-server[1130]: 2026/09/21 13:51:40 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6448lb # [ 177.240410] niks3-server[1130]: 2026/09/21 13:51:40 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6449worker1 # [ 177.247984] nix-grpc-daemon[1635]: time=2026-09-21T13:51:40.892Z level=INFO msg="Upload complete. (322ms)"6450lb # [ 177.248824] niks3-server[1130]: 2026/09/21 13:51:40 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6451lb # [ 177.254014] niks3-server[1130]: 2026/09/21 13:51:40 INFO Completed upload id=636452lb # [ 177.257086] niks3-server[1130]: 2026/09/21 13:51:40 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6453worker1 # [ 177.260989] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16454worker1 # [ 177.285219] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16455worker1 # [ 177.294189] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266456worker1 # [ 177.298614] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06457worker1 # [ 177.302865] nix-daemon[860]: reaped child process 2024, status = succeeded6458worker1 # [ 177.316966] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266459worker1 # [ 177.328904] nix-grpc-daemon[1635]: time=2026-09-21T13:51:40.973Z level=INFO msg="All 2 paths already cached"6460worker1 # [ 177.336890] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266461worker1 # [ 177.350810] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6462worker1 # [ 177.543096] nix-daemon[860]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6463worker1 # [ 177.559988] nix-daemon[860]: reaped child process 2054, status = succeeded6464worker1 # [ 177.731312] nix-daemon[860]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6465worker1 # [ 177.747480] nix-daemon[860]: reaped child process 2059, status = succeeded6466lb # [ 177.758186] niks3-server[1130]: 2026/09/21 13:51:41 INFO Received uploads request method=POST path=/api/pending_closures6467worker1 # [ 177.767313] nix-grpc-daemon[1635]: time=2026-09-21T13:51:41.411Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6468worker1 # [ 177.768918] nix-grpc-daemon[1635]: time=2026-09-21T13:51:41.411Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6469lb # [ 177.785682] niks3-server[1130]: 2026/09/21 13:51:41 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6470worker1 # [ 177.796888] nix-grpc-daemon[1635]: time=2026-09-21T13:51:41.440Z level=INFO msg="Uploading 1 narinfos"6471lb # [ 177.800920] niks3-server[1130]: 2026/09/21 13:51:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6472lb # [ 177.812298] niks3-server[1130]: 2026/09/21 13:51:41 INFO Signed narinfos id=64 count=16473lb # [ 177.816366] niks3-server[1130]: 2026/09/21 13:51:41 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6474lb # [ 177.824121] niks3-server[1130]: 2026/09/21 13:51:41 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6475lb # [ 177.828956] niks3-server[1130]: 2026/09/21 13:51:41 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6476lb # [ 177.832780] niks3-server[1130]: 2026/09/21 13:51:41 INFO Completed upload id=646477worker1 # [ 177.833324] nix-grpc-daemon[1635]: time=2026-09-21T13:51:41.477Z level=INFO msg="Upload complete. (470ms)"6478worker1 # [ 177.840945] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=7526479worker1 # [ 177.853909] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266480worker1 # [ 177.859655] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6481worker1 # [ 177.864634] nix-daemon[860]: reaped child process 2049, status = succeeded6482worker1 # [ 177.877717] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6483worker1 # [ 179.101124] nix-daemon[860]: reaped child process 2063, status = succeeded6484worker1 # [ 179.276117] nix-daemon[860]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)6485worker1 # [ 179.292971] nix-daemon[860]: reaped child process 2082, status = succeeded6486lb # [ 179.305824] niks3-server[1130]: 2026/09/21 13:51:42 INFO Received uploads request method=POST path=/api/pending_closures6487worker1 # [ 179.316365] nix-grpc-daemon[1635]: time=2026-09-21T13:51:42.960Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6488worker1 # [ 179.317852] nix-grpc-daemon[1635]: time=2026-09-21T13:51:42.960Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6489lb # [ 179.340933] niks3-server[1130]: 2026/09/21 13:51:42 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6490lb # [ 179.352372] niks3-server[1130]: 2026/09/21 13:51:42 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6491worker1 # [ 179.365990] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.009Z level=INFO msg="Uploading 1 narinfos"6492lb # [ 179.368524] niks3-server[1130]: 2026/09/21 13:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6493lb # [ 179.376218] niks3-server[1130]: 2026/09/21 13:51:42 INFO Signed narinfos id=65 count=16494lb # [ 179.378446] niks3-server[1130]: 2026/09/21 13:51:43 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6495lb # [ 179.384598] niks3-server[1130]: 2026/09/21 13:51:43 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6496lb # [ 179.390247] niks3-server[1130]: 2026/09/21 13:51:43 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6497lb # [ 179.393937] niks3-server[1130]: 2026/09/21 13:51:43 INFO Completed upload id=656498worker1 # [ 179.394458] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.038Z level=INFO msg="Upload complete. (307ms)"6499worker1 # [ 179.409029] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16500worker1 # [ 179.420281] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16501worker1 # [ 179.425073] nix-daemon[860]: reaped child process 2061, status = succeeded6502worker1 # [ 179.448760] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266503worker1 # [ 179.462681] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266504worker1 # [ 179.476393] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.120Z level=INFO msg="All 2 paths already cached"6505worker1 # [ 179.485027] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266506worker1 # [ 179.498454] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6507worker1 # [ 179.692849] nix-daemon[860]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6508worker1 # [ 179.709129] nix-daemon[860]: reaped child process 2089, status = succeeded6509worker1 # [ 179.880120] nix-daemon[860]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)6510worker1 # [ 179.896968] nix-daemon[860]: reaped child process 2094, status = succeeded6511lb # [ 179.909534] niks3-server[1130]: 2026/09/21 13:51:43 INFO Received uploads request method=POST path=/api/pending_closures6512worker1 # [ 179.918458] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.562Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6513worker1 # [ 179.920080] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.562Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6514lb # [ 179.934147] niks3-server[1130]: 2026/09/21 13:51:43 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6515worker1 # [ 179.947502] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.591Z level=INFO msg="Uploading 1 narinfos"6516lb # [ 179.951004] niks3-server[1130]: 2026/09/21 13:51:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6517lb # [ 179.961439] niks3-server[1130]: 2026/09/21 13:51:43 INFO Signed narinfos id=66 count=16518lb # [ 179.972732] niks3-server[1130]: 2026/09/21 13:51:43 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6519worker1 # [ 179.978158] nix-grpc-daemon[1635]: time=2026-09-21T13:51:43.622Z level=INFO msg="Upload complete. (464ms)"6520lb # [ 179.978251] niks3-server[1130]: 2026/09/21 13:51:43 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6521lb # [ 179.984164] niks3-server[1130]: 2026/09/21 13:51:43 INFO Completed upload id=666522worker1 # [ 179.984436] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=7526523lb # [ 179.987816] niks3-server[1130]: 2026/09/21 13:51:43 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6524worker1 # [ 179.994375] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266525worker1 # [ 179.997574] nix-daemon[860]: reaped child process 2084, status = succeeded6526worker1 # [ 180.004119] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6527worker1 # [ 180.017329] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6528worker1 # [ 180.838002] nix-daemon[860]: reaped child process 2098, status = succeeded6529worker1 # [ 181.006563] nix-daemon[860]: accepted connection from pid 2114, user nix-grpc-daemon (trusted)6530worker1 # [ 181.026139] nix-daemon[860]: reaped child process 2117, status = succeeded6531lb # [ 181.038099] niks3-server[1130]: 2026/09/21 13:51:44 INFO Received uploads request method=POST path=/api/pending_closures6532worker1 # [ 181.049539] nix-grpc-daemon[1635]: time=2026-09-21T13:51:44.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6533worker1 # [ 181.051027] nix-grpc-daemon[1635]: time=2026-09-21T13:51:44.692Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6534lb # [ 181.074793] niks3-server[1130]: 2026/09/21 13:51:44 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6535worker1 # [ 181.091620] nix-grpc-daemon[1635]: time=2026-09-21T13:51:44.735Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 181.091443] niks3-server[1130]: 2026/09/21 13:51:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6537lb # [ 181.102682] niks3-server[1130]: 2026/09/21 13:51:44 INFO Signed narinfos id=67 count=16538lb # [ 181.108368] niks3-server[1130]: 2026/09/21 13:51:44 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6539lb # [ 181.111791] niks3-server[1130]: 2026/09/21 13:51:44 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6540lb # [ 181.120147] niks3-server[1130]: 2026/09/21 13:51:44 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6541lb # [ 181.124081] niks3-server[1130]: 2026/09/21 13:51:44 INFO Completed upload id=676542worker1 # [ 181.124855] nix-grpc-daemon[1635]: time=2026-09-21T13:51:44.769Z level=INFO msg="Upload complete. (292ms)"6543lb # [ 181.126244] niks3-server[1130]: 2026/09/21 13:51:44 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6544worker1 # [ 181.137110] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16545worker1 # [ 181.151453] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:44Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16546worker1 # [ 181.155689] nix-daemon[860]: reaped child process 2096, status = succeeded6547worker1 # [ 181.176734] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266548worker1 # [ 181.190901] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266549worker1 # [ 181.205206] nix-grpc-daemon[1635]: time=2026-09-21T13:51:44.849Z level=INFO msg="All 2 paths already cached"6550worker1 # [ 181.214433] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440266551worker1 # [ 181.227808] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6552worker1 # [ 181.426946] nix-daemon[860]: accepted connection from pid 2121, user nix-grpc-daemon (trusted)6553worker1 # [ 181.444874] nix-daemon[860]: reaped child process 2124, status = succeeded6554worker1 # [ 181.620457] nix-daemon[860]: accepted connection from pid 2126, user nix-grpc-daemon (trusted)6555worker1 # [ 181.636916] nix-daemon[860]: reaped child process 2129, status = succeeded6556lb # [ 181.650233] niks3-server[1130]: 2026/09/21 13:51:45 INFO Received uploads request method=POST path=/api/pending_closures6557worker1 # [ 181.659793] nix-grpc-daemon[1635]: time=2026-09-21T13:51:45.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6558worker1 # [ 181.661527] nix-grpc-daemon[1635]: time=2026-09-21T13:51:45.303Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6559lb # [ 181.676676] niks3-server[1130]: 2026/09/21 13:51:45 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6560worker1 # [ 181.691192] nix-grpc-daemon[1635]: time=2026-09-21T13:51:45.335Z level=INFO msg="Uploading 1 narinfos"6561lb # [ 181.693448] niks3-server[1130]: 2026/09/21 13:51:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6562lb # [ 181.703703] niks3-server[1130]: 2026/09/21 13:51:45 INFO Signed narinfos id=68 count=16563lb # [ 181.708540] niks3-server[1130]: 2026/09/21 13:51:45 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6564lb # [ 181.716875] niks3-server[1130]: 2026/09/21 13:51:45 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6565lb # [ 181.721964] niks3-server[1130]: 2026/09/21 13:51:45 INFO Completed upload id=686566worker1 # [ 181.722716] nix-grpc-daemon[1635]: time=2026-09-21T13:51:45.366Z level=INFO msg="Upload complete. (480ms)"6567lb # [ 181.724979] niks3-server[1130]: 2026/09/21 13:51:45 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6568worker1 # [ 181.732300] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=0 paths=1 nar_bytes_in=7526569worker1 # [ 181.741014] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440266570worker1 # [ 181.747293] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6571worker1 # [ 181.752203] nix-daemon[860]: reaped child process 2119, status = succeeded6572worker1 # [ 181.764198] nix-daemon[860]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6573worker1 # [ 182.793247] nix-daemon[860]: reaped child process 2133, status = succeeded6574worker1 # [ 182.955759] nix-daemon[860]: accepted connection from pid 2149, user nix-grpc-daemon (trusted)6575worker1 # [ 182.973556] nix-daemon[860]: reaped child process 2152, status = succeeded6576lb # [ 182.987389] niks3-server[1130]: 2026/09/21 13:51:46 INFO Received uploads request method=POST path=/api/pending_closures6577worker1 # [ 182.999266] nix-grpc-daemon[1635]: time=2026-09-21T13:51:46.642Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6578worker1 # [ 183.000864] nix-grpc-daemon[1635]: time=2026-09-21T13:51:46.642Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6579lb # [ 183.021471] niks3-server[1130]: 2026/09/21 13:51:46 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6580lb # [ 183.033723] niks3-server[1130]: 2026/09/21 13:51:46 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6581lb # [ 183.044113] niks3-server[1130]: 2026/09/21 13:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6582worker1 # [ 183.049616] nix-grpc-daemon[1635]: time=2026-09-21T13:51:46.693Z level=INFO msg="Uploading 1 narinfos"6583lb # [ 183.052054] niks3-server[1130]: 2026/09/21 13:51:46 INFO Signed narinfos id=69 count=16584lb # [ 183.054368] niks3-server[1130]: 2026/09/21 13:51:46 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6585lb # [ 183.064971] niks3-server[1130]: 2026/09/21 13:51:46 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6586lb # [ 183.071956] niks3-server[1130]: 2026/09/21 13:51:46 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6587lb # [ 183.076861] niks3-server[1130]: 2026/09/21 13:51:46 INFO Completed upload id=696588worker1 # [ 183.077268] nix-grpc-daemon[1635]: time=2026-09-21T13:51:46.721Z level=INFO msg="Upload complete. (304ms)"6589worker1 # [ 183.088977] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44026 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16590worker1 # [ 183.104184] nix-daemon[860]: reaped child process 2131, status = succeeded6591client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.52 seconds)6592worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6593worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6594client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6595client # [ 183.354347] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6596client # Running as unit: gpu.service; invocation ID: e5759e1f9520478ab96ee5b689e916a16597client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6598lb: must succeed: curl -sf localhost:9901/clusters6599lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6600lb: must succeed: curl -sf localhost:9901/clusters6601lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6602worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6603worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6604worker2 # [ 183.609313] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:513206605client # [ 183.611234] nix[1360]: don't know how to build these paths:6606client # [ 183.612471] nix[1360]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6607worker2 # [ 183.622412] nix-grpc-daemon[1815]: ts=2026-09-21T13:51:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513206608client # [ 183.643495] nix[1360]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6609worker1 # [ 183.595705] nix-grpc-daemon[1543]: ts=2026-09-21T13:51:47Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06610lb: must succeed: curl -sf localhost:9901/clusters6611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6612lb: must succeed: curl -sf localhost:9901/clusters6613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6614worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6615worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6616client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6617lb # [ 190.292498] envoy[912]: conn peer=[2001:db8:1::1]:49072 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27101 tx=11521 ms=191966618client # [ 213.645630] nix[1360]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6619client # [ 213.654925] nix[1360]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6620client # [ 213.698851] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6621client # [ 213.706845] systemd[1]: gpu.service: Failed with result 'exit-code'.6622client # [ 213.710083] systemd[1]: gpu.service: Consumed 174ms CPU time over 30.349s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6623client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.26 seconds)6624client: must succeed: journalctl -u gpu -o cat6625client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6626lb: must succeed: curl -sf localhost:9901/clusters6627lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6628lb: must succeed: curl -sf localhost:9901/clusters6629lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6630worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6631worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6632(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.63 seconds)6633subtest: low disk drains a worker and builds go to the other6634worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6635worker1 # [ 215.337966] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:18Z level=info event=unhealthy reason=min_free6636worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.07 seconds)6637worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6638worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)6639client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26640worker1 # [ 215.656118] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440266641client # don't know how to build these paths:6642client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6643worker1 # [ 215.665999] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440266644worker1 # [ 215.689096] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:19Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06645worker1 # [ 215.694375] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:19Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16646worker2 # [ 215.791422] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206647worker2 # [ 215.808975] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513206648client # copying 1 paths to 'lb:50051'...6649client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6650worker2 # [ 215.824786] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:513206651worker2 # [ 215.840644] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6652worker2 # [ 215.868731] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:19Z level=info event=niks3_push_started pid=18756653worker2 # [ 216.082590] nix-grpc-daemon[1875]: time=2026-09-21T13:52:19.726Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6654worker2 # [ 216.262759] nix-daemon[848]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6655worker2 # [ 216.279767] nix-daemon[848]: reaped child process 1891, status = succeeded6656worker2 # [ 216.458121] nix-daemon[848]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6657worker2 # [ 216.474692] nix-daemon[848]: reaped child process 1896, status = succeeded6658lb # [ 216.433518] niks3-server[1130]: 2026/09/21 13:52:20 INFO Received uploads request method=POST path=/api/pending_closures6659worker2 # [ 216.498185] nix-grpc-daemon[1875]: time=2026-09-21T13:52:20.141Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6660worker2 # [ 216.499883] nix-grpc-daemon[1875]: time=2026-09-21T13:52:20.141Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6661lb # [ 216.474911] niks3-server[1130]: 2026/09/21 13:52:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6662worker2 # [ 216.538966] nix-grpc-daemon[1875]: time=2026-09-21T13:52:20.182Z level=INFO msg="Uploading 1 narinfos"6663lb # [ 216.488454] niks3-server[1130]: 2026/09/21 13:52:20 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6664lb # [ 216.499659] niks3-server[1130]: 2026/09/21 13:52:20 INFO Signed narinfos id=70 count=16665lb # [ 216.507588] niks3-server[1130]: 2026/09/21 13:52:20 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6666worker2 # [ 216.565737] nix-grpc-daemon[1875]: time=2026-09-21T13:52:20.210Z level=INFO msg="Upload complete. (477ms)"6667lb # [ 216.513060] niks3-server[1130]: 2026/09/21 13:52:20 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6668lb # [ 216.518696] niks3-server[1130]: 2026/09/21 13:52:20 INFO Completed upload id=706669worker2 # [ 216.571996] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=0 paths=1 nar_bytes_in=5126670lb # [ 216.522163] niks3-server[1130]: 2026/09/21 13:52:20 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6671worker2 # [ 216.580711] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206672worker2 # [ 216.591920] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6673worker2 # [ 216.594706] nix-daemon[848]: reaped child process 1873, status = succeeded6674client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6675worker2 # [ 216.606317] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6676client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6677client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6678client # farm-a> LOG-farm-a-drain6679worker2 # [ 218.050166] nix-daemon[848]: reaped child process 1900, status = succeeded6680worker2 # [ 218.235810] nix-daemon[848]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)6681worker2 # [ 218.253728] nix-daemon[848]: reaped child process 1919, status = succeeded6682lb # [ 218.213173] niks3-server[1130]: 2026/09/21 13:52:21 INFO Received uploads request method=POST path=/api/pending_closures6683worker2 # [ 218.283072] nix-grpc-daemon[1875]: time=2026-09-21T13:52:21.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6684worker2 # [ 218.285360] nix-grpc-daemon[1875]: time=2026-09-21T13:52:21.924Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6685lb # [ 218.248104] niks3-server[1130]: 2026/09/21 13:52:21 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6686lb # [ 218.263380] niks3-server[1130]: 2026/09/21 13:52:21 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6687worker2 # [ 218.334454] nix-grpc-daemon[1875]: time=2026-09-21T13:52:21.977Z level=INFO msg="Uploading 1 narinfos"6688lb # [ 218.279572] niks3-server[1130]: 2026/09/21 13:52:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6689lb # [ 218.291984] niks3-server[1130]: 2026/09/21 13:52:21 INFO Signed narinfos id=71 count=16690lb # [ 218.297934] niks3-server[1130]: 2026/09/21 13:52:21 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6691lb # [ 218.307725] niks3-server[1130]: 2026/09/21 13:52:21 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6692lb # [ 218.312509] niks3-server[1130]: 2026/09/21 13:52:21 INFO Completed upload id=716693worker2 # [ 218.366353] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.010Z level=INFO msg="Upload complete. (350ms)"6694lb # [ 218.316087] niks3-server[1130]: 2026/09/21 13:52:21 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6695worker2 # [ 218.381151] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16696worker1 # [ 218.331239] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:21Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06697worker1 # [ 218.337315] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:21Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16698worker2 # [ 218.397557] nix-daemon[848]: reaped child process 1898, status = succeeded6699worker2 # [ 218.429918] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206700worker2 # [ 218.444950] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513206701worker2 # [ 218.458606] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.102Z level=INFO msg="All 1 paths already cached"6702client # copying 1 paths to 'lb:50051'...6703client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6704worker2 # [ 218.468964] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:513206705worker2 # [ 218.482462] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6706worker2 # [ 218.694420] nix-daemon[848]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)6707worker2 # [ 218.711862] nix-daemon[848]: reaped child process 1926, status = succeeded6708worker2 # [ 218.883662] nix-daemon[848]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6709worker2 # [ 218.899092] nix-daemon[848]: reaped child process 1931, status = succeeded6710lb # [ 218.860414] niks3-server[1130]: 2026/09/21 13:52:22 INFO Received uploads request method=POST path=/api/pending_closures6711worker2 # [ 218.923491] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.567Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6712worker2 # [ 218.925089] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.567Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6713lb # [ 218.889968] niks3-server[1130]: 2026/09/21 13:52:22 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6714worker2 # [ 218.952787] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.597Z level=INFO msg="Uploading 1 narinfos"6715lb # [ 218.904922] niks3-server[1130]: 2026/09/21 13:52:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6716lb # [ 218.914841] niks3-server[1130]: 2026/09/21 13:52:22 INFO Signed narinfos id=72 count=16717lb # [ 218.921829] niks3-server[1130]: 2026/09/21 13:52:22 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6718worker2 # [ 218.979444] nix-grpc-daemon[1875]: time=2026-09-21T13:52:22.623Z level=INFO msg="Upload complete. (480ms)"6719lb # [ 218.926437] niks3-server[1130]: 2026/09/21 13:52:22 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6720lb # [ 218.931197] niks3-server[1130]: 2026/09/21 13:52:22 INFO Completed upload id=726721worker2 # [ 218.988242] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=0 paths=1 nar_bytes_in=6326722lb # [ 218.934363] niks3-server[1130]: 2026/09/21 13:52:22 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6723worker2 # [ 218.996090] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206724worker2 # [ 218.998987] nix-daemon[848]: reaped child process 1921, status = succeeded6725worker2 # [ 219.005526] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6726client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6727client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6728worker2 # [ 219.018291] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6729client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6730client # farm-b> LOG-farm-b-drain6731worker2 # [ 220.058830] nix-daemon[848]: reaped child process 1935, status = succeeded6732worker2 # [ 220.247000] nix-daemon[848]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6733worker2 # [ 220.263709] nix-daemon[848]: reaped child process 1954, status = succeeded6734lb # [ 220.222727] niks3-server[1130]: 2026/09/21 13:52:23 INFO Received uploads request method=POST path=/api/pending_closures6735worker2 # [ 220.287323] nix-grpc-daemon[1875]: time=2026-09-21T13:52:23.930Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6736worker2 # [ 220.288987] nix-grpc-daemon[1875]: time=2026-09-21T13:52:23.930Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6737lb # [ 220.266061] niks3-server[1130]: 2026/09/21 13:52:23 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6738worker2 # [ 220.330303] nix-grpc-daemon[1875]: time=2026-09-21T13:52:23.974Z level=INFO msg="Uploading 1 narinfos"6739lb # [ 220.274690] niks3-server[1130]: 2026/09/21 13:52:23 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6740lb # [ 220.289128] niks3-server[1130]: 2026/09/21 13:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6741lb # [ 220.293868] niks3-server[1130]: 2026/09/21 13:52:23 INFO Signed narinfos id=73 count=16742lb # [ 220.298957] niks3-server[1130]: 2026/09/21 13:52:23 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6743worker2 # [ 220.356822] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.001Z level=INFO msg="Upload complete. (329ms)"6744lb # [ 220.305688] niks3-server[1130]: 2026/09/21 13:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6745lb # [ 220.311681] niks3-server[1130]: 2026/09/21 13:52:23 INFO Completed upload id=736746lb # [ 220.315495] envoy[912]: conn peer=[2001:db8:1::1]:54236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300686747worker2 # [ 220.368856] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16748lb # [ 220.319785] niks3-server[1130]: 2026/09/21 13:52:23 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6749worker2 # [ 220.382398] nix-daemon[848]: reaped child process 1933, status = succeeded6750worker1 # [ 220.328858] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:23Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06751worker1 # [ 220.329214] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:23Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16752worker2 # [ 220.425644] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206753worker2 # [ 220.441200] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513206754worker2 # [ 220.455137] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.099Z level=INFO msg="All 2 paths already cached"6755client # copying 1 paths to 'lb:50051'...6756client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6757worker2 # [ 220.465248] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:513206758worker2 # [ 220.478524] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6759worker2 # [ 220.675653] nix-daemon[848]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6760worker2 # [ 220.691884] nix-daemon[848]: reaped child process 1961, status = succeeded6761worker2 # [ 220.868189] nix-daemon[848]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6762worker2 # [ 220.884853] nix-daemon[848]: reaped child process 1966, status = succeeded6763lb # [ 220.844835] niks3-server[1130]: 2026/09/21 13:52:24 INFO Received uploads request method=POST path=/api/pending_closures6764worker2 # [ 220.907096] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.551Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6765worker2 # [ 220.908705] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.551Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6766lb # [ 220.871744] niks3-server[1130]: 2026/09/21 13:52:24 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6767lb # [ 220.885256] niks3-server[1130]: 2026/09/21 13:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6768worker2 # [ 220.944615] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.588Z level=INFO msg="Uploading 1 narinfos"6769lb # [ 220.895362] niks3-server[1130]: 2026/09/21 13:52:24 INFO Signed narinfos id=74 count=16770lb # [ 220.901634] niks3-server[1130]: 2026/09/21 13:52:24 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6771lb # [ 220.910129] niks3-server[1130]: 2026/09/21 13:52:24 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6772lb # [ 220.916469] niks3-server[1130]: 2026/09/21 13:52:24 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6773lb # [ 220.921886] niks3-server[1130]: 2026/09/21 13:52:24 INFO Completed upload id=746774worker2 # [ 220.975587] nix-grpc-daemon[1875]: time=2026-09-21T13:52:24.620Z level=INFO msg="Upload complete. (481ms)"6775worker2 # [ 220.984324] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=0 paths=1 nar_bytes_in=7606776worker2 # [ 220.997625] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513206777worker2 # [ 221.003472] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6778worker2 # [ 221.009240] nix-daemon[848]: reaped child process 1956, status = succeeded6779client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6780client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6781worker2 # [ 221.022412] nix-daemon[848]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6782client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6783client # farm-top> LOG-farm-top-drain6784worker2 # [ 222.448382] nix-daemon[848]: reaped child process 1970, status = succeeded6785worker2 # [ 222.608681] nix-daemon[848]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6786worker2 # [ 222.627281] nix-daemon[848]: reaped child process 1989, status = succeeded6787lb # [ 222.587631] niks3-server[1130]: 2026/09/21 13:52:26 INFO Received uploads request method=POST path=/api/pending_closures6788worker2 # [ 222.652088] nix-grpc-daemon[1875]: time=2026-09-21T13:52:26.294Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6789worker2 # [ 222.653557] nix-grpc-daemon[1875]: time=2026-09-21T13:52:26.294Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6790lb # [ 222.617341] niks3-server[1130]: 2026/09/21 13:52:26 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6791lb # [ 222.630737] niks3-server[1130]: 2026/09/21 13:52:26 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6792worker2 # [ 222.698219] nix-grpc-daemon[1875]: time=2026-09-21T13:52:26.342Z level=INFO msg="Uploading 1 narinfos"6793lb # [ 222.645677] niks3-server[1130]: 2026/09/21 13:52:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6794lb # [ 222.653829] niks3-server[1130]: 2026/09/21 13:52:26 INFO Signed narinfos id=75 count=16795lb # [ 222.658733] niks3-server[1130]: 2026/09/21 13:52:26 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6796lb # [ 222.666211] niks3-server[1130]: 2026/09/21 13:52:26 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6797lb # [ 222.670535] niks3-server[1130]: 2026/09/21 13:52:26 INFO Completed upload id=756798worker2 # [ 222.723629] nix-grpc-daemon[1875]: time=2026-09-21T13:52:26.368Z level=INFO msg="Upload complete. (291ms)"6799lb # [ 222.673206] niks3-server[1130]: 2026/09/21 13:52:26 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6800worker2 # [ 222.732896] nix-grpc-daemon[1815]: ts=2026-09-21T13:52:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51320 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16801worker2 # [ 222.747459] nix-daemon[848]: reaped child process 1968, status = succeeded6802client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.35 seconds)6803client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6804client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6805worker1: must succeed: rm /nix/.rw-store/fill6806worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)6807worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6808worker1 # [ 223.347734] nix-grpc-daemon[1543]: ts=2026-09-21T13:52:26Z level=info event=healthy reason=min_free6809worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.13 seconds)6810(finished: subtest: low disk drains a worker and builds go to the other, in 10.08 seconds)6811(finished: run the VM test script, in 225.27 seconds)6812test script finished in 225.56s6813cleanup6814kill QemuMachine (pid 49)6815client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6816kill QemuMachine (pid 51)6817lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6818kill QemuMachine (pid 50)6819worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6820kill QemuMachine (pid 52)6821worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6822(finished: cleanup, in 1.16 seconds)6823additionally exposed symbols:6824 client, lb, worker1, worker2,6825 vlan1,6826 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh