vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #344
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.FnupkDuFq2', fmt=raw size=107374182414client: QEMU running (pid 49)15client # mke2fs 1.47.4 (6-Mar-2025)16worker2 # Disk image does not exist, creating the virtualisation disk image...17worker1 # Disk image does not exist, creating the virtualisation disk image...18worker1 # Formatting '/build/vm-state-worker1/tmp.yCULaDm2nP', fmt=raw size=107374182419worker1 # mke2fs 1.47.4 (6-Mar-2025)20worker1 # Discarding device blocks: 0/262144 done21worker2: QEMU running (pid 52)22worker1: QEMU running (pid 51)23worker2 # Formatting '/build/vm-state-worker2/tmp.UcrQSVRvEG', fmt=raw size=107374182424client # Discarding device blocks: 0/262144 done25worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes26worker1 # Filesystem UUID: e713126a-ace6-4981-a9e2-c0c326cef6e927worker1 # Superblock backups stored on blocks:28worker1 # 32768, 98304, 163840, 22937629worker1 # 30worker1 # Allocating group tables: 0/8 done31client # Creating filesystem with 262144 4k blocks and 65536 inodes32client # Filesystem UUID: 8eb6c99e-ea5d-4c70-9d5a-c15100a3aa5833client # Superblock backups stored on blocks:34client # 32768, 98304, 163840, 22937635client # 36client # Allocating group tables: 0/8 done37client # Writing inode tables: 0/8 done38worker2 # mke2fs 1.47.4 (6-Mar-2025)39client # Creating journal (8192 blocks): done40worker1 # Writing inode tables: 0/8 done41client # Writing superblocks and filesystem accounting information: 0/8 done42worker1 # Creating journal (8192 blocks): done43worker1 # Writing superblocks and filesystem accounting information: 0/8 done44worker1 # 45worker1 # Virtualisation disk image created.46worker2 # Discarding device blocks: 0/262144 done47worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes48worker2 # Filesystem UUID: e5b31632-f9ca-480f-bb97-673908c74cd449worker2 # Superblock backups stored on blocks:50worker2 # 32768, 98304, 163840, 22937651worker2 # 52worker2 # Allocating group tables: 0/8 done53worker2 # Writing inode tables: 0/8 done54worker2 # Creating journal (8192 blocks): done55worker2 # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # 57worker2 # Virtualisation disk image created.58client # 59client # Virtualisation disk image created.60worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]61worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202662worker1 # [ 0.000000] KASLR enabled63worker1 # [ 0.000000] random: crng init done64worker1 # [ 0.000000] Machine model: linux,dummy-virt65worker1 # [ 0.000000] efi: UEFI not found.66worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT67worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]68worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]69worker1 # [ 0.000000] Zone ranges:70worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]71worker1 # [ 0.000000] DMA32 empty72worker1 # [ 0.000000] Normal empty73worker1 # [ 0.000000] Device empty74worker1 # [ 0.000000] Movable zone start for each node75worker1 # [ 0.000000] Early memory node ranges76worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]77lb: QEMU running (pid 50)78lb # Disk image does not exist, creating the virtualisation disk image...79worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]80lb # Formatting '/build/vm-state-lb/tmp.PDkwBsaGhB', fmt=raw size=107374182481worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000082lb # mke2fs 1.47.4 (6-Mar-2025)83lb # Discarding device blocks: 0/262144 done84worker1 # [ 0.000000] psci: probing for conduit method from DT.85worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.86worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs87lb # Creating filesystem with 262144 4k blocks and 65536 inodes88lb # Filesystem UUID: 9e3ddd88-7b5e-4aa2-a6e9-82bf171896e589lb # Superblock backups stored on blocks:90lb # 32768, 98304, 163840, 22937691lb # 92lb # Allocating group tables: 0/8 done93lb # Writing inode tables: 0/8 done94lb # Creating journal (8192 blocks): done95lb # Writing superblocks and filesystem accounting information: 0/8 done96lb # 97lb # Virtualisation disk image created.98client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]99worker1 # [ 0.000000] psci: Trusted OS migration not required100worker1 # [ 0.000000] psci: SMC Calling Convention v1.1101(finished: start all VMs, in 1.43 seconds)102worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)103client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026104client # [ 0.000000] KASLR enabled105client # [ 0.000000] random: crng init done106worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296107worker1 # [ 0.000000] Detected PIPT I-cache on CPU0108client: waiting for unit network-addresses-eth1.service109client: waiting for the VM to finish booting110worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)111client # [ 0.000000] Machine model: linux,dummy-virt112client # [ 0.000000] efi: UEFI not found.113client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT114worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface115worker1 # [ 0.000000] CPU features: detected: Spectre-v4116worker1 # [ 0.000000] CPU features: detected: Spectre-BHB117client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]118worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38119client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]120client # [ 0.000000] Zone ranges:121worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23122client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]123worker1 # [ 0.000000] alternatives: applying boot alternatives124client # [ 0.000000] DMA32 empty125client # [ 0.000000] Normal empty126client # [ 0.000000] Device empty127client # [ 0.000000] Movable zone start for each node128client # [ 0.000000] Early memory node ranges129client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]130client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]131client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000132worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9gg6bqj6936y7fn6kvx26f1n44j8gpbl-nixos-system-worker1-test/init regInfo=/nix/store/r8kflbsh1zmif219y6bjg8z5vag6dy1v-closure-info/registration console=ttyAMA0,115200n8 console=tty0133client # [ 0.000000] psci: probing for conduit method from DT.134client # [ 0.000000] psci: PSCIv1.3 detected in firmware.135worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/r8kflbsh1zmif219y6bjg8z5vag6dy1v-closure-info/registration", will be passed to user space.136client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs137worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes138client # [ 0.000000] psci: Trusted OS migration not required139client # [ 0.000000] psci: SMC Calling Convention v1.1140worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)141worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)142client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)143worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB144client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296145worker1 # [ 0.000000] software IO TLB: area num 1.146client # [ 0.000000] Detected PIPT I-cache on CPU0147worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB148client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)149client # [ 0.000000] CPU features: detected: GICv3 CPU interface150worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)151worker1 # [ 0.000000] Fallback order for Node 0: 0152client # [ 0.000000] CPU features: detected: Spectre-v4153client # [ 0.000000] CPU features: detected: Spectre-BHB154worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216155worker1 # [ 0.000000] Policy zone: DMA156client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38157worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off158client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23159client # [ 0.000000] alternatives: applying boot alternatives160worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1161worker1 # [ 0.000000] allocated 3145728 bytes of page_ext162worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages163worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups164worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.165worker1 # [ 0.000000] rcu: RCU event tracing is enabled.166client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test/init regInfo=/nix/store/1014gcc8vm0ydc1b8g1pnhswd2qhvga4-closure-info/registration console=ttyAMA0,115200n8 console=tty0167worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.168worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.169worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.170client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/1014gcc8vm0ydc1b8g1pnhswd2qhvga4-closure-info/registration", will be passed to user space.171worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.172client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes173worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.174client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)175worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1176client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)177worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.178client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB179worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.180client # [ 0.000000] software IO TLB: area num 1.181client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB182worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.183client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)184worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0185client # [ 0.000000] Fallback order for Node 0: 0186worker1 # [ 0.000000] GICv3: 256 SPIs implemented187worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented188client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216189worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq190client # [ 0.000000] Policy zone: DMA191worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI192client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off193worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0194client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1195worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000196client # [ 0.000000] allocated 3145728 bytes of page_ext197worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]198client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages199client # [ 0.000000] ftrace: allocated 294 pages with 4 groups200worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)201client # [ 0.000000] rcu: Hierarchical RCU implementation.202client # [ 0.000000] rcu: RCU event tracing is enabled.203client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.204client # [ 0.000000] Trampoline variant of Tasks RCU enabled.205client # [ 0.000000] Rude variant of Tasks RCU enabled.206worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)207client # [ 0.000000] Tracing variant of Tasks RCU enabled.208worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000209client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.210worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]211worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000212client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1213worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.214worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026215client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.216worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns217worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).218client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.219client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.220client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0221client # [ 0.000000] GICv3: 256 SPIs implemented222client # [ 0.000000] GICv3: 0 Extended SPIs implemented223client # [ 0.000000] Root IRQ handler: gic_handle_irq224client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI225client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0226client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000227client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]228worker2 # [ 0.000000] KASLR enabled229client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)230client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)231client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000232client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000233client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.234client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns235client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).236client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns237client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns238client # [ 0.000032] arm-pv: using stolen time PV239client # [ 0.000467] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)240client # [ 0.000683] Console: colour dummy device 80x25241client # [ 0.000692] printk: legacy console [tty0] enabled242client # [ 0.000887] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)243client # [ 0.000894] pid_max: default: 32768 minimum: 301244worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns245worker2 # [ 0.000000] random: crng init done246worker2 # [ 0.000000] Machine model: linux,dummy-virt247worker2 # [ 0.000000] efi: UEFI not found.248worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT249worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]250worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]251worker2 # [ 0.000000] Zone ranges:252worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]253worker2 # [ 0.000000] DMA32 empty254worker2 # [ 0.000000] Normal empty255worker2 # [ 0.000000] Device empty256worker2 # [ 0.000000] Movable zone start for each node257worker2 # [ 0.000000] Early memory node ranges258worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]259worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]260worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000261worker2 # [ 0.000000] psci: probing for conduit method from DT.262worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.263worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs264worker2 # [ 0.000000] psci: Trusted OS migration not required265worker2 # [ 0.000000] psci: SMC Calling Convention v1.1266worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)267worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296268worker2 # [ 0.000000] Detected PIPT I-cache on CPU0269worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)270worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface271worker2 # [ 0.000000] CPU features: detected: Spectre-v4272worker2 # [ 0.000000] CPU features: detected: Spectre-BHB273worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38274worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23275worker2 # [ 0.000000] alternatives: applying boot alternatives276worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns277worker1 # [ 0.000033] arm-pv: using stolen time PV278client # [ 0.000977] LSM: initializing lsm=capability,landlock,yama,bpf,ima279client # [ 0.001151] landlock: Up and running.280client # [ 0.001156] Yama: becoming mindful.281client # [ 0.001647] LSM support for eBPF active282worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5pw6fnvwaf9lkcgs5rfp8j5wx00ap0rn-nixos-system-worker2-test/init regInfo=/nix/store/pka1z0bf00479w25dzm2g81g6w34m1zm-closure-info/registration console=ttyAMA0,115200n8 console=tty0283client # [ 0.001802] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)284worker1 # [ 0.000620] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)285client # [ 0.001841] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)286worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pka1z0bf00479w25dzm2g81g6w34m1zm-closure-info/registration", will be passed to user space.287worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes288client # [ 0.003071] cacheinfo: Unable to detect cache hierarchy for CPU 0289client # [ 0.004414] rcu: Hierarchical SRCU implementation.290worker1 # [ 0.000818] Console: colour dummy device 80x25291worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)292worker1 # [ 0.000827] printk: legacy console [tty0] enabled293client # [ 0.004420] rcu: Max phase no-delay instances is 1000.294worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)295client # [ 0.005644] fsl-mc MSI: its@8080000 domain created296client # [ 0.005739] EFI services will not be available.297worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB298worker2 # [ 0.000000] software IO TLB: area num 1.299client # [ 0.005837] smp: Bringing up secondary CPUs ...300worker1 # [ 0.001022] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)301worker1 # [ 0.001030] pid_max: default: 32768 minimum: 301302client # [ 0.005846] smp: Brought up 1 node, 1 CPU303worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB304worker1 # [ 0.001132] LSM: initializing lsm=capability,landlock,yama,bpf,ima305client # [ 0.005850] SMP: Total of 1 processors activated.306client # [ 0.005855] CPU: All CPU(s) started at EL1307worker1 # [ 0.001290] landlock: Up and running.308worker1 # [ 0.001292] Yama: becoming mindful.309worker1 # [ 0.001934] LSM support for eBPF active310worker1 # [ 0.002106] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)311worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)312worker2 # [ 0.000000] Fallback order for Node 0: 0313worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216314worker2 # [ 0.000000] Policy zone: DMA315worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off316worker1 # [ 0.002142] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)317client # [ 0.005866] CPU features: detected: Branch Target Identification318client # [ 0.005871] CPU features: detected: ARMv8.4 Translation Table Level319client # [ 0.005875] CPU features: detected: Instruction cache invalidation not required for I/D coherence320worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1321client # [ 0.005878] CPU features: detected: Data cache clean to the PoU not required for I/D coherence322worker1 # [ 0.003403] cacheinfo: Unable to detect cache hierarchy for CPU 0323worker2 # [ 0.000000] allocated 3145728 bytes of page_ext324client # [ 0.005882] CPU features: detected: Common not Private translations325client # [ 0.005885] CPU features: detected: CRC32 instructions326worker1 # [ 0.004185] rcu: Hierarchical SRCU implementation.327worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages328worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups329worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.330client # [ 0.005888] CPU features: detected: Data cache clean to Point of Deep Persistence331worker2 # [ 0.000000] rcu: RCU event tracing is enabled.332client # [ 0.005892] CPU features: detected: Data cache clean to Point of Persistence333worker1 # [ 0.004189] rcu: Max phase no-delay instances is 1000.334worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.335worker1 # [ 0.005473] fsl-mc MSI: its@8080000 domain created336worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.337client # [ 0.005895] CPU features: detected: Data independent timing control (DIT)338client # [ 0.005899] CPU features: detected: E0PD339worker1 # [ 0.005600] EFI services will not be available.340worker1 # [ 0.005680] smp: Bringing up secondary CPUs ...341client # [ 0.005901] CPU features: detected: Enhanced Counter Virtualization342worker1 # [ 0.005689] smp: Brought up 1 node, 1 CPU343worker1 # [ 0.005692] SMP: Total of 1 processors activated.344worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.345worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.346worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.347worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1348worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.349worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.350client # [ 0.005904] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)351worker1 # [ 0.005699] CPU: All CPU(s) started at EL1352worker1 # [ 0.005711] CPU features: detected: Branch Target Identification353worker1 # [ 0.005716] CPU features: detected: ARMv8.4 Translation Table Level354worker1 # [ 0.005719] CPU features: detected: Instruction cache invalidation not required for I/D coherence355worker1 # [ 0.005723] CPU features: detected: Data cache clean to the PoU not required for I/D coherence356worker1 # [ 0.005727] CPU features: detected: Common not Private translations357worker1 # [ 0.005729] CPU features: detected: CRC32 instructions358worker1 # [ 0.005732] CPU features: detected: Data cache clean to Point of Deep Persistence359worker1 # [ 0.005736] CPU features: detected: Data cache clean to Point of Persistence360worker1 # [ 0.005739] CPU features: detected: Data independent timing control (DIT)361worker1 # [ 0.005742] CPU features: detected: E0PD362worker1 # [ 0.005745] CPU features: detected: Enhanced Counter Virtualization363worker1 # [ 0.005748] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)364worker1 # [ 0.005751] CPU features: detected: Enhanced Virtualization Traps365worker1 # [ 0.005754] CPU features: detected: Fine Grained Traps366worker1 # [ 0.005758] CPU features: detected: Generic authentication (architected QARMA5 algorithm)367worker1 # [ 0.005763] CPU features: detected: RCpc load-acquire (LDAPR)368worker1 # [ 0.005766] CPU features: detected: LSE atomic instructions369worker1 # [ 0.005769] CPU features: detected: Privileged Access Never370worker1 # [ 0.005771] CPU features: detected: PMUv3371worker1 # [ 0.005774] CPU features: detected: RAS Extension Support372worker1 # [ 0.005776] CPU features: detected: RASv1p1 Extension Support373worker1 # [ 0.005779] CPU features: detected: Random Number Generator374worker1 # [ 0.005781] CPU features: detected: Speculation barrier (SB)375worker1 # [ 0.005784] CPU features: detected: Stage-2 Force Write-Back376worker1 # [ 0.005786] CPU features: detected: TLB range maintenance instructions377worker1 # [ 0.005790] CPU features: detected: Speculative Store Bypassing Safe (SSBS)378worker1 # [ 0.005828] alternatives: applying system-wide alternatives379worker1 # [ 0.009024] CPU features: detected: BBM Level 2 without TLB conflict abort380worker1 # [ 0.009208] Memory: 1406768K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)381worker1 # [ 0.009802] devtmpfs: initialized382worker1 # [ 0.012728] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)383worker1 # [ 0.012753] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).384worker1 # [ 0.012950] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL385worker1 # [ 0.012955] 0 pages in range for non-PLT usage386worker1 # [ 0.012956] 508288 pages in range for PLT usage387worker1 # [ 0.013131] pinctrl core: initialized pinctrl subsystem388worker1 # [ 0.017839] DMI not present or invalid.389worker1 # [ 0.021866] NET: Registered PF_NETLINK/PF_ROUTE protocol family390worker1 # [ 0.024354] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations391worker1 # [ 0.024570] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations392worker1 # [ 0.024787] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations393worker1 # [ 0.024821] audit: initializing netlink subsys (disabled)394worker1 # [ 0.025536] thermal_sys: Registered thermal governor 'fair_share'395worker1 # [ 0.025538] thermal_sys: Registered thermal governor 'bang_bang'396worker1 # [ 0.025542] thermal_sys: Registered thermal governor 'step_wise'397worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.398worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0399worker2 # [ 0.000000] GICv3: 256 SPIs implemented400worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented401worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq402worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI403client # [ 0.005908] CPU features: detected: Enhanced Virtualization Traps404client # [ 0.005911] CPU features: detected: Fine Grained Traps405client # [ 0.005914] CPU features: detected: Generic authentication (architected QARMA5 algorithm)406client # [ 0.005919] CPU features: detected: RCpc load-acquire (LDAPR)407client # [ 0.005922] CPU features: detected: LSE atomic instructions408client # [ 0.005925] CPU features: detected: Privileged Access Never409client # [ 0.005928] CPU features: detected: PMUv3410client # [ 0.005930] CPU features: detected: RAS Extension Support411client # [ 0.005933] CPU features: detected: RASv1p1 Extension Support412client # [ 0.005936] CPU features: detected: Random Number Generator413client # [ 0.005938] CPU features: detected: Speculation barrier (SB)414client # [ 0.005941] CPU features: detected: Stage-2 Force Write-Back415client # [ 0.005944] CPU features: detected: TLB range maintenance instructions416client # [ 0.005949] CPU features: detected: Speculative Store Bypassing Safe (SSBS)417client # [ 0.005988] alternatives: applying system-wide alternatives418client # [ 0.035771] CPU features: detected: BBM Level 2 without TLB conflict abort419client # [ 0.036086] Memory: 1406512K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)420client # [ 0.036678] devtmpfs: initialized421client # [ 0.038754] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)422client # [ 0.038781] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).423client # [ 0.039033] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL424client # [ 0.039038] 0 pages in range for non-PLT usage425client # [ 0.039039] 508288 pages in range for PLT usage426client # [ 0.039163] pinctrl core: initialized pinctrl subsystem427client # [ 0.040026] DMI not present or invalid.428client # [ 0.043347] NET: Registered PF_NETLINK/PF_ROUTE protocol family429client # [ 0.047340] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations430client # [ 0.047557] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations431client # [ 0.047782] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations432client # [ 0.047834] audit: initializing netlink subsys (disabled)433client # [ 0.048507] thermal_sys: Registered thermal governor 'fair_share'434client # [ 0.048510] thermal_sys: Registered thermal governor 'bang_bang'435client # [ 0.048513] thermal_sys: Registered thermal governor 'step_wise'436client # [ 0.048519] thermal_sys: Registered thermal governor 'user_space'437client # [ 0.048522] thermal_sys: Registered thermal governor 'power_allocator'438client # [ 0.048556] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1439client # [ 0.048567] cpuidle: using governor ladder440client # [ 0.048573] cpuidle: using governor menu441client # [ 0.048785] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.442client # [ 0.048804] ASID allocator initialised with 65536 entries443client # [ 0.050054] Serial: AMBA PL011 UART driver444client # [ 0.055462] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1445client # [ 0.055788] printk: console [ttyAMA0] enabled446worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0447worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000448worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]449worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)450worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)451worker1 # [ 0.025548] thermal_sys: Registered thermal governor 'user_space'452worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000453worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000454worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.455worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns456worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).457worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns458worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns459worker2 # [ 0.000032] arm-pv: using stolen time PV460worker2 # [ 0.000473] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)461worker2 # [ 0.000655] Console: colour dummy device 80x25462worker2 # [ 0.000663] printk: legacy console [tty0] enabled463worker2 # [ 0.000862] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)464worker2 # [ 0.000869] pid_max: default: 32768 minimum: 301465worker2 # [ 0.000944] LSM: initializing lsm=capability,landlock,yama,bpf,ima466worker2 # [ 0.001088] landlock: Up and running.467worker2 # [ 0.001091] Yama: becoming mindful.468worker2 # [ 0.001577] LSM support for eBPF active469worker2 # [ 0.001729] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)470worker2 # [ 0.001765] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)471worker2 # [ 0.003113] cacheinfo: Unable to detect cache hierarchy for CPU 0472worker2 # [ 0.004669] rcu: Hierarchical SRCU implementation.473worker2 # [ 0.004675] rcu: Max phase no-delay instances is 1000.474worker2 # [ 0.005912] fsl-mc MSI: its@8080000 domain created475worker2 # [ 0.006007] EFI services will not be available.476worker2 # [ 0.006115] smp: Bringing up secondary CPUs ...477worker2 # [ 0.006124] smp: Brought up 1 node, 1 CPU478worker2 # [ 0.006128] SMP: Total of 1 processors activated.479worker2 # [ 0.006133] CPU: All CPU(s) started at EL1480worker2 # [ 0.006146] CPU features: detected: Branch Target Identification481worker2 # [ 0.006152] CPU features: detected: ARMv8.4 Translation Table Level482worker2 # [ 0.006155] CPU features: detected: Instruction cache invalidation not required for I/D coherence483worker2 # [ 0.006158] CPU features: detected: Data cache clean to the PoU not required for I/D coherence484worker2 # [ 0.006162] CPU features: detected: Common not Private translations485worker2 # [ 0.006165] CPU features: detected: CRC32 instructions486worker1 # [ 0.025551] thermal_sys: Registered thermal governor 'power_allocator'487worker2 # [ 0.006168] CPU features: detected: Data cache clean to Point of Deep Persistence488worker1 # [ 0.025594] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1489worker2 # [ 0.006172] CPU features: detected: Data cache clean to Point of Persistence490worker1 # [ 0.025608] cpuidle: using governor ladder491worker2 # [ 0.006175] CPU features: detected: Data independent timing control (DIT)492worker1 # [ 0.025613] cpuidle: using governor menu493worker2 # [ 0.006178] CPU features: detected: E0PD494worker2 # [ 0.006181] CPU features: detected: Enhanced Counter Virtualization495worker2 # [ 0.006184] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)496worker2 # [ 0.006187] CPU features: detected: Enhanced Virtualization Traps497worker2 # [ 0.006190] CPU features: detected: Fine Grained Traps498worker1 # [ 0.025820] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.499worker1 # [ 0.025840] ASID allocator initialised with 65536 entries500worker1 # [ 0.027080] Serial: AMBA PL011 UART driver501worker1 # [ 0.032578] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1502client # [ 0.178344] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages503worker2 # [ 0.006194] CPU features: detected: Generic authentication (architected QARMA5 algorithm)504worker2 # [ 0.006198] CPU features: detected: RCpc load-acquire (LDAPR)505worker2 # [ 0.006201] CPU features: detected: LSE atomic instructions506client # [ 0.178381] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page507worker1 # [ 0.032785] printk: console [ttyAMA0] enabled508worker2 # [ 0.006204] CPU features: detected: Privileged Access Never509worker2 # [ 0.006207] CPU features: detected: PMUv3510client # [ 0.178387] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages511client # [ 0.178392] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page512client # [ 0.178396] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages513client # [ 0.178401] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page514client # [ 0.178405] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages515client # [ 0.178410] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page516worker2 # [ 0.006210] CPU features: detected: RAS Extension Support517worker2 # [ 0.006213] CPU features: detected: RASv1p1 Extension Support518worker2 # [ 0.006215] CPU features: detected: Random Number Generator519worker2 # [ 0.006218] CPU features: detected: Speculation barrier (SB)520worker2 # [ 0.006221] CPU features: detected: Stage-2 Force Write-Back521worker2 # [ 0.006224] CPU features: detected: TLB range maintenance instructions522worker2 # [ 0.006228] CPU features: detected: Speculative Store Bypassing Safe (SSBS)523worker2 # [ 0.006266] alternatives: applying system-wide alternatives524worker2 # [ 0.034080] CPU features: detected: BBM Level 2 without TLB conflict abort525worker2 # [ 0.035263] Memory: 1406552K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)526worker2 # [ 0.035813] devtmpfs: initialized527worker1 # [ 0.180199] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages528worker2 # [ 0.037718] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)529worker1 # [ 0.180230] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page530worker2 # [ 0.037740] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).531worker1 # [ 0.180235] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages532worker2 # [ 0.037945] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL533worker1 # [ 0.180239] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page534worker2 # [ 0.037949] 0 pages in range for non-PLT usage535worker2 # [ 0.037950] 508288 pages in range for PLT usage536worker1 # [ 0.180243] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages537worker2 # [ 0.038108] pinctrl core: initialized pinctrl subsystem538worker1 # [ 0.180248] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page539worker2 # [ 0.038933] DMI not present or invalid.540worker1 # [ 0.180252] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages541worker2 # [ 0.043165] NET: Registered PF_NETLINK/PF_ROUTE protocol family542worker1 # [ 0.180256] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page543worker2 # [ 0.045552] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations544worker2 # [ 0.045759] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations545worker2 # [ 0.045971] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations546worker2 # [ 0.046000] audit: initializing netlink subsys (disabled)547worker2 # [ 0.046669] thermal_sys: Registered thermal governor 'fair_share'548worker2 # [ 0.046671] thermal_sys: Registered thermal governor 'bang_bang'549worker2 # [ 0.046675] thermal_sys: Registered thermal governor 'step_wise'550worker2 # [ 0.046681] thermal_sys: Registered thermal governor 'user_space'551worker1 # [ 0.193479] fbcon: Taking over console552worker2 # [ 0.046683] thermal_sys: Registered thermal governor 'power_allocator'553worker1 # [ 0.193531] ACPI: Interpreter disabled.554worker2 # [ 0.046708] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1555worker2 # [ 0.046719] cpuidle: using governor ladder556worker2 # [ 0.046747] cpuidle: using governor menu557worker2 # [ 0.046942] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.558worker2 # [ 0.046959] ASID allocator initialised with 65536 entries559worker2 # [ 0.048216] Serial: AMBA PL011 UART driver560worker2 # [ 0.053555] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1561worker2 # [ 0.053727] printk: console [ttyAMA0] enabled562worker2 # [ 0.175254] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages563worker2 # [ 0.175278] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page564client # [ 0.187379] fbcon: Taking over console565worker2 # [ 0.175284] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages566client # [ 0.187414] ACPI: Interpreter disabled.567worker2 # [ 0.175289] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page568worker2 # [ 0.175293] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages569worker2 # [ 0.175297] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page570worker2 # [ 0.175302] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages571worker2 # [ 0.175306] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page572worker1 # [ 0.215876] iommu: Default domain type: Translated573worker1 # [ 0.215896] iommu: DMA domain TLB invalidation policy: strict mode574worker1 # [ 0.216374] SCSI subsystem initialized575worker2 # [ 0.182868] fbcon: Taking over console576worker2 # [ 0.182886] ACPI: Interpreter disabled.577worker1 # [ 0.218527] usbcore: registered new interface driver usbfs578worker1 # [ 0.218555] usbcore: registered new interface driver hub579worker1 # [ 0.218588] usbcore: registered new device driver usb580worker1 # [ 0.218920] pps_core: LinuxPPS API ver. 1 registered581worker1 # [ 0.218927] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>582worker1 # [ 0.218936] PTP clock support registered583worker1 # [ 0.218987] EDAC MC: Ver: 3.0.0584client # [ 0.232319] iommu: Default domain type: Translated585worker1 # [ 0.223771] scmi_core: SCMI protocol bus registered586client # [ 0.232351] iommu: DMA domain TLB invalidation policy: strict mode587client # [ 0.234065] SCSI subsystem initialized588worker1 # [ 0.224776] FPGA manager framework589worker1 # [ 0.225790] vgaarb: loaded590client # [ 0.234785] usbcore: registered new interface driver usbfs591worker2 # [ 0.191522] iommu: Default domain type: Translated592client # [ 0.234832] usbcore: registered new interface driver hub593worker1 # [ 0.226492] clocksource: Switched to clocksource arch_sys_counter594worker2 # [ 0.191538] iommu: DMA domain TLB invalidation policy: strict mode595client # [ 0.234863] usbcore: registered new device driver usb596client # [ 0.235168] pps_core: LinuxPPS API ver. 1 registered597worker2 # [ 0.193293] SCSI subsystem initialized598worker1 # [ 0.228651] VFS: Disk quotas dquot_6.6.0599client # [ 0.235176] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>600worker2 # [ 0.194004] usbcore: registered new interface driver usbfs601client # [ 0.235186] PTP clock support registered602client # [ 0.235237] EDAC MC: Ver: 3.0.0603worker2 # [ 0.194039] usbcore: registered new interface driver hub604worker1 # [ 0.228699] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)605worker2 # [ 0.194065] usbcore: registered new device driver usb606client # [ 0.240015] scmi_core: SCMI protocol bus registered607worker1 # [ 0.231076] netfs: FS-Cache loaded608worker2 # [ 0.194373] pps_core: LinuxPPS API ver. 1 registered609worker1 # [ 0.231256] pnp: PnP ACPI: disabled610client # [ 0.241038] FPGA manager framework611worker2 # [ 0.194380] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>612client # [ 0.242154] vgaarb: loaded613worker2 # [ 0.194390] PTP clock support registered614worker2 # [ 0.194445] EDAC MC: Ver: 3.0.0615worker2 # [ 0.199199] scmi_core: SCMI protocol bus registered616worker2 # [ 0.200218] FPGA manager framework617worker2 # [ 0.201189] vgaarb: loaded618worker2 # [ 0.201844] clocksource: Switched to clocksource arch_sys_counter619client # [ 0.242884] clocksource: Switched to clocksource arch_sys_counter620client # [ 0.244312] VFS: Disk quotas dquot_6.6.0621client # [ 0.244362] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)622client # [ 0.249207] netfs: FS-Cache loaded623client # [ 0.249341] pnp: PnP ACPI: disabled624worker2 # [ 0.206667] VFS: Disk quotas dquot_6.6.0625worker2 # [ 0.206717] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)626worker1 # [ 0.239059] NET: Registered PF_INET protocol family627worker2 # [ 0.208385] netfs: FS-Cache loaded628worker1 # [ 0.239361] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)629worker2 # [ 0.208528] pnp: PnP ACPI: disabled630client # [ 0.253540] NET: Registered PF_INET protocol family631client # [ 0.253827] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)632worker2 # [ 0.212510] NET: Registered PF_INET protocol family633worker2 # [ 0.212802] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)634client # [ 0.285026] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)635client # [ 0.285095] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)636client # [ 0.285130] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)637client # [ 0.285228] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)638client # [ 0.285373] TCP: Hash tables configured (established 16384 bind 16384)639client # [ 0.285473] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)640client # [ 0.285515] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)641client # [ 0.285555] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)642client # [ 0.285653] NET: Registered PF_UNIX/PF_LOCAL protocol family643client # [ 0.285699] NET: Registered PF_XDP protocol family644client # [ 0.285720] PCI: CLS 0 bytes, default 64645client # [ 0.285983] Trying to unpack rootfs image as initramfs...646worker1 # [ 0.290111] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)647worker1 # [ 0.290184] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)648worker1 # [ 0.290221] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)649client # [ 0.300368] kvm [1]: HYP mode not available650worker1 # [ 0.290327] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)651worker1 # [ 0.290475] TCP: Hash tables configured (established 16384 bind 16384)652worker2 # [ 0.244746] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)653worker2 # [ 0.244806] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)654worker2 # [ 0.244842] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)655worker2 # [ 0.244934] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)656worker2 # [ 0.245081] TCP: Hash tables configured (established 16384 bind 16384)657worker2 # [ 0.245180] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)658worker2 # [ 0.245218] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)659worker2 # [ 0.245258] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)660worker2 # [ 0.245370] NET: Registered PF_UNIX/PF_LOCAL protocol family661worker2 # [ 0.245401] NET: Registered PF_XDP protocol family662worker2 # [ 0.245422] PCI: CLS 0 bytes, default 64663worker2 # [ 0.245718] Trying to unpack rootfs image as initramfs...664worker2 # [ 0.263166] kvm [1]: HYP mode not available665lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]666lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026667lb # [ 0.000000] KASLR enabled668lb # [ 0.000000] random: crng init done669lb # [ 0.000000] Machine model: linux,dummy-virt670lb # [ 0.000000] efi: UEFI not found.671lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT672lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]673lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]674lb # [ 0.000000] Zone ranges:675lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]676lb # [ 0.000000] DMA32 empty677lb # [ 0.000000] Normal empty678lb # [ 0.000000] Device empty679lb # [ 0.000000] Movable zone start for each node680lb # [ 0.000000] Early memory node ranges681lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]682lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]683lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000684lb # [ 0.000000] psci: probing for conduit method from DT.685lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.686lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs687lb # [ 0.000000] psci: Trusted OS migration not required688lb # [ 0.000000] psci: SMC Calling Convention v1.1689lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)690lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296691lb # [ 0.000000] Detected PIPT I-cache on CPU0692lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)693lb # [ 0.000000] CPU features: detected: GICv3 CPU interface694lb # [ 0.000000] CPU features: detected: Spectre-v4695lb # [ 0.000000] CPU features: detected: Spectre-BHB696lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38697lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23698lb # [ 0.000000] alternatives: applying boot alternatives699lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0700lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.701lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes702lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)703lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)704lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB705lb # [ 0.000000] software IO TLB: area num 1.706lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB707lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)708lb # [ 0.000000] Fallback order for Node 0: 0709worker1 # [ 0.420362] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)710lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216711lb # [ 0.000000] Policy zone: DMA712worker1 # [ 0.420467] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)713lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off714worker1 # [ 0.420511] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)715lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1716worker1 # [ 0.420739] NET: Registered PF_UNIX/PF_LOCAL protocol family717worker1 # [ 0.420776] NET: Registered PF_XDP protocol family718worker1 # [ 0.420796] PCI: CLS 0 bytes, default 64719worker1 # [ 0.421094] Trying to unpack rootfs image as initramfs...720lb # [ 0.000000] allocated 3145728 bytes of page_ext721lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages722lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups723lb # [ 0.000000] rcu: Hierarchical RCU implementation.724lb # [ 0.000000] rcu: RCU event tracing is enabled.725worker1 # [ 0.438583] kvm [1]: HYP mode not available726lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.727lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.728lb # [ 0.000000] Rude variant of Tasks RCU enabled.729lb # [ 0.000000] Tracing variant of Tasks RCU enabled.730lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.731lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1732lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.733lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.734lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.735lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0736lb # [ 0.000000] GICv3: 256 SPIs implemented737lb # [ 0.000000] GICv3: 0 Extended SPIs implemented738lb # [ 0.000000] Root IRQ handler: gic_handle_irq739lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI740lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0741lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000742lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]743lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)744lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)745lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000746lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000747worker2 # [ 0.428830] Initialise system trusted keyrings748lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.749worker2 # [ 0.429603] workingset: timestamp_bits=42 max_order=19 bucket_order=0750lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns751worker2 # [ 0.431000] squashfs: version 4.0 (2009/01/31) Phillip Lougher752lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).753lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns754lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns755lb # [ 0.000030] arm-pv: using stolen time PV756worker2 # [ 0.431778] 9p: Installing v9fs 9p2000 file system support757lb # [ 0.000459] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)758lb # [ 0.000632] Console: colour dummy device 80x25759lb # [ 0.000643] printk: legacy console [tty0] enabled760lb # [ 0.000836] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)761lb # [ 0.000844] pid_max: default: 32768 minimum: 301762lb # [ 0.000924] LSM: initializing lsm=capability,landlock,yama,bpf,ima763lb # [ 0.001069] landlock: Up and running.764lb # [ 0.001072] Yama: becoming mindful.765lb # [ 0.001594] LSM support for eBPF active766lb # [ 0.001743] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767lb # [ 0.001785] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768lb # [ 0.002968] cacheinfo: Unable to detect cache hierarchy for CPU 0769lb # [ 0.003800] rcu: Hierarchical SRCU implementation.770lb # [ 0.003805] rcu: Max phase no-delay instances is 1000.771lb # [ 0.005063] fsl-mc MSI: its@8080000 domain created772lb # [ 0.005165] EFI services will not be available.773lb # [ 0.005259] smp: Bringing up secondary CPUs ...774lb # [ 0.005269] smp: Brought up 1 node, 1 CPU775lb # [ 0.005291] SMP: Total of 1 processors activated.776lb # [ 0.005299] CPU: All CPU(s) started at EL1777lb # [ 0.005311] CPU features: detected: Branch Target Identification778lb # [ 0.005316] CPU features: detected: ARMv8.4 Translation Table Level779lb # [ 0.005319] CPU features: detected: Instruction cache invalidation not required for I/D coherence780lb # [ 0.005323] CPU features: detected: Data cache clean to the PoU not required for I/D coherence781lb # [ 0.005326] CPU features: detected: Common not Private translations782lb # [ 0.005330] CPU features: detected: CRC32 instructions783lb # [ 0.005333] CPU features: detected: Data cache clean to Point of Deep Persistence784lb # [ 0.005336] CPU features: detected: Data cache clean to Point of Persistence785lb # [ 0.005340] CPU features: detected: Data independent timing control (DIT)786lb # [ 0.005343] CPU features: detected: E0PD787worker2 # [ 0.455804] Key type asymmetric registered788worker2 # [ 0.455839] Asymmetric key parser 'x509' registered789lb # [ 0.005345] CPU features: detected: Enhanced Counter Virtualization790lb # [ 0.005348] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)791lb # [ 0.005351] CPU features: detected: Enhanced Virtualization Traps792worker2 # [ 0.455921] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)793worker2 # [ 0.458079] io scheduler mq-deadline registered794lb # [ 0.005354] CPU features: detected: Fine Grained Traps795worker2 # [ 0.458089] io scheduler kyber registered796lb # [ 0.005358] CPU features: detected: Generic authentication (architected QARMA5 algorithm)797lb # [ 0.005363] CPU features: detected: RCpc load-acquire (LDAPR)798lb # [ 0.005366] CPU features: detected: LSE atomic instructions799lb # [ 0.005369] CPU features: detected: Privileged Access Never800lb # [ 0.005372] CPU features: detected: PMUv3801lb # [ 0.005374] CPU features: detected: RAS Extension Support802lb # [ 0.005377] CPU features: detected: RASv1p1 Extension Support803lb # [ 0.005380] CPU features: detected: Random Number Generator804lb # [ 0.005382] CPU features: detected: Speculation barrier (SB)805lb # [ 0.005386] CPU features: detected: Stage-2 Force Write-Back806lb # [ 0.005388] CPU features: detected: TLB range maintenance instructions807lb # [ 0.005393] CPU features: detected: Speculative Store Bypassing Safe (SSBS)808lb # [ 0.005434] alternatives: applying system-wide alternatives809lb # [ 0.008564] CPU features: detected: BBM Level 2 without TLB conflict abort810lb # [ 0.008754] Memory: 1406544K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)811lb # [ 0.009296] devtmpfs: initialized812lb # [ 0.011271] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)813lb # [ 0.011295] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).814lb # [ 0.011534] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL815lb # [ 0.011540] 0 pages in range for non-PLT usage816lb # [ 0.011541] 508288 pages in range for PLT usage817lb # [ 0.011671] pinctrl core: initialized pinctrl subsystem818lb # [ 0.012500] DMI not present or invalid.819lb # [ 0.015953] NET: Registered PF_NETLINK/PF_ROUTE protocol family820lb # [ 0.019275] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations821lb # [ 0.019469] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations822lb # [ 0.019681] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations823lb # [ 0.019709] audit: initializing netlink subsys (disabled)824lb # [ 0.020320] thermal_sys: Registered thermal governor 'fair_share'825lb # [ 0.020322] thermal_sys: Registered thermal governor 'bang_bang'826lb # [ 0.020326] thermal_sys: Registered thermal governor 'step_wise'827lb # [ 0.020331] thermal_sys: Registered thermal governor 'user_space'828lb # [ 0.020334] thermal_sys: Registered thermal governor 'power_allocator'829lb # [ 0.020363] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1830lb # [ 0.020372] cpuidle: using governor ladder831lb # [ 0.020377] cpuidle: using governor menu832lb # [ 0.020587] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.833lb # [ 0.020622] ASID allocator initialised with 65536 entries834lb # [ 0.021876] Serial: AMBA PL011 UART driver835lb # [ 0.029531] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1836lb # [ 0.029708] printk: console [ttyAMA0] enabled837lb # [ 0.172525] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages838lb # [ 0.172552] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page839lb # [ 0.172558] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages840lb # [ 0.172563] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page841lb # [ 0.172567] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages842lb # [ 0.172571] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page843lb # [ 0.172575] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages844lb # [ 0.172579] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page845worker2 # [ 0.508335] pl061_gpio 9030000.pl061: PL061 GPIO chip registered846worker2 # [ 0.508921] ledtrig-cpu: registered to indicate activity on CPUs847worker2 # [ 0.509332] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:848lb # [ 0.186214] fbcon: Taking over console849worker2 # [ 0.509360] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000850lb # [ 0.186239] ACPI: Interpreter disabled.851worker2 # [ 0.509379] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000852worker2 # [ 0.509389] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000853lb # [ 0.190121] iommu: Default domain type: Translated854lb # [ 0.190137] iommu: DMA domain TLB invalidation policy: strict mode855lb # [ 0.191883] SCSI subsystem initialized856lb # [ 0.192621] usbcore: registered new interface driver usbfs857lb # [ 0.192658] usbcore: registered new interface driver hub858lb # [ 0.192677] usbcore: registered new device driver usb859client # [ 0.567685] Initialise system trusted keyrings860client # [ 0.568466] workingset: timestamp_bits=42 max_order=19 bucket_order=0861lb # [ 0.194940] pps_core: LinuxPPS API ver. 1 registered862worker2 # [ 0.509415] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits863lb # [ 0.194951] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>864lb # [ 0.194961] PTP clock support registered865lb # [ 0.195012] EDAC MC: Ver: 3.0.0866lb # [ 0.197850] scmi_core: SCMI protocol bus registered867lb # [ 0.198838] FPGA manager framework868lb # [ 0.200009] vgaarb: loaded869worker2 # [ 0.509439] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]870worker2 # [ 0.509549] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00871worker2 # [ 0.509562] pci_bus 0000:00: root bus resource [bus 00-ff]872worker2 # [ 0.509569] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]873worker2 # [ 0.509574] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]874client # [ 0.569788] squashfs: version 4.0 (2009/01/31) Phillip Lougher875worker2 # [ 0.509579] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]876lb # [ 0.200672] clocksource: Switched to clocksource arch_sys_counter877lb # [ 0.201335] VFS: Disk quotas dquot_6.6.0878client # [ 0.569849] 9p: Installing v9fs 9p2000 file system support879worker2 # [ 0.509649] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint880worker2 # [ 0.510125] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint881worker2 # [ 0.510314] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]882worker2 # [ 0.510331] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]883worker2 # [ 0.510364] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]884worker2 # [ 0.510381] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]885worker2 # [ 0.510883] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint886worker2 # [ 0.511075] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]887worker2 # [ 0.511091] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]888worker2 # [ 0.511121] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]889worker2 # [ 0.511565] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint890worker2 # [ 0.511747] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]891worker2 # [ 0.511763] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]892worker2 # [ 0.511793] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]893worker2 # [ 0.512258] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint894worker2 # [ 0.512441] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]895worker2 # [ 0.512459] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]896worker2 # [ 0.512492] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]897lb # [ 0.216746] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)898worker2 # [ 0.512958] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint899lb # [ 0.217935] netfs: FS-Cache loaded900lb # [ 0.218093] pnp: PnP ACPI: disabled901client # [ 0.591715] Key type asymmetric registered902client # [ 0.591754] Asymmetric key parser 'x509' registered903worker2 # [ 0.513142] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]904worker2 # [ 0.513158] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]905client # [ 0.591841] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)906worker2 # [ 0.513188] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]907worker2 # [ 0.513652] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint908client # [ 0.593931] io scheduler mq-deadline registered909client # [ 0.593944] io scheduler kyber registered910worker2 # [ 0.513840] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]911lb # [ 0.222863] NET: Registered PF_INET protocol family912lb # [ 0.223147] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)913worker2 # [ 0.562455] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]914client # [ 0.610137] pl061_gpio 9030000.pl061: PL061 GPIO chip registered915worker2 # [ 0.562512] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]916worker2 # [ 0.563117] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint917worker2 # [ 0.563328] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]918worker2 # [ 0.563347] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]919worker2 # [ 0.563382] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]920worker2 # [ 0.563400] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]921worker2 # [ 0.563934] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint922worker2 # [ 0.564129] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]923worker2 # [ 0.564161] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]924client # [ 0.611654] ledtrig-cpu: registered to indicate activity on CPUs925worker2 # [ 0.564625] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint926client # [ 0.612081] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:927client # [ 0.612109] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000928worker2 # [ 0.564815] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]929worker2 # [ 0.564846] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]930client # [ 0.612123] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000931worker2 # [ 0.565246] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint932client # [ 0.612132] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000933worker2 # [ 0.565432] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]934worker1 # [ 0.613345] Initialise system trusted keyrings935client # [ 0.612156] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits936worker1 # [ 0.614181] workingset: timestamp_bits=42 max_order=19 bucket_order=0937worker2 # [ 0.565687] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint938client # [ 0.612195] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]939client # [ 0.612284] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00940client # [ 0.612293] pci_bus 0000:00: root bus resource [bus 00-ff]941worker1 # [ 0.615557] squashfs: version 4.0 (2009/01/31) Phillip Lougher942client # [ 0.612300] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]943worker1 # [ 0.616283] 9p: Installing v9fs 9p2000 file system support944client # [ 0.612305] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]945client # [ 0.612311] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]946client # [ 0.612413] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint947client # [ 0.612855] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint948client # [ 0.613043] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]949client # [ 0.613059] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]950client # [ 0.613088] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]951client # [ 0.613104] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]952worker2 # [ 0.586092] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]953worker2 # [ 0.586144] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]954client # [ 0.613553] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint955client # [ 0.613736] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]956worker2 # [ 0.586177] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]957client # [ 0.613752] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.586847] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint959client # [ 0.613781] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]960worker2 # [ 0.587047] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]961worker2 # [ 0.587066] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]962client # [ 0.614237] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint963client # [ 0.614421] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]964lb # [ 0.255584] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)965worker2 # [ 0.587097] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966client # [ 0.614437] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]967worker2 # [ 0.587753] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned968lb # [ 0.255654] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)969client # [ 0.614467] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]970worker2 # [ 0.587768] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned971lb # [ 0.255695] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)972client # [ 0.615016] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint973worker2 # [ 0.587774] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned974lb # [ 0.255794] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)975client # [ 0.615203] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]976lb # [ 0.255941] TCP: Hash tables configured (established 16384 bind 16384)977worker2 # [ 0.587822] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned978client # [ 0.615219] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]979lb # [ 0.256060] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)980worker2 # [ 0.587871] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned981client # [ 0.615251] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]982lb # [ 0.256114] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)983worker2 # [ 0.587921] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned984client # [ 0.615717] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint985lb # [ 0.256159] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)986client # [ 0.615899] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]987worker2 # [ 0.587971] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned988lb # [ 0.256329] NET: Registered PF_UNIX/PF_LOCAL protocol family989client # [ 0.615915] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]990lb # [ 0.256369] NET: Registered PF_XDP protocol family991worker2 # [ 0.588020] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned992client # [ 0.615944] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993lb # [ 0.256392] PCI: CLS 0 bytes, default 64994worker2 # [ 0.588071] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned995lb # [ 0.256716] Trying to unpack rootfs image as initramfs...996client # [ 0.616390] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint997worker2 # [ 0.588123] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned998client # [ 0.616574] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]999client # [ 0.616589] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker2 # [ 0.588174] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1001client # [ 0.616619] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002client # [ 0.617073] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1003client # [ 0.617255] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1004client # [ 0.617271] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1005worker2 # [ 0.588225] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1006worker2 # [ 0.588379] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1007worker2 # [ 0.588431] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1008worker2 # [ 0.588454] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1009client # [ 0.617300] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010worker2 # [ 0.588476] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1011client # [ 0.617317] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1012lb # [ 0.274802] kvm [1]: HYP mode not available1013worker2 # [ 0.588499] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1014client # [ 0.617817] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1015worker2 # [ 0.588526] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1016client # [ 0.618010] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1017worker2 # [ 0.588550] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1018client # [ 0.618041] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019worker2 # [ 0.588573] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1020client # [ 0.618517] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1021worker2 # [ 0.588597] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1022client # [ 0.618708] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1023worker2 # [ 0.588621] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1024client # [ 0.618738] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025worker2 # [ 0.588646] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1026worker2 # [ 0.588671] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1027worker2 # [ 0.588696] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1028worker2 # [ 0.588718] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1029worker2 # [ 0.588741] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1030worker2 # [ 0.588764] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1031worker2 # [ 0.588787] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1032worker2 # [ 0.588808] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1033worker2 # [ 0.588831] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1034worker2 # [ 0.588853] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1035worker2 # [ 0.588876] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1036worker2 # [ 0.588898] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1037worker2 # [ 0.588928] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1038worker2 # [ 0.588938] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1039worker2 # [ 0.588943] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1040worker2 # [ 0.634290] pci 0000:00:0a.0: enabling device (0000 -> 0002)1041worker2 # [ 0.656133] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1042worker1 # [ 0.692871] Key type asymmetric registered1043worker1 # [ 0.692903] Asymmetric key parser 'x509' registered1044worker1 # [ 0.693001] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1045worker1 # [ 0.695061] io scheduler mq-deadline registered1046worker1 # [ 0.695071] io scheduler kyber registered1047worker2 # [ 0.666576] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1048worker2 # [ 0.672325] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1049worker1 # [ 0.700210] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1050worker1 # [ 0.700764] ledtrig-cpu: registered to indicate activity on CPUs1051worker1 # [ 0.701244] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1052worker1 # [ 0.701268] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001053worker1 # [ 0.701281] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001054worker1 # [ 0.701289] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001055worker1 # [ 0.701319] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1056worker1 # [ 0.701341] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1057worker1 # [ 0.701418] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001058worker1 # [ 0.701427] pci_bus 0000:00: root bus resource [bus 00-ff]1059worker1 # [ 0.701433] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1060worker1 # [ 0.701438] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1061worker1 # [ 0.701443] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1062worker2 # [ 0.678581] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1063worker1 # [ 0.701525] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1064worker2 # [ 0.680778] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1065worker1 # [ 0.701967] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1066worker1 # [ 0.702155] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1067worker1 # [ 0.702171] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker1 # [ 0.702200] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069worker1 # [ 0.702216] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1070worker1 # [ 0.702701] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1071worker1 # [ 0.702883] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1072worker1 # [ 0.702899] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1073worker1 # [ 0.702929] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker1 # [ 0.703373] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075worker1 # [ 0.703565] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1076worker1 # [ 0.703580] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1077worker1 # [ 0.703609] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker1 # [ 0.704067] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1079worker1 # [ 0.704248] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1080worker1 # [ 0.704263] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1081worker1 # [ 0.704295] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082worker1 # [ 0.704775] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1083worker1 # [ 0.704958] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1084worker1 # [ 0.704975] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1085worker1 # [ 0.705005] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker1 # [ 0.705453] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1087worker1 # [ 0.705660] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1088worker1 # [ 0.705677] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1089worker1 # [ 0.705707] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1090worker1 # [ 0.706176] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1091worker1 # [ 0.706361] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1092worker1 # [ 0.706377] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1093worker1 # [ 0.706406] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094worker1 # [ 0.706422] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1095client # [ 0.679082] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1096client # [ 0.679315] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1097client # [ 0.679601] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1098client # [ 0.679858] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1099client # [ 0.679877] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1100client # [ 0.679908] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker1 # [ 0.754966] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1102client # [ 0.680442] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1103worker1 # [ 0.755170] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1104client # [ 0.680635] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1105worker1 # [ 0.755199] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106client # [ 0.680652] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1107worker1 # [ 0.755667] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1108client # [ 0.680683] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109worker1 # [ 0.755860] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1110client # [ 0.681308] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1111worker1 # [ 0.755891] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112client # [ 0.681320] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1113worker1 # [ 0.756280] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1114client # [ 0.681327] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1115worker1 # [ 0.756464] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1116client # [ 0.681378] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1117worker1 # [ 0.756722] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1118worker1 # [ 0.757048] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1119client # [ 0.681427] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1120client # [ 0.681475] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1121worker1 # [ 0.757067] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1122client # [ 0.681522] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1123worker1 # [ 0.757096] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124client # [ 0.681575] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1125worker1 # [ 0.757588] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1126worker1 # [ 0.757775] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1127client # [ 0.681623] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1128worker1 # [ 0.757791] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1129client # [ 0.681673] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1130worker1 # [ 0.757820] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131client # [ 0.681724] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1132worker1 # [ 0.758396] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1133client # [ 0.681774] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1134worker1 # [ 0.758408] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1135client # [ 0.681862] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1136worker1 # [ 0.758413] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1137client # [ 0.681909] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1138worker1 # [ 0.758461] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1139client # [ 0.681932] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1140worker1 # [ 0.758531] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1141client # [ 0.681953] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1142worker1 # [ 0.758579] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1143client # [ 0.681975] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1144client # [ 0.682002] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1145worker1 # [ 0.758627] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1146client # [ 0.682027] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1147worker1 # [ 0.758673] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1148client # [ 0.682049] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1149worker1 # [ 0.758721] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1150client # [ 0.682073] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1151worker1 # [ 0.758769] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1152worker1 # [ 0.758816] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1153worker1 # [ 0.758864] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1154worker1 # [ 0.758955] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1155worker1 # [ 0.759003] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1156worker1 # [ 0.759025] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1157worker1 # [ 0.759047] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1158worker1 # [ 0.759069] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1159worker1 # [ 0.759094] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1160client # [ 0.682096] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1161client # [ 0.682120] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1162worker1 # [ 0.759116] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1163client # [ 0.682145] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1164client # [ 0.682169] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1165client # [ 0.682193] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1166client # [ 0.682217] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1167client # [ 0.682239] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1168client # [ 0.682263] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1169client # [ 0.682287] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1170client # [ 0.682310] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1171client # [ 0.682332] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1172client # [ 0.682355] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1173worker1 # [ 0.759138] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1174client # [ 0.682379] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1175worker1 # [ 0.759160] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1176client # [ 0.682408] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1177client # [ 0.682419] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1178worker1 # [ 0.759182] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1179client # [ 0.682424] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1180worker1 # [ 0.759205] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1181client # [ 0.683663] pci 0000:00:0a.0: enabling device (0000 -> 0002)1182worker1 # [ 0.759229] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1183worker1 # [ 0.759252] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1184worker1 # [ 0.759275] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1185worker1 # [ 0.759297] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1186worker1 # [ 0.759318] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1187worker1 # [ 0.759340] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1188lb # [ 0.437557] Initialise system trusted keyrings1189worker1 # [ 0.759362] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1190worker1 # [ 0.759384] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1191lb # [ 0.438351] workingset: timestamp_bits=42 max_order=19 bucket_order=01192worker1 # [ 0.759406] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1193lb # [ 0.439737] squashfs: version 4.0 (2009/01/31) Phillip Lougher1194worker1 # [ 0.759428] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1195lb # [ 0.440520] 9p: Installing v9fs 9p2000 file system support1196worker1 # [ 0.759450] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1197worker1 # [ 0.759478] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1198worker1 # [ 0.759487] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1199worker1 # [ 0.759492] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1200worker1 # [ 0.760674] pci 0000:00:0a.0: enabling device (0000 -> 0002)1201worker2 # [ 0.769991] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1202worker2 # [ 0.772416] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1203lb # [ 0.465178] Key type asymmetric registered1204lb # [ 0.465203] Asymmetric key parser 'x509' registered1205worker1 # [ 0.827628] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1206lb # [ 0.465297] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1207lb # [ 0.467419] io scheduler mq-deadline registered1208lb # [ 0.467433] io scheduler kyber registered1209worker2 # [ 0.799845] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1210worker2 # [ 0.807294] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1211lb # [ 0.481872] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1212lb # [ 0.483197] ledtrig-cpu: registered to indicate activity on CPUs1213lb # [ 0.483602] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1214lb # [ 0.483619] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001215lb # [ 0.483630] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001216lb # [ 0.483639] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001217lb # [ 0.483659] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1218lb # [ 0.483681] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1219lb # [ 0.483759] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001220lb # [ 0.483769] pci_bus 0000:00: root bus resource [bus 00-ff]1221lb # [ 0.483775] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1222lb # [ 0.483781] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1223worker2 # [ 0.818519] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1224worker2 # [ 0.821538] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1225lb # [ 0.483786] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1226lb # [ 0.483842] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1227lb # [ 0.484278] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1228lb # [ 0.484462] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1229lb # [ 0.484478] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1230lb # [ 0.484507] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1231lb # [ 0.484523] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1232lb # [ 0.505055] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1233lb # [ 0.505257] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1234lb # [ 0.505288] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1235client # [ 0.883165] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 194829 usecs1236lb # [ 0.505318] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1237lb # [ 0.505777] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1238lb # [ 0.505956] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1239lb # [ 0.505972] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1240lb # [ 0.506001] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241lb # [ 0.506446] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1242lb # [ 0.506624] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1243lb # [ 0.506640] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1244lb # [ 0.506672] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1245lb # [ 0.507123] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1246lb # [ 0.507312] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1247lb # [ 0.507328] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1248lb # [ 0.507358] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1249lb # [ 0.507831] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1250lb # [ 0.508012] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1251lb # [ 0.508028] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1252lb # [ 0.508057] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1253lb # [ 0.508565] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1254worker1 # [ 0.839212] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1255client # [ 0.925972] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1256client # [ 0.936416] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1257client # [ 0.938485] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1258client # [ 0.942188] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1259client # [ 0.945808] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1260client # [ 0.948419] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1261client # [ 0.950315] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1262worker2 # [ 0.911148] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1263client # [ 0.973190] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1264client # [ 0.977281] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1265worker2 # [ 0.935437] msm_serial: driver initialized1266worker2 # [ 0.935627] SuperH (H)SCI(F) driver initialized1267worker2 # [ 0.935681] STM32 USART driver initialized1268client # [ 0.980144] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1269client # [ 0.994968] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1270client # [ 1.009701] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1271worker2 # [ 0.970883] loop: module loaded1272worker2 # [ 0.971092] virtio_blk virtio5: 1/0/0 default/read/poll queues1273worker2 # [ 0.971874] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1274client # [ 1.012668] msm_serial: driver initialized1275client # [ 1.012869] SuperH (H)SCI(F) driver initialized1276client # [ 1.012923] STM32 USART driver initialized1277worker1 # [ 0.986068] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1278worker1 # [ 0.989887] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1279worker1 # [ 0.992318] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1280lb # [ 0.525799] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1281worker1 # [ 0.994760] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1282worker1 # [ 0.997061] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1283worker1 # [ 1.007918] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1284lb # [ 0.525819] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1285lb # [ 0.525849] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1286lb # [ 0.525866] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1287lb # [ 0.526350] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1288lb # [ 0.526538] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1289lb # [ 0.526568] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1290lb # [ 0.527026] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1291lb # [ 0.527212] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1292lb # [ 0.527243] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1293lb # [ 0.527632] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1294lb # [ 0.527811] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1295lb # [ 0.528068] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1296worker1 # [ 1.023584] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1297worker1 # [ 1.025520] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1298worker2 # [ 0.990751] megasas: 07.734.00.00-rc11299worker2 # [ 0.994805] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1300worker2 # [ 0.997649] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001301worker2 # [ 0.997721] Intel/Sharp Extended Query Table at 0x00311302worker2 # [ 0.999587] Using buffer write method1303worker2 # [ 0.999695] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1304worker2 # [ 1.009913] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001305worker2 # [ 1.009981] Intel/Sharp Extended Query Table at 0x00311306worker2 # [ 1.013300] Using buffer write method1307worker2 # [ 1.013345] Concatenating MTD devices:1308worker2 # [ 1.013349] (0): "0.flash"1309worker2 # [ 1.013353] (1): "0.flash"1310worker2 # [ 1.013357] into device "0.flash"1311worker1 # [ 1.086845] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1312client # [ 1.098125] loop: module loaded1313client # [ 1.098375] virtio_blk virtio5: 1/0/0 default/read/poll queues1314client # [ 1.119585] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1315worker1 # [ 1.122585] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1316client # [ 1.128916] megasas: 07.734.00.00-rc11317worker1 # [ 1.126482] msm_serial: driver initialized1318worker1 # [ 1.128186] SuperH (H)SCI(F) driver initialized1319worker1 # [ 1.128310] STM32 USART driver initialized1320client # [ 1.129917] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1321client # [ 1.134772] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001322client # [ 1.134827] Intel/Sharp Extended Query Table at 0x00311323client # [ 1.139930] Using buffer write method1324client # [ 1.140059] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1325client # [ 1.142709] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001326client # [ 1.142756] Intel/Sharp Extended Query Table at 0x00311327client # [ 1.149656] Using buffer write method1328client # [ 1.149720] Concatenating MTD devices:1329client # [ 1.149725] (0): "0.flash"1330client # [ 1.149730] (1): "0.flash"1331client # [ 1.149733] into device "0.flash"1332lb # [ 0.796282] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1333lb # [ 0.796343] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1334lb # [ 0.796375] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1335lb # [ 0.799080] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1336lb # [ 0.799274] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1337lb # [ 0.799291] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1338lb # [ 0.799320] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1339lb # [ 0.799931] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1340lb # [ 0.799944] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1341lb # [ 0.799951] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1342lb # [ 0.799998] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1343lb # [ 0.800046] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1344lb # [ 0.800095] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1345lb # [ 0.800142] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1346lb # [ 0.800190] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1347lb # [ 0.800241] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1348lb # [ 0.800290] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1349worker1 # [ 1.175740] loop: module loaded1350lb # [ 0.800341] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1351worker1 # [ 1.175998] virtio_blk virtio5: 1/0/0 default/read/poll queues1352lb # [ 0.800392] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1353worker1 # [ 1.176896] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1354lb # [ 0.800515] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1355lb # [ 0.800563] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1356lb # [ 0.800584] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1357lb # [ 0.800606] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1358lb # [ 0.800628] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1359lb # [ 0.800653] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1360lb # [ 0.820489] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1361lb # [ 0.820518] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1362lb # [ 0.820541] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1363worker1 # [ 1.183421] megasas: 07.734.00.00-rc11364lb # [ 0.820564] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1365worker1 # [ 1.184225] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1366lb # [ 0.820587] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1367lb # [ 0.820611] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1368lb # [ 0.820636] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1369lb # [ 0.820658] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1370worker1 # [ 1.189195] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001371worker1 # [ 1.189240] Intel/Sharp Extended Query Table at 0x00311372lb # [ 0.827223] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1373lb # [ 0.827250] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1374lb # [ 0.827272] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1375lb # [ 0.827295] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1376lb # [ 0.827317] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1377lb # [ 0.827339] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1378lb # [ 0.827362] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1379lb # [ 0.827384] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1380lb # [ 0.827419] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1381lb # [ 0.827428] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1382lb # [ 0.827433] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1383lb # [ 0.828596] pci 0000:00:0a.0: enabling device (0000 -> 0002)1384worker1 # [ 1.205620] Using buffer write method1385worker1 # [ 1.205763] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1386worker1 # [ 1.207855] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387worker1 # [ 1.207909] Intel/Sharp Extended Query Table at 0x00311388worker1 # [ 1.214178] Using buffer write method1389worker1 # [ 1.214215] Concatenating MTD devices:1390worker1 # [ 1.214219] (0): "0.flash"1391worker1 # [ 1.214223] (1): "0.flash"1392worker1 # [ 1.214226] into device "0.flash"1393lb # [ 0.864335] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1394lb # [ 0.874374] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1395lb # [ 0.885446] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1396lb # [ 0.887778] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1397lb # [ 0.891647] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1398lb # [ 0.925451] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1399lb # [ 0.928524] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1400lb # [ 0.938242] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1401lb # [ 0.944962] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1402lb # [ 0.947082] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1403lb # [ 0.958039] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1404lb # [ 0.974610] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1405lb # [ 0.977793] msm_serial: driver initialized1406lb # [ 0.977983] SuperH (H)SCI(F) driver initialized1407lb # [ 0.978052] STM32 USART driver initialized1408lb # [ 1.019625] loop: module loaded1409lb # [ 1.020501] virtio_blk virtio5: 1/0/0 default/read/poll queues1410lb # [ 1.021393] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1411lb # [ 1.060843] megasas: 07.734.00.00-rc11412lb # [ 1.061872] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413lb # [ 1.080119] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414lb # [ 1.080210] Intel/Sharp Extended Query Table at 0x00311415lb # [ 1.085876] Using buffer write method1416lb # [ 1.086041] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1417lb # [ 1.088222] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418lb # [ 1.088299] Intel/Sharp Extended Query Table at 0x00311419lb # [ 1.097487] Using buffer write method1420lb # [ 1.097544] Concatenating MTD devices:1421lb # [ 1.097549] (0): "0.flash"1422lb # [ 1.097554] (1): "0.flash"1423lb # [ 1.097557] into device "0.flash"1424worker2 # [ 1.556148] Freeing initrd memory: 26148K1425worker2 # [ 1.562532] tun: Universal TUN/TAP device driver, 1.61426worker2 # [ 1.567284] thunder_xcv, ver 1.01427worker2 # [ 1.567338] thunder_bgx, ver 1.01428worker2 # [ 1.567361] nicpf, ver 1.01429worker2 # [ 1.567907] e1000: Intel(R) PRO/1000 Network Driver1430worker2 # [ 1.567915] e1000: Copyright (c) 1999-2006 Intel Corporation.1431worker2 # [ 1.567942] e1000e: Intel(R) PRO/1000 Network Driver1432worker2 # [ 1.567950] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1433worker2 # [ 1.567979] igb: Intel(R) Gigabit Ethernet Network Driver1434worker2 # [ 1.567985] igb: Copyright (c) 2007-2014 Intel Corporation.1435worker2 # [ 1.568007] igbvf: Intel(R) Gigabit Virtual Function Network Driver1436worker2 # [ 1.568013] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1437worker2 # [ 1.568143] sky2: driver version 1.301438worker2 # [ 1.569767] usbcore: registered new interface driver usb-storage1439worker2 # [ 1.570553] ehci-pci 0000:00:0a.0: EHCI Host Controller1440worker2 # [ 1.570582] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11441worker2 # [ 1.570882] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001442worker2 # [ 1.580315] usbcore: registered new interface driver usbserial_generic1443worker2 # [ 1.580340] usbserial: USB Serial support registered for generic1444worker2 # [ 1.581117] hv_vmbus: registering driver hyperv_keyboard1445worker2 # [ 1.598914] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001446worker2 # [ 1.599373] hub 1-0:1.0: USB hub found1447worker2 # [ 1.599395] hub 1-0:1.0: 6 ports detected1448worker2 # [ 1.604533] rtc-pl031 9010000.pl031: registered as rtc01449worker2 # [ 1.604569] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:07 UTC (1789908367)1450worker2 # [ 1.604905] i2c_dev: i2c /dev entries driver1451worker2 # [ 1.610572] sdhci: Secure Digital Host Controller Interface driver1452worker2 # [ 1.610588] sdhci: Copyright(c) Pierre Ossman1453worker2 # [ 1.610876] Synopsys Designware Multimedia Card Interface Driver1454worker2 # [ 1.611291] sdhci-pltfm: SDHCI platform and OF driver helper1455worker2 # [ 1.613103] hid: raw HID events driver (C) Jiri Kosina1456worker2 # [ 1.613351] usbcore: registered new interface driver usbhid1457worker2 # [ 1.613358] usbhid: USB HID core driver1458worker2 # [ 1.618958] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1459worker2 # [ 1.620570] drop_monitor: Initializing network drop monitor service1460worker2 # [ 1.620719] NET: Registered PF_INET6 protocol family1461worker2 # [ 1.624764] Segment Routing with IPv61462worker2 # [ 1.624787] In-situ OAM (IOAM) with IPv61463worker2 # [ 1.624820] NET: Registered PF_PACKET protocol family1464client # [ 1.699663] Freeing initrd memory: 26160K1465worker1 # [ 1.706721] Freeing initrd memory: 26148K1466worker1 # [ 1.714071] tun: Universal TUN/TAP device driver, 1.61467worker1 # [ 1.718475] thunder_xcv, ver 1.01468worker2 # [ 1.626415] 9pnet: Installing 9P2000 support1469worker1 # [ 1.719285] thunder_bgx, ver 1.01470worker1 # [ 1.719324] nicpf, ver 1.01471worker1 # [ 1.719931] e1000: Intel(R) PRO/1000 Network Driver1472worker1 # [ 1.719941] e1000: Copyright (c) 1999-2006 Intel Corporation.1473worker1 # [ 1.719973] e1000e: Intel(R) PRO/1000 Network Driver1474client # [ 1.735125] tun: Universal TUN/TAP device driver, 1.61475worker1 # [ 1.719982] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1476worker1 # [ 1.720021] igb: Intel(R) Gigabit Ethernet Network Driver1477worker1 # [ 1.720027] igb: Copyright (c) 2007-2014 Intel Corporation.1478worker1 # [ 1.720053] igbvf: Intel(R) Gigabit Virtual Function Network Driver1479worker1 # [ 1.720058] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1480worker1 # [ 1.720217] sky2: driver version 1.301481worker1 # [ 1.722111] usbcore: registered new interface driver usb-storage1482worker1 # [ 1.722280] usbcore: registered new interface driver usbserial_generic1483worker1 # [ 1.722301] usbserial: USB Serial support registered for generic1484worker1 # [ 1.723311] ehci-pci 0000:00:0a.0: EHCI Host Controller1485worker1 # [ 1.723349] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11486worker1 # [ 1.723545] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001487client # [ 1.740641] thunder_xcv, ver 1.01488client # [ 1.740720] thunder_bgx, ver 1.01489client # [ 1.740753] nicpf, ver 1.01490client # [ 1.741572] e1000: Intel(R) PRO/1000 Network Driver1491client # [ 1.741587] e1000: Copyright (c) 1999-2006 Intel Corporation.1492client # [ 1.741631] e1000e: Intel(R) PRO/1000 Network Driver1493worker1 # [ 1.735171] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001494worker1 # [ 1.736113] hub 1-0:1.0: USB hub found1495client # [ 1.741647] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1496worker1 # [ 1.736153] hub 1-0:1.0: 6 ports detected1497client # [ 1.741692] igb: Intel(R) Gigabit Ethernet Network Driver1498client # [ 1.741699] igb: Copyright (c) 2007-2014 Intel Corporation.1499worker1 # [ 1.739096] hv_vmbus: registering driver hyperv_keyboard1500client # [ 1.741733] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501client # [ 1.741740] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1502client # [ 1.741952] sky2: driver version 1.301503worker1 # [ 1.740834] rtc-pl031 9010000.pl031: registered as rtc01504worker1 # [ 1.740873] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:07 UTC (1789908367)1505worker1 # [ 1.741254] i2c_dev: i2c /dev entries driver1506worker2 # [ 1.709839] Key type dns_resolver registered1507worker1 # [ 1.746823] sdhci: Secure Digital Host Controller Interface driver1508worker1 # [ 1.746841] sdhci: Copyright(c) Pierre Ossman1509worker1 # [ 1.747120] Synopsys Designware Multimedia Card Interface Driver1510worker1 # [ 1.747517] sdhci-pltfm: SDHCI platform and OF driver helper1511worker1 # [ 1.749514] hid: raw HID events driver (C) Jiri Kosina1512worker1 # [ 1.749778] usbcore: registered new interface driver usbhid1513worker1 # [ 1.749785] usbhid: USB HID core driver1514worker2 # [ 1.720468] registered taskstats version 11515worker2 # [ 1.720678] Loading compiled-in X.509 certificates1516worker1 # [ 1.755614] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1517worker1 # [ 1.757295] drop_monitor: Initializing network drop monitor service1518worker1 # [ 1.757511] NET: Registered PF_INET6 protocol family1519worker1 # [ 1.760952] Segment Routing with IPv61520worker1 # [ 1.760982] In-situ OAM (IOAM) with IPv61521worker1 # [ 1.761032] NET: Registered PF_PACKET protocol family1522client # [ 1.753261] usbcore: registered new interface driver usb-storage1523client # [ 1.753434] usbcore: registered new interface driver usbserial_generic1524worker1 # [ 1.762799] 9pnet: Installing 9P2000 support1525client # [ 1.753455] usbserial: USB Serial support registered for generic1526worker1 # [ 1.765371] Key type dns_resolver registered1527worker2 # [ 1.734555] Demotion targets for Node 0: null1528worker1 # [ 1.772982] registered taskstats version 11529worker2 # [ 1.734755] Key type .fscrypt registered1530client # [ 1.787743] ehci-pci 0000:00:0a.0: EHCI Host Controller1531worker2 # [ 1.734762] Key type fscrypt-provisioning registered1532worker1 # [ 1.773150] Loading compiled-in X.509 certificates1533worker2 # [ 1.734876] ima: No TPM chip found, activating TPM-bypass!1534worker2 # [ 1.734897] ima: Allocated hash algorithm: sha11535worker2 # [ 1.734921] ima: No architecture policies found1536worker2 # [ 1.739225] input: gpio-keys as /devices/platform/gpio-keys/input/input01537client # [ 1.787835] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11538client # [ 1.788078] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001539client # [ 1.790872] hv_vmbus: registering driver hyperv_keyboard1540lb # [ 1.405596] Freeing initrd memory: 26140K1541client # [ 1.792935] rtc-pl031 9010000.pl031: registered as rtc01542client # [ 1.792980] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:07 UTC (1789908367)1543worker1 # [ 1.782143] Demotion targets for Node 0: null1544client # [ 1.793364] i2c_dev: i2c /dev entries driver1545worker1 # [ 1.787485] Key type .fscrypt registered1546worker1 # [ 1.787499] Key type fscrypt-provisioning registered1547worker1 # [ 1.787620] ima: No TPM chip found, activating TPM-bypass!1548worker1 # [ 1.787642] ima: Allocated hash algorithm: sha11549worker1 # [ 1.787673] ima: No architecture policies found1550lb # [ 1.428038] tun: Universal TUN/TAP device driver, 1.61551client # [ 1.799499] sdhci: Secure Digital Host Controller Interface driver1552client # [ 1.799519] sdhci: Copyright(c) Pierre Ossman1553client # [ 1.799853] Synopsys Designware Multimedia Card Interface Driver1554client # [ 1.800305] sdhci-pltfm: SDHCI platform and OF driver helper1555lb # [ 1.432654] thunder_xcv, ver 1.01556client # [ 1.807104] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001557client # [ 1.807649] hub 1-0:1.0: USB hub found1558client # [ 1.807678] hub 1-0:1.0: 6 ports detected1559lb # [ 1.433352] thunder_bgx, ver 1.01560lb # [ 1.433390] nicpf, ver 1.01561lb # [ 1.434031] e1000: Intel(R) PRO/1000 Network Driver1562lb # [ 1.434039] e1000: Copyright (c) 1999-2006 Intel Corporation.1563client # [ 1.811737] hid: raw HID events driver (C) Jiri Kosina1564lb # [ 1.434072] e1000e: Intel(R) PRO/1000 Network Driver1565client # [ 1.812112] usbcore: registered new interface driver usbhid1566lb # [ 1.434083] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1567client # [ 1.812126] usbhid: USB HID core driver1568lb # [ 1.434118] igb: Intel(R) Gigabit Ethernet Network Driver1569lb # [ 1.434124] igb: Copyright (c) 2007-2014 Intel Corporation.1570lb # [ 1.434149] igbvf: Intel(R) Gigabit Virtual Function Network Driver1571lb # [ 1.434155] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1572lb # [ 1.434302] sky2: driver version 1.301573worker2 # [ 1.773543] clk: Disabling unused clocks1574worker2 # [ 1.773592] PM: genpd: Disabling unused power domains1575lb # [ 1.436081] usbcore: registered new interface driver usb-storage1576lb # [ 1.436240] usbcore: registered new interface driver usbserial_generic1577lb # [ 1.436261] usbserial: USB Serial support registered for generic1578lb # [ 1.437763] ehci-pci 0000:00:0a.0: EHCI Host Controller1579lb # [ 1.437815] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11580lb # [ 1.438007] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001581worker2 # [ 1.778212] Freeing unused kernel memory: 4736K1582lb # [ 1.449507] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001583worker2 # [ 1.778483] Run /init as init process1584lb # [ 1.453423] hub 1-0:1.0: USB hub found1585lb # [ 1.453934] hub 1-0:1.0: 6 ports detected1586lb # [ 1.455297] hv_vmbus: registering driver hyperv_keyboard1587worker1 # [ 1.791837] input: gpio-keys as /devices/platform/gpio-keys/input/input01588client # [ 1.815872] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1589lb # [ 1.457057] rtc-pl031 9010000.pl031: registered as rtc01590lb # [ 1.457091] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:07 UTC (1789908367)1591lb # [ 1.457499] i2c_dev: i2c /dev entries driver1592lb # [ 1.463289] sdhci: Secure Digital Host Controller Interface driver1593lb # [ 1.463302] sdhci: Copyright(c) Pierre Ossman1594lb # [ 1.463603] Synopsys Designware Multimedia Card Interface Driver1595lb # [ 1.464031] sdhci-pltfm: SDHCI platform and OF driver helper1596lb # [ 1.469219] hid: raw HID events driver (C) Jiri Kosina1597worker2 # [ 1.800818] systemd[1]: Successfully made /usr/ read-only.1598lb # [ 1.469544] usbcore: registered new interface driver usbhid1599lb # [ 1.469552] usbhid: USB HID core driver1600lb # [ 1.472648] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1601client # [ 1.834160] drop_monitor: Initializing network drop monitor service1602client # [ 1.851841] NET: Registered PF_INET6 protocol family1603client # [ 1.853191] Segment Routing with IPv61604client # [ 1.853217] In-situ OAM (IOAM) with IPv61605client # [ 1.853284] NET: Registered PF_PACKET protocol family1606lb # [ 1.479302] drop_monitor: Initializing network drop monitor service1607lb # [ 1.479534] NET: Registered PF_INET6 protocol family1608client # [ 1.855505] 9pnet: Installing 9P2000 support1609lb # [ 1.487184] Segment Routing with IPv61610worker1 # [ 1.851871] clk: Disabling unused clocks1611worker1 # [ 1.851934] PM: genpd: Disabling unused power domains1612lb # [ 1.487216] In-situ OAM (IOAM) with IPv61613lb # [ 1.487258] NET: Registered PF_PACKET protocol family1614worker1 # [ 1.856466] Freeing unused kernel memory: 4736K1615worker1 # [ 1.856706] Run /init as init process1616client # [ 1.863336] Key type dns_resolver registered1617lb # [ 1.493243] 9pnet: Installing 9P2000 support1618lb # [ 1.496009] Key type dns_resolver registered1619client # [ 1.870391] registered taskstats version 11620client # [ 1.870565] Loading compiled-in X.509 certificates1621lb # [ 1.503855] registered taskstats version 11622lb # [ 1.504095] Loading compiled-in X.509 certificates1623worker1 # [ 1.873455] systemd[1]: Successfully made /usr/ read-only.1624client # [ 1.885659] Demotion targets for Node 0: null1625client # [ 1.885859] Key type .fscrypt registered1626lb # [ 1.514512] Demotion targets for Node 0: null1627client # [ 1.885872] Key type fscrypt-provisioning registered1628lb # [ 1.514705] Key type .fscrypt registered1629lb # [ 1.514713] Key type fscrypt-provisioning registered1630client # [ 1.886008] ima: No TPM chip found, activating TPM-bypass!1631lb # [ 1.514834] ima: No TPM chip found, activating TPM-bypass!1632lb # [ 1.514857] ima: Allocated hash algorithm: sha11633client # [ 1.886029] ima: Allocated hash algorithm: sha11634lb # [ 1.514882] ima: No architecture policies found1635client # [ 1.886059] ima: No architecture policies found1636lb # [ 1.519319] input: gpio-keys as /devices/platform/gpio-keys/input/input01637worker2 # [ 1.849949] usb 1-1: new high-speed USB device number 2 using ehci-pci1638client # [ 1.894111] input: gpio-keys as /devices/platform/gpio-keys/input/input01639lb # [ 1.542181] clk: Disabling unused clocks1640lb # [ 1.542233] PM: genpd: Disabling unused power domains1641client # [ 1.923053] clk: Disabling unused clocks1642lb # [ 1.552355] Freeing unused kernel memory: 4736K1643lb # [ 1.552588] Run /init as init process1644client # [ 1.923090] PM: genpd: Disabling unused power domains1645client # [ 1.927921] Freeing unused kernel memory: 4736K1646client # [ 1.928220] Run /init as init process1647lb # [ 1.572533] systemd[1]: Successfully made /usr/ read-only.1648worker1 # [ 1.982626] usb 1-1: new high-speed USB device number 2 using ehci-pci1649client # [ 1.995521] systemd[1]: Successfully made /usr/ read-only.1650worker2 # [ 2.004028] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11651client # [ 2.059063] usb 1-1: new high-speed USB device number 2 using ehci-pci1652lb # [ 1.700753] usb 1-1: new high-speed USB device number 2 using ehci-pci1653worker2 # [ 2.074138] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01654worker1 # [ 2.134571] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11655worker2 # [ 2.135950] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1656worker2 # [ 2.140138] systemd[1]: Detected virtualization qemu.1657worker2 # [ 2.140785] systemd[1]: Detected architecture arm64.1658worker2 # [ 2.141492] systemd[1]: Running in initrd.1659worker2 # [ 2.142688] systemd[1]: Initializing machine ID from random generator.1660worker2 # [ 2.143719] systemd[1]: Hostname set to <worker2>.1661client # [ 2.218543] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11662worker1 # [ 2.208620] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1663worker1 # [ 2.213847] systemd[1]: Detected virtualization qemu.1664worker1 # [ 2.214843] systemd[1]: Detected architecture arm64.1665worker1 # [ 2.215460] systemd[1]: Running in initrd.1666worker1 # [ 2.216484] systemd[1]: Initializing machine ID from random generator.1667worker1 # [ 2.217352] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01668worker1 # [ 2.219475] systemd[1]: Hostname set to <worker1>.1669lb # [ 1.867168] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11670worker2 # [ 2.197927] usb 1-2: new high-speed USB device number 3 using ehci-pci1671lb # [ 1.908236] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1672lb # [ 1.913629] systemd[1]: Detected virtualization qemu.1673client # [ 2.287131] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01674lb # [ 1.914673] systemd[1]: Detected architecture arm64.1675lb # [ 1.915451] systemd[1]: Running in initrd.1676lb # [ 1.916939] systemd[1]: Initializing machine ID from random generator.1677lb # [ 1.918374] systemd[1]: Hostname set to <lb>.1678client # [ 2.331132] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1679client # [ 2.338181] systemd[1]: Detected virtualization qemu.1680client # [ 2.338941] systemd[1]: Detected architecture arm64.1681client # [ 2.339712] systemd[1]: Running in initrd.1682lb # [ 1.965065] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683client # [ 2.340886] systemd[1]: Initializing machine ID from random generator.1684client # [ 2.341982] systemd[1]: Hostname set to <client>.1685worker1 # [ 2.338588] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker2 # [ 2.348069] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21687worker2 # [ 2.350099] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01688client # [ 2.410938] usb 1-2: new high-speed USB device number 3 using ehci-pci1689lb # [ 2.084752] usb 1-2: new high-speed USB device number 3 using ehci-pci1690worker2 # [ 2.429387] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691worker1 # [ 2.487762] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker1 # [ 2.497648] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker1 # [ 2.499777] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694worker2 # [ 2.502506] systemd[1]: Queued start job for default target Initrd Default Target.1695worker2 # [ 2.512639] systemd[1]: Created slice Slice /system/modprobe.1696worker2 # [ 2.513686] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1697worker2 # [ 2.514868] systemd[1]: Expecting device /dev/disk/by-label/nixos...1698worker2 # [ 2.515758] systemd[1]: Reached target Path Units.1699worker2 # [ 2.516417] systemd[1]: Reached target Slice Units.1700worker2 # [ 2.517067] systemd[1]: Reached target Swaps.1701worker2 # [ 2.517649] systemd[1]: Reached target Timer Units.1702worker2 # [ 2.518602] systemd[1]: Listening on D-Bus System Message Bus Socket.1703worker2 # [ 2.519635] systemd[1]: Listening on Journal Socket (/dev/log).1704worker2 # [ 2.520545] systemd[1]: Listening on Journal Sockets.1705worker2 # [ 2.521345] systemd[1]: Listening on udev Control Socket.1706worker2 # [ 2.522165] systemd[1]: Listening on udev Kernel Socket.1707worker2 # [ 2.522905] systemd[1]: Reached target Socket Units.1708worker2 # [ 2.525158] systemd[1]: Starting Create List of Static Device Nodes...1709worker1 # [ 2.560028] systemd[1]: Queued start job for default target Initrd Default Target.1710client # [ 2.568417] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21711client # [ 2.571609] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01712worker1 # [ 2.568736] systemd[1]: Created slice Slice /system/modprobe.1713worker1 # [ 2.569856] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1714worker1 # [ 2.571071] systemd[1]: Expecting device /dev/disk/by-label/nixos...1715worker1 # [ 2.571920] systemd[1]: Reached target Path Units.1716worker2 # [ 2.534239] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1717worker1 # [ 2.572575] systemd[1]: Reached target Slice Units.1718worker2 # [ 2.534388] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1719worker1 # [ 2.573169] systemd[1]: Reached target Swaps.1720worker1 # [ 2.573726] systemd[1]: Reached target Timer Units.1721worker1 # [ 2.574610] systemd[1]: Listening on D-Bus System Message Bus Socket.1722worker1 # [ 2.575597] systemd[1]: Listening on Journal Socket (/dev/log).1723worker1 # [ 2.576529] systemd[1]: Listening on Journal Sockets.1724worker1 # [ 2.577327] systemd[1]: Listening on udev Control Socket.1725worker2 # [ 2.542254] systemd[1]: Mounting Kernel Configuration File System...1726worker1 # [ 2.578192] systemd[1]: Listening on udev Kernel Socket.1727worker1 # [ 2.578941] systemd[1]: Reached target Socket Units.1728worker1 # [ 2.581167] systemd[1]: Starting Create List of Static Device Nodes...1729worker1 # [ 2.588026] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1730lb # [ 2.222096] systemd[1]: bpf-restrict-fs: LSM BPF program attached1731worker1 # [ 2.589019] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1732client # [ 2.606583] systemd[1]: bpf-restrict-fs: LSM BPF program attached1733worker2 # [ 2.562486] systemd[1]: Starting Journal Service...1734worker1 # [ 2.598760] systemd[1]: Mounting Kernel Configuration File System...1735worker1 # [ 2.608794] systemd[1]: Starting Journal Service...1736worker2 # [ 2.582300] systemd[1]: Starting Load Kernel Modules...1737worker2 # [ 2.583232] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1738lb # [ 2.254109] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21739lb # [ 2.258191] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01740worker2 # [ 2.590246] systemd[1]: Starting Coldplug All udev Devices...1741worker1 # [ 2.635938] systemd[1]: Starting Load Kernel Modules...1742worker1 # [ 2.636946] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1743worker2 # [ 2.610404] systemd[1]: Finished Create List of Static Device Nodes.1744worker2 # [ 2.612320] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1745worker1 # [ 2.648143] systemd[1]: Starting Coldplug All udev Devices...1746worker2 # [ 2.626494] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1747client # [ 2.672747] systemd[1]: Queued start job for default target Initrd Default Target.1748worker2 # [ 2.626997] systemd[1]: Mounted Kernel Configuration File System.1749worker2 # [ 2.630635] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1750client # [ 2.681206] systemd[1]: Created slice Slice /system/modprobe.1751client # [ 2.682283] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1752client # [ 2.683491] systemd[1]: Expecting device /dev/disk/by-label/nixos...1753worker1 # [ 2.671681] systemd[1]: Finished Create List of Static Device Nodes.1754client # [ 2.684348] systemd[1]: Reached target Path Units.1755worker1 # [ 2.672862] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1756worker1 # [ 2.673197] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1757client # [ 2.684998] systemd[1]: Reached target Slice Units.1758client # [ 2.685637] systemd[1]: Reached target Swaps.1759client # [ 2.686185] systemd[1]: Reached target Timer Units.1760worker1 # [ 2.673465] systemd[1]: Mounted Kernel Configuration File System.1761client # [ 2.687110] systemd[1]: Listening on D-Bus System Message Bus Socket.1762client # [ 2.688136] systemd[1]: Listening on Journal Socket (/dev/log).1763client # [ 2.689063] systemd[1]: Listening on Journal Sockets.1764client # [ 2.689877] systemd[1]: Listening on udev Control Socket.1765client # [ 2.691244] systemd[1]: Listening on udev Kernel Socket.1766client # [ 2.691971] systemd[1]: Reached target Socket Units.1767worker1 # [ 2.683914] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1768client # [ 2.694290] systemd[1]: Starting Create List of Static Device Nodes...1769worker1 # [ 2.690002] systemd-journald[73]: Collecting audit messages is disabled.1770worker2 # [ 2.656903] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1771client # [ 2.703222] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1772client # [ 2.704216] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1773worker2 # [ 2.664271] systemd-journald[74]: Collecting audit messages is disabled.1774client # [ 2.709893] systemd[1]: Mounting Kernel Configuration File System...1775lb # [ 2.347332] systemd[1]: Queued start job for default target Initrd Default Target.1776worker1 # [ 2.712149] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1777worker2 # [ 2.680455] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1778client # [ 2.731884] systemd[1]: Starting Journal Service...1779lb # [ 2.361960] systemd[1]: Created slice Slice /system/modprobe.1780lb # [ 2.363389] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1781lb # [ 2.364891] systemd[1]: Expecting device /dev/disk/by-label/nixos...1782lb # [ 2.366093] systemd[1]: Reached target Path Units.1783lb # [ 2.366955] systemd[1]: Reached target Slice Units.1784worker1 # [ 2.730671] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1785lb # [ 2.367833] systemd[1]: Reached target Swaps.1786worker2 # [ 2.694592] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01787lb # [ 2.368803] systemd[1]: Reached target Timer Units.1788worker2 # [ 2.694894] [drm] features: -virgl +edid -resource_blob -host_visible1789client # [ 2.743224] systemd[1]: Starting Load Kernel Modules...1790worker2 # [ 2.694907] [drm] features: -context_init1791worker2 # [ 2.695697] [drm] number of scanouts: 11792lb # [ 2.370002] systemd[1]: Listening on D-Bus System Message Bus Socket.1793worker2 # [ 2.695716] [drm] number of cap sets: 01794client # [ 2.744179] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1795lb # [ 2.371353] systemd[1]: Listening on Journal Socket (/dev/log).1796lb # [ 2.372585] systemd[1]: Listening on Journal Sockets.1797lb # [ 2.373814] systemd[1]: Listening on udev Control Socket.1798lb # [ 2.373989] systemd[1]: Listening on udev Kernel Socket.1799lb # [ 2.374045] systemd[1]: Reached target Socket Units.1800lb # [ 2.379111] systemd[1]: Starting Create List of Static Device Nodes...1801worker2 # [ 2.710746] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1802worker1 # [ 2.745498] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01803worker1 # [ 2.745775] [drm] features: -virgl +edid -resource_blob -host_visible1804worker1 # [ 2.745787] [drm] features: -context_init1805client # [ 2.759343] systemd[1]: Starting Coldplug All udev Devices...1806lb # [ 2.387039] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1807lb # [ 2.388239] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1808worker2 # [ 2.718350] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1809worker2 # [ 2.718388] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01810worker1 # [ 2.762685] [drm] number of scanouts: 11811lb # [ 2.398429] systemd[1]: Mounting Kernel Configuration File System...1812worker1 # [ 2.762723] [drm] number of cap sets: 01813worker2 # [ 2.728698] systemd[1]: Starting Create Static Device Nodes in /dev...1814worker1 # [ 2.764396] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1815worker1 # [ 2.764417] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01816worker1 # [ 2.783436] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1817client # [ 2.794846] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1818client # [ 2.797187] systemd[1]: Finished Create List of Static Device Nodes.1819client # [ 2.798638] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1820worker1 # [ 2.791099] systemd[1]: Starting Create Static Device Nodes in /dev...1821lb # [ 2.429178] systemd[1]: Starting Journal Service...1822client # [ 2.803904] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1823client # [ 2.804663] systemd[1]: Mounted Kernel Configuration File System.1824worker2 # [ 2.758348] Console: switching to colour frame buffer device 160x501825lb # [ 2.443299] systemd[1]: Starting Load Kernel Modules...1826client # [ 2.811472] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827worker1 # [ 2.798973] Console: switching to colour frame buffer device 160x501828lb # [ 2.446722] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1829client # [ 2.823082] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1830lb # [ 2.452874] systemd[1]: Starting Coldplug All udev Devices...1831client # [ 2.831647] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01832client # [ 2.831900] [drm] features: -virgl +edid -resource_blob -host_visible1833client # [ 2.831909] [drm] features: -context_init1834client # [ 2.832674] [drm] number of scanouts: 11835client # [ 2.832691] [drm] number of cap sets: 01836worker2 # [ 2.791423] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1837client # [ 2.844610] systemd-journald[73]: Collecting audit messages is disabled.1838worker2 # [ 2.798737] systemd[1]: Finished Create Static Device Nodes in /dev.1839worker2 # [ 2.799242] systemd[1]: Reached target Preparation for Local File Systems.1840worker2 # [ 2.799293] systemd[1]: Reached target Local File Systems.1841worker1 # [ 2.839278] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1842worker2 # [ 2.807377] systemd[1]: Starting Rule-based Manager for Device Events and Files...1843lb # [ 2.480957] systemd[1]: Finished Create List of Static Device Nodes.1844lb # [ 2.482260] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1845worker1 # [ 2.848209] systemd[1]: Finished Load Kernel Modules.1846worker1 # [ 2.851988] systemd[1]: Starting Apply Kernel Variables...1847client # [ 2.859413] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1848client # [ 2.859447] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01849worker2 # [ 2.822605] systemd[1]: Finished Load Kernel Modules.1850lb # [ 2.497929] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1851lb # [ 2.498521] systemd[1]: Mounted Kernel Configuration File System.1852lb # [ 2.505326] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1853worker2 # [ 2.834523] systemd[1]: Starting Apply Kernel Variables...1854lb # [ 2.516240] systemd-journald[73]: Collecting audit messages is disabled.1855client # [ 2.891286] Console: switching to colour frame buffer device 160x501856client # [ 2.901915] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1857worker1 # [ 2.895043] systemd[1]: Finished Create Static Device Nodes in /dev.1858worker1 # [ 2.895586] systemd[1]: Reached target Preparation for Local File Systems.1859worker1 # [ 2.895631] systemd[1]: Reached target Local File Systems.1860worker1 # [ 2.846793] systemd-modules-load[74]: Inserted module 'dm_mod'1861client # [ 2.911282] systemd[1]: Starting Create Static Device Nodes in /dev...1862worker1 # [ 2.852612] systemd-modules-load[74]: Module 'virtio_balloon' is built in1863worker1 # [ 2.853789] systemd-modules-load[74]: Module 'virtio_console' is built in1864worker1 # [ 2.854841] systemd-modules-load[74]: Inserted module 'virtio_gpu'1865worker1 # [ 2.855895] systemd-modules-load[74]: Module 'virtio_rng' is built in1866worker1 # [ 2.915071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1867worker1 # [ 2.915377] systemd[1]: Started Journal Service.1868client # [ 2.927743] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1869worker2 # [ 2.886197] systemd[1]: Started Journal Service.1870worker1 # [ 2.876127] systemd[1]: Finished Apply Kernel Variables.1871worker2 # [ 2.856592] systemd-modules-load[75]: Inserted module 'dm_mod'1872worker2 # [ 2.859539] systemd-modules-load[75]: Module 'virtio_balloon' is built in1873client # [ 2.943945] systemd[1]: Finished Load Kernel Modules.1874client # [ 2.947844] systemd[1]: Starting Apply Kernel Variables...1875worker2 # [ 2.864679] systemd-modules-load[75]: Module 'virtio_console' is built in1876worker2 # [ 2.865990] systemd-modules-load[75]: Inserted module 'virtio_gpu'1877worker2 # [ 2.867064] systemd-modules-load[75]: Module 'virtio_rng' is built in1878lb # [ 2.585428] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1879worker2 # [ 2.877854] systemd[1]: Starting Create System Files and Directories...1880worker1 # [ 2.904615] systemd[1]: Starting Create System Files and Directories...1881lb # [ 2.593549] systemd[1]: Starting Create Static Device Nodes in /dev...1882worker2 # [ 2.888617] systemd[1]: Finished Apply Kernel Variables.1883lb # [ 2.606248] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1884client # [ 2.989122] systemd[1]: Finished Create Static Device Nodes in /dev.1885client # [ 2.989439] systemd[1]: Reached target Preparation for Local File Systems.1886client # [ 2.989481] systemd[1]: Reached target Local File Systems.1887lb # [ 2.620891] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1888lb # [ 2.626217] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01889worker2 # [ 2.917798] systemd-udevd[80]: Using default interface naming scheme 'v261'.1890lb # [ 2.626479] [drm] features: -virgl +edid -resource_blob -host_visible1891lb # [ 2.626492] [drm] features: -context_init1892lb # [ 2.627238] [drm] number of scanouts: 11893lb # [ 2.627260] [drm] number of cap sets: 01894client # [ 3.007343] systemd[1]: Starting Rule-based Manager for Device Events and Files...1895worker1 # [ 2.950076] systemd[1]: Finished Create System Files and Directories.1896worker2 # [ 2.929940] systemd[1]: Finished Create System Files and Directories.1897worker1 # [ 2.960873] systemd-udevd[81]: Using default interface naming scheme 'v261'.1898lb # [ 2.653211] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1899lb # [ 2.653247] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01900client # [ 3.039562] systemd[1]: Finished Apply Kernel Variables.1901worker2 # [ 2.956454] systemd[1]: Started Rule-based Manager for Device Events and Files.1902lb # [ 2.675533] systemd[1]: Finished Create Static Device Nodes in /dev.1903client # [ 3.046128] systemd[1]: Started Journal Service.1904lb # [ 2.676026] systemd[1]: Reached target Preparation for Local File Systems.1905lb # [ 2.676061] systemd[1]: Reached target Local File Systems.1906worker1 # [ 2.993569] systemd[1]: Started Rule-based Manager for Device Events and Files.1907client # [ 2.988628] systemd-modules-load[74]: Inserted module 'dm_mod'1908lb # [ 2.689239] systemd[1]: Starting Rule-based Manager for Device Events and Files...1909client # [ 2.996332] systemd-modules-load[74]: Module 'virtio_balloon' is built in1910client # [ 2.998702] systemd-modules-load[74]: Module 'virtio_console' is built in1911client # [ 3.005721] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912lb # [ 2.694178] Console: switching to colour frame buffer device 160x501913client # [ 3.016635] systemd-modules-load[74]: Module 'virtio_rng' is built in1914client # [ 3.019363] systemd[1]: Starting Create System Files and Directories...1915worker2 # [ 3.018381] systemd[1]: Starting Virtual Console Setup...1916lb # [ 2.733093] systemd[1]: Started Journal Service.1917lb # [ 2.703112] systemd-modules-load[74]: Inserted module 'dm_mod'1918lb # [ 2.741554] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1919client # [ 3.046384] systemd[1]: Finished Create System Files and Directories.1920lb # [ 2.712137] systemd-modules-load[74]: Module 'virtio_balloon' is built in1921client # [ 3.058061] systemd-udevd[81]: Using default interface naming scheme 'v261'.1922lb # [ 2.724991] systemd-modules-load[74]: Module 'virtio_console' is built in1923worker1 # [ 3.068291] systemd[1]: Starting Virtual Console Setup...1924lb # [ 2.728487] systemd[1]: Starting Create System Files and Directories...1925lb # [ 2.732365] systemd-modules-load[74]: Inserted module 'virtio_gpu'1926lb # [ 2.740372] systemd-modules-load[74]: Module 'virtio_rng' is built in1927lb # [ 2.746130] systemd[1]: Finished Load Kernel Modules.1928client # [ 3.091763] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker2 # [ 3.088708] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1930worker2 # [ 3.092814] systemd[1]: Finished Virtual Console Setup.1931worker1 # [ 3.120623] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1932worker1 # [ 3.124583] systemd[1]: Finished Virtual Console Setup.1933lb # [ 2.785902] systemd-udevd[80]: Using default interface naming scheme 'v261'.1934lb # [ 2.800336] systemd[1]: Starting Apply Kernel Variables...1935lb # [ 2.801622] systemd[1]: Finished Create System Files and Directories.1936client # [ 3.156167] systemd[1]: Starting Virtual Console Setup...1937lb # [ 2.840437] systemd[1]: Finished Apply Kernel Variables.1938lb # [ 2.845256] systemd[1]: Started Rule-based Manager for Device Events and Files.1939client # [ 3.220695] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940client # [ 3.224818] systemd[1]: Finished Virtual Console Setup.1941lb # [ 2.915109] systemd[1]: Starting Virtual Console Setup...1942lb # [ 2.978338] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943lb # [ 2.980520] systemd[1]: Finished Virtual Console Setup.1944worker2 # [ 3.720632] systemd[1]: Finished Coldplug All udev Devices.1945worker2 # [ 3.721711] systemd[1]: Reached target System Initialization.1946worker2 # [ 3.722593] systemd[1]: Reached target Basic System.1947worker1 # [ 3.762352] systemd[1]: Finished Coldplug All udev Devices.1948worker1 # [ 3.765927] systemd[1]: Reached target System Initialization.1949worker1 # [ 3.768174] systemd[1]: Reached target Basic System.1950client # [ 3.895366] systemd[1]: Finished Coldplug All udev Devices.1951client # [ 3.896440] systemd[1]: Reached target System Initialization.1952client # [ 3.897279] systemd[1]: Reached target Basic System.1953worker2 # [ 3.895818] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1954worker1 # [ 3.927757] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1955lb # [ 3.599925] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 3.601029] systemd[1]: Reached target System Initialization.1957lb # [ 3.601878] systemd[1]: Reached target Basic System.1958worker2 # [ 3.927149] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1959worker1 # [ 3.967041] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker2 # [ 3.945221] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1961worker1 # [ 3.975189] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1962worker2 # [ 4.010120] systemd[1]: Found device /dev/disk/by-label/nixos.1963worker2 # [ 4.011804] systemd[1]: Reached target Initrd Root Device.1964worker2 # [ 4.016267] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1965worker1 # [ 4.051145] systemd[1]: Found device /dev/disk/by-label/nixos.1966worker1 # [ 4.056863] systemd[1]: Reached target Initrd Root Device.1967worker1 # [ 4.064723] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968client # [ 4.070782] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 4.082990] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970client # [ 4.088221] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1971worker2 # [ 4.078691] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1972worker2 # [ 4.088387] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1973worker2 # [ 4.094589] systemd[1]: Mounting /sysroot...1974lb # [ 3.777584] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 4.124231] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1976worker1 # [ 4.132177] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1977worker1 # [ 4.144181] systemd[1]: Mounting /sysroot...1978lb # [ 3.815790] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1979worker2 # [ 4.179653] EXT4-fs (vda): mounted filesystem e5b31632-f9ca-480f-bb97-673908c74cd4 r/w with ordered data mode. Quota mode: none.1980lb # [ 3.821194] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1981worker2 # [ 4.146560] systemd[1]: Mounted /sysroot.1982worker2 # [ 4.149180] systemd[1]: Reached target Initrd Root File System.1983worker2 # [ 4.160177] systemd[1]: Starting Mountpoints Configured in the Real Root...1984worker1 # [ 4.240030] EXT4-fs (vda): mounted filesystem e713126a-ace6-4981-a9e2-c0c326cef6e9 r/w with ordered data mode. Quota mode: none.1985worker1 # [ 4.194316] systemd[1]: Mounted /sysroot.1986worker1 # [ 4.196519] systemd[1]: Reached target Initrd Root File System.1987client # [ 4.191971] systemd[1]: Found device /dev/disk/by-label/nixos.1988client # [ 4.198770] systemd[1]: Reached target Initrd Root Device.1989client # [ 4.203425] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1990worker2 # [ 4.188874] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1991worker1 # [ 4.215609] systemd[1]: Starting Mountpoints Configured in the Real Root...1992lb # [ 3.873934] systemd[1]: Found device /dev/disk/by-label/nixos.1993worker2 # [ 4.195706] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1994lb # [ 3.876781] systemd[1]: Reached target Initrd Root Device.1995lb # [ 3.880750] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1996worker2 # [ 4.202274] systemd[1]: Reloading...1997worker1 # [ 4.251465] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1998worker1 # [ 4.256252] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1999worker1 # [ 4.262189] systemd[1]: Reloading...2000client # [ 4.260199] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 1,601 hidden lines…
3602worker1 # [ 27.949808] nix-grpc-daemon[786]: ts=2026-09-20T12:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3603worker2 # [ 27.932481] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3604lb # [ 27.611684] envoy[909]: [2026-09-20 12:46:33.686][909][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3605lb # [ 27.618900] envoy[909]: [2026-09-20 12:46:33.686][909][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3606worker2 # [ 27.945162] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3607worker2 # [ 27.947476] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13608lb # [ 27.624648] envoy[909]: [2026-09-20 12:46:33.686][909][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3609lb # [ 27.630001] envoy[909]: [2026-09-20 12:46:33.686][909][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3610worker2 # [ 28.000330] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3611worker2 # [ 28.006512] systemd[1]: Started Nix worker-protocol over gRPC.3612worker2 # [ 28.045320] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 28.050476] nix-grpc-daemon[776]: ts=2026-09-20T12:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker1 # [ 28.151082] nix-grpc-daemon[786]: ts=2026-09-20T12:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3615worker2 # [ 28.252539] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_connected addr=lb:500513616worker2 # [ 28.258139] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker2 # [ 28.459621] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_connected addr=lb:500513618worker2 # [ 28.465274] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker1 # [ 28.551799] nix-grpc-daemon[786]: ts=2026-09-20T12:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3620worker2 # [ 28.666572] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_connected addr=lb:500513621worker2 # [ 28.672171] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3622worker2 # [ 28.873597] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 28.879340] nix-grpc-daemon[776]: ts=2026-09-20T12:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker2 # [ 29.082422] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513625worker2 # [ 29.086853] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker2 # [ 29.283507] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513627worker2 # [ 29.289326] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 29.355537] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513629worker1 # [ 29.355947] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker2 # [ 29.490806] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 29.496418] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker1 # [ 29.556955] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 29.562456] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker2 # [ 29.697795] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 29.703606] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 29.763647] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 29.769303] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 29.905339] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 29.910841] nix-grpc-daemon[776]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 29.969934] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 29.975687] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 30.114249] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 30.118781] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 30.176587] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 30.181827] nix-grpc-daemon[786]: ts=2026-09-20T12:46:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 30.314948] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 30.317491] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 30.383031] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 30.387645] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 30.519016] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 30.524760] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 30.584316] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 30.590629] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 30.726292] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 30.732255] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 30.791533] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 30.795747] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 30.933357] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 30.938368] nix-grpc-daemon[776]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 30.996549] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 31.000223] nix-grpc-daemon[786]: ts=2026-09-20T12:46:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 31.141648] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 31.146223] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 31.201592] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 31.207406] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 31.343081] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 31.348978] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 31.409408] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 31.411592] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 31.550306] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 31.556186] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 31.610220] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 31.614145] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 31.757393] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 31.762104] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 31.815428] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 31.821164] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 31.963732] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 31.969552] nix-grpc-daemon[776]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 32.022023] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 32.027601] nix-grpc-daemon[786]: ts=2026-09-20T12:46:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 32.172754] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 32.177028] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 32.228793] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 32.234319] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 32.373299] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 32.376911] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 32.435423] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 32.437588] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 32.578208] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 32.581986] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 32.638918] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 32.644673] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 32.783269] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 32.786964] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 32.845573] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 32.851334] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 32.989007] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 32.995369] nix-grpc-daemon[776]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 33.052459] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 33.058076] nix-grpc-daemon[786]: ts=2026-09-20T12:46:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 33.198636] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 33.203455] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 33.258982] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 33.264683] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 33.399072] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 33.401906] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 33.467892] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 33.472429] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 33.603002] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 33.606586] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 33.668984] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 33.674343] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 33.808183] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 33.813804] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 33.875597] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 33.881420] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 34.015184] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 34.021367] nix-grpc-daemon[776]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 34.082388] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 34.087929] nix-grpc-daemon[786]: ts=2026-09-20T12:46:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 34.224981] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 34.229311] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 34.289265] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 34.294698] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 34.426301] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 34.432115] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 34.497793] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 34.502287] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 34.633405] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 34.638949] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 34.698448] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 34.704109] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 34.840321] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 34.843542] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 34.904997] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 34.910452] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 35.044801] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 35.050480] nix-grpc-daemon[776]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 35.111421] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 35.116873] nix-grpc-daemon[786]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 35.252261] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 35.255350] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 35.317850] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 35.323292] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 35.457075] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 35.462585] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 35.526497] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 35.530815] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 35.663901] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 35.669266] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 35.726983] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 35.732459] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 35.870222] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 35.875476] nix-grpc-daemon[776]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 35.933430] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 35.939180] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 36.076106] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 36.078275] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 36.140102] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 36.145437] nix-grpc-daemon[786]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 36.281911] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 36.286278] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 36.346749] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 36.352442] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 36.483418] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 36.489229] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 36.555235] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 36.559253] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 36.690549] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 36.696148] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 36.755906] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 36.759105] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 36.898060] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 36.909209] nix-grpc-daemon[776]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 36.959664] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 36.961874] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 37.106890] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 37.110925] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 37.163130] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 37.168549] nix-grpc-daemon[786]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 37.314364] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 37.318769] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 37.369536] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513785worker1 # [ 37.375255] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker2 # [ 37.516004] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 37.521721] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788worker1 # [ 37.578713] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513789worker1 # [ 37.583056] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790worker2 # [ 37.723328] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513791worker2 # [ 37.729095] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792worker1 # [ 37.779516] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513793worker1 # [ 37.785423] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3794worker2 # [ 37.930467] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513795worker2 # [ 37.936105] nix-grpc-daemon[776]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3796worker1 # [ 37.986227] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500513797worker1 # [ 37.991853] nix-grpc-daemon[786]: ts=2026-09-20T12:46:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3798worker2 # [ 38.136874] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513799worker2 # [ 38.139875] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3800worker1 # [ 38.192804] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513801worker1 # [ 38.196849] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3802worker2 # [ 38.343659] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 38.348294] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker1 # [ 38.398185] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 38.404366] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker2 # [ 38.544579] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 38.547821] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808worker1 # [ 38.607580] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 38.611992] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 38.749617] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 38.755488] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812worker1 # [ 38.808739] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513813worker1 # [ 38.814763] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814worker2 # [ 38.956933] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513815worker2 # [ 38.962460] nix-grpc-daemon[776]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3816worker1 # [ 39.015822] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_connected addr=lb:500513817worker1 # [ 39.021548] nix-grpc-daemon[786]: ts=2026-09-20T12:46:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3818worker2 # [ 39.163244] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513819worker2 # [ 39.166089] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3820worker1 # [ 39.222797] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513821worker1 # [ 39.228364] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3822worker2 # [ 39.369222] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513823worker2 # [ 39.374197] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3824worker1 # [ 39.429339] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513825worker1 # [ 39.434833] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3826worker2 # [ 39.575728] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513827worker2 # [ 39.581364] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3828worker1 # [ 39.638102] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513829worker1 # [ 39.642438] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3830worker2 # [ 39.782490] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513831worker2 # [ 39.788116] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3832worker1 # [ 39.838702] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513833worker1 # [ 39.844660] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3834worker2 # [ 39.989416] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513835worker2 # [ 39.994986] nix-grpc-daemon[776]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3836worker1 # [ 40.045610] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_connected addr=lb:500513837worker1 # [ 40.051109] nix-grpc-daemon[786]: ts=2026-09-20T12:46:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3838worker2 # [ 40.196824] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513839worker2 # [ 40.203679] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3840worker1 # [ 40.251837] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513841worker1 # [ 40.256339] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842worker2 # [ 40.404360] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513843worker2 # [ 40.407447] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3844worker1 # [ 40.456942] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513845worker1 # [ 40.462326] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3846worker2 # [ 40.608823] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513847worker2 # [ 40.614350] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3848worker1 # [ 40.665596] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513849worker1 # [ 40.670043] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850worker2 # [ 40.815711] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513851worker2 # [ 40.821439] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3852worker1 # [ 40.866190] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513853worker1 # [ 40.872003] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3854worker2 # [ 41.022730] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513855worker2 # [ 41.028706] nix-grpc-daemon[776]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3856worker1 # [ 41.073140] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500513857worker1 # [ 41.078978] nix-grpc-daemon[786]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3858worker2 # [ 41.230029] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513859worker2 # [ 41.235845] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3860worker1 # [ 41.280349] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513861worker1 # [ 41.285991] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3862worker2 # [ 41.439395] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513863worker2 # [ 41.444158] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3864worker1 # [ 41.487146] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513865worker1 # [ 41.493423] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3866worker2 # [ 41.640741] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513867worker2 # [ 41.646674] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3868worker1 # [ 41.697339] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513869worker1 # [ 41.701865] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3870worker2 # [ 41.848134] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513871worker2 # [ 41.854101] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3872worker1 # [ 41.897357] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513873worker1 # [ 41.903463] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3874worker2 # [ 42.055051] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513875worker2 # [ 42.058996] nix-grpc-daemon[776]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3876worker1 # [ 42.104735] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500513877worker1 # [ 42.110716] nix-grpc-daemon[786]: ts=2026-09-20T12:46:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3878worker2 # [ 42.260588] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513879worker2 # [ 42.266517] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3880worker1 # [ 42.311630] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513881worker1 # [ 42.317531] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3882worker2 # [ 42.470384] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513883worker2 # [ 42.474969] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3884worker1 # [ 42.518802] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513885worker1 # [ 42.524837] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886worker2 # [ 42.671879] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513887worker2 # [ 42.677738] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3888worker1 # [ 42.728245] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513889worker1 # [ 42.732678] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3890worker2 # [ 42.879206] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513891worker2 # [ 42.885040] nix-grpc-daemon[776]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3892worker1 # [ 42.929754] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513893worker1 # [ 42.935703] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3894worker2 # [ 43.086305] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513895worker2 # [ 43.092242] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3896worker1 # [ 43.136827] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_connected addr=lb:500513897worker1 # [ 43.142490] nix-grpc-daemon[786]: ts=2026-09-20T12:46:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3898worker2 # [ 43.293698] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513899worker2 # [ 43.299389] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3900worker1 # [ 43.344089] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513901worker1 # [ 43.349602] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3902worker2 # [ 43.504655] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513903worker2 # [ 43.513390] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3904worker1 # [ 43.550742] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513905worker1 # [ 43.563318] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3906worker2 # [ 43.704386] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513907worker2 # [ 43.709602] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3908worker1 # [ 43.762190] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513909worker1 # [ 43.766512] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3910worker2 # [ 43.911136] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513911worker2 # [ 43.917174] nix-grpc-daemon[776]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3912worker1 # [ 43.962931] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513913worker1 # [ 43.968615] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3914worker2 # [ 44.118385] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513915worker2 # [ 44.124283] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3916worker1 # [ 44.169763] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_connected addr=lb:500513917worker1 # [ 44.175665] nix-grpc-daemon[786]: ts=2026-09-20T12:46:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3918worker2 # [ 44.326304] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513919worker2 # [ 44.332211] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3920worker1 # [ 44.376377] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513921worker1 # [ 44.380098] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3922worker2 # [ 44.533823] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513923worker2 # [ 44.536541] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3924worker1 # [ 44.581295] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513925worker1 # [ 44.587131] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3926worker2 # [ 44.738310] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513927worker2 # [ 44.744579] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3928worker1 # [ 44.790262] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513929worker1 # [ 44.794961] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3930worker1 # [ 44.956412] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3931worker2 # [ 44.945618] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513932worker2 # [ 44.951313] nix-grpc-daemon[776]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3933worker1 # [ 44.991195] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_connected addr=lb:500513934worker1 # [ 44.997171] nix-grpc-daemon[786]: ts=2026-09-20T12:46:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3935lb # [ 44.725067] rustfs-bucket-start[1084]: mb s3://farm3936lb # [ 44.734561] systemd[1]: Finished rustfs-bucket.service.3937lb # [ 44.743881] systemd[1]: Starting niks3 server...3938worker2 # [ 45.152667] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513939worker2 # [ 45.157554] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3940worker1 # [ 45.198481] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513941worker1 # [ 45.204507] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3942worker2 # [ 45.237347] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3943lb # [ 44.983638] postgres[1099]: [1099] ERROR: relation "goose_db_version" does not exist at character 363944lb # [ 44.985441] postgres[1099]: [1099] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3945lb # [ 45.019305] niks3-server[1094]: 2026/09/20 12:46:51 OK 20241026095416_initial_model.sql (20.22ms)3946lb # [ 45.022868] niks3-server[1094]: 2026/09/20 12:46:51 OK 20251210153512_drop_unused_gin_index.sql (3.96ms)3947lb # [ 45.028489] niks3-server[1094]: 2026/09/20 12:46:51 OK 20251218171726_add_pins.sql (5.58ms)3948lb # [ 45.033687] niks3-server[1094]: 2026/09/20 12:46:51 OK 20260628120000_add_object_size_and_stats.sql (5.14ms)3949lb # [ 45.039576] niks3-server[1094]: 2026/09/20 12:46:51 OK 20260905000000_add_claims.sql (5.84ms)3950worker2 # [ 45.358874] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513951lb # [ 45.044753] niks3-server[1094]: 2026/09/20 12:46:51 OK 20260920000000_drop_claims.sql (5.05ms)3952lb # [ 45.046155] niks3-server[1094]: 2026/09/20 12:46:51 goose: successfully migrated database to version: 202609200000003953worker2 # [ 45.365022] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3954lb # [ 45.051774] niks3-server[1094]: 2026/09/20 12:46:51 OK 1_commit_pending_closure.sql (6.98ms)3955lb # [ 45.055052] niks3-server[1094]: 2026/09/20 12:46:51 OK 2_object_stats_trigger.sql (3.13ms)3956lb # [ 45.056769] niks3-server[1094]: 2026/09/20 12:46:51 goose: up to current file version: 23957lb # [ 45.060924] niks3-server[1094]: 2026/09/20 12:46:51 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3958worker1 # [ 45.405506] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513959worker1 # [ 45.411504] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3960lb # [ 45.130502] niks3-server[1094]: 2026/09/20 12:46:51 INFO Read proxy enabled — serving cache objects from S33961lb # [ 45.136812] niks3-server[1094]: 2026/09/20 12:46:51 INFO Using socket-activated listener address=0.0.0.0:57513962lb # [ 45.143510] niks3-server[1094]: 2026/09/20 12:46:51 INFO systemd watchdog enabled interval=15s3963lb # [ 45.151358] niks3-server[1094]: 2026/09/20 12:46:51 INFO Starting HTTP server address=0.0.0.0:57513964lb # [ 45.159269] systemd[1]: Started niks3 server.3965lb # [ 45.165505] niks3-server[1094]: 2026/09/20 12:46:51 INFO lead: acquired remote=192.168.1.4:436483966lb # [ 45.166934] niks3-server[1094]: 2026/09/20 12:46:51 INFO lead: released remote=192.168.1.4:436483967lb # [ 45.172618] systemd[1]: Reached target Multi-User System.3968lb # [ 45.176715] niks3-server[1094]: 2026/09/20 12:46:51 INFO lead: acquired remote=192.168.1.3:329923969lb # [ 45.178427] niks3-server[1094]: 2026/09/20 12:46:51 INFO lead: released remote=192.168.1.3:329923970lb # [ 45.179635] systemd[1]: Startup finished in 1.526s (kernel) + 7.101s (initrd) + 36.526s (userspace) = 45.154s.3971worker2 # [ 45.566573] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513972worker2 # [ 45.570408] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3973worker1 # [ 45.612034] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513974worker1 # [ 45.617767] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3975worker2 # [ 45.771954] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513976worker2 # [ 45.778065] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3977worker1 # [ 45.821191] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513978worker1 # [ 45.827682] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3979lb # [ 45.634310] niks3-server[1094]: 2026/09/20 12:46:51 INFO lead: acquired remote=192.168.1.3:353103980worker1 # [ 45.979000] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_take_over3981worker2 # [ 45.978985] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513982worker2 # [ 45.981405] nix-grpc-daemon[776]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3983worker1 # [ 46.021761] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=scheduler_connected addr=lb:500513984worker1 # [ 46.046461] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:415443985worker1 # [ 46.057013] nix-grpc-daemon[786]: ts=2026-09-20T12:46:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03986worker2 # [ 46.182965] nix-grpc-daemon[776]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513987worker1 # [ 46.222873] nix-grpc-daemon[786]: ts=2026-09-20T12:46:52Z level=info event=oidc_jwks provider=mock keys=13988worker1 # [ 46.228183] nix-grpc-daemon[786]: ts=2026-09-20T12:46:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:415443989worker1 # [ 46.234316] nix-grpc-daemon[786]: ts=2026-09-20T12:46:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03990lb: (finished: waiting for unit niks3.service, in 24.98 seconds)3991lb: waiting for TCP port 5751 on localhost3992lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3993lb: (finished: waiting for TCP port 5751 on localhost, in 0.10 seconds)3994lb: waiting for unit envoy.service3995lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3996lb: waiting for TCP port 50051 on localhost3997lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3998lb # [ 46.472628] envoy[909]: conn peer=[::1]:36014 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33999lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)4000worker2 # [ 46.990273] systemd[1]: Starting scheduler-token.service...4001worker2 # [ 47.082891] systemd[1]: scheduler-token.service: Deactivated successfully.4002worker2 # [ 47.085295] systemd[1]: Finished scheduler-token.service.4003worker2 # [ 47.088379] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4004lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4005??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4006 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284007lb: must succeed: curl -sf localhost:9901/clusters4008lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4009lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4010subtest: one node schedules and both builders hold a WorkerSession on it4011lb: must succeed: curl -sf localhost:9901/clusters4012lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4013worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4014worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4015lb: must succeed: curl -sf localhost:9901/clusters4016lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4017lb: must succeed: curl -sf localhost:9901/clusters4018lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4019worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4020worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4021(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.33 seconds)4022subtest: balancer auth: client cert, bearer token, nothing, unknown CN4023worker2 # [ 47.908980] nix-grpc-daemon[776]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587704024worker2 # [ 47.928207] systemd[1]: Started Nix Daemon.4025worker2 # [ 48.250453] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4026worker2 # [ 48.290939] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)4027worker2 # [ 48.300770] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 2/5)4028worker2 # [ 48.391763] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)4029worker2 # [ 48.602496] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 463 ms (attempt 4/5)4030worker2 # [ 49.079289] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4031worker1 # [ 49.145346] nix-grpc-daemon[786]: ts=2026-09-20T12:46:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584032worker1 # [ 49.158972] systemd[1]: Started Nix Daemon.4033worker1 # [ 49.446194] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4034worker1 # [ 49.475546] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 1/5)4035worker1 # [ 49.534404] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)4036worker1 # [ 49.630115] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 201 ms (attempt 3/5)4037worker1 # [ 49.839434] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 715 ms (attempt 4/5)4038worker1 # [ 50.557556] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4039worker1 # [ 50.569237] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=path_info_miss path=vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test error="error: path '/nix/store/vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test' is required, but there is no substituter that can build it"4040worker1 # [ 50.582195] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584041worker1 # [ 50.596582] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=path_info_miss path=vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test error="error: path '/nix/store/vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test' is required, but there is no substituter that can build it"4042client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4043client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)4044client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4045client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4046worker1 # [ 51.071434] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:415584047worker1 # [ 51.089334] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:415584048worker1 # [ 51.103606] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=path_info_miss path=vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test error="error: path '/nix/store/vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test' is required, but there is no substituter that can build it"4049worker1 # [ 51.120370] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:415584050worker1 # [ 51.130207] nix-grpc-daemon[786]: ts=2026-09-20T12:46:56Z level=debug event=path_info_miss path=vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test error="error: path '/nix/store/vjqwk0w01cvrc7s0v48m3lmih9i5s8yp-nixos-system-client-test' is required, but there is no substituter that can build it"4051client: must succeed: rm -r /var/lib/nix-grpc-store4052client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4053worker1 # [ 51.443082] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=info event=denied method=QueryMissing cn=- role=none4054worker2 # [ 51.707511] nix-grpc-daemon[776]: ts=2026-09-20T12:46:57Z level=info event=denied method=QueryMissing cn=stranger role=none4055(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.20 seconds)4056subtest: DAG build is scheduled across workers and lands in the cache4057client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24058worker2 # [ 51.979430] nix-grpc-daemon[776]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587704059client # don't know how to build these paths:4060client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4061worker2 # [ 51.997449] nix-grpc-daemon[776]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704062worker1 # [ 52.060189] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04063worker1 # [ 52.066079] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14064worker1 # [ 52.109704] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584065worker1 # [ 52.131857] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584066client # copying 1 paths to 'lb:50051'...4067worker1 # [ 52.145364] nix-grpc-daemon[786]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584068client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4069worker1 # [ 52.171111] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4070worker1 # [ 52.191908] nix-grpc-daemon[786]: ts=2026-09-20T12:46:58Z level=info event=niks3_push_started pid=8684071worker1 # [ 52.498187] nix-grpc-daemon[868]: time=2026-09-20T12:46:58.309Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4072worker1 # [ 52.716256] nix-daemon[856]: accepted connection from pid 881, user nix-grpc-daemon (trusted)4073worker1 # [ 52.738448] nix-daemon[856]: reaped child process 884, status = succeeded4074worker1 # [ 52.952859] nix-daemon[856]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4075worker1 # [ 52.977847] nix-daemon[856]: reaped child process 889, status = succeeded4076lb # [ 52.651170] niks3-server[1094]: 2026/09/20 12:46:58 INFO Received uploads request method=POST path=/api/pending_closures4077worker1 # [ 53.020713] nix-grpc-daemon[868]: time=2026-09-20T12:46:58.834Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4078worker1 # [ 53.022816] nix-grpc-daemon[868]: time=2026-09-20T12:46:58.836Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4079lb # [ 52.757717] niks3-server[1094]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4080lb # [ 52.771974] niks3-server[1094]: 2026/09/20 12:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4081lb # [ 52.797657] niks3-server[1094]: 2026/09/20 12:46:58 INFO Signed narinfos id=1 count=14082worker1 # [ 53.148284] nix-grpc-daemon[868]: time=2026-09-20T12:46:58.961Z level=INFO msg="Uploading 1 narinfos"4083lb # [ 52.830160] niks3-server[1094]: 2026/09/20 12:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4084lb # [ 52.856564] niks3-server[1094]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4085lb # [ 52.887783] niks3-server[1094]: 2026/09/20 12:46:58 INFO Completed upload id=14086worker1 # [ 53.230912] nix-grpc-daemon[868]: time=2026-09-20T12:46:59.044Z level=INFO msg="Upload complete. (723ms)"4087worker1 # [ 53.242805] nix-grpc-daemon[786]: ts=2026-09-20T12:46:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 paths=1 nar_bytes_in=5044088worker1 # [ 53.256464] nix-grpc-daemon[786]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584089lb # [ 52.916384] niks3-server[1094]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4090worker1 # [ 53.272525] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4091worker1 # [ 53.275105] nix-daemon[856]: reaped child process 866, status = succeeded4092client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4093client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4094worker1 # [ 53.294225] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4095client # farm-a> LOG-farm-a-t14096worker1 # [ 54.721965] nix-daemon[856]: reaped child process 893, status = succeeded4097worker1 # [ 54.910496] nix-daemon[856]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4098worker1 # [ 54.933146] nix-daemon[856]: reaped child process 912, status = succeeded4099lb # [ 54.605456] niks3-server[1094]: 2026/09/20 12:47:00 INFO Received uploads request method=POST path=/api/pending_closures4100worker1 # [ 54.960729] nix-grpc-daemon[868]: time=2026-09-20T12:47:00.773Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4101worker1 # [ 54.963305] nix-grpc-daemon[868]: time=2026-09-20T12:47:00.775Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4102lb # [ 54.641089] niks3-server[1094]: 2026/09/20 12:47:00 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4103lb # [ 54.649859] niks3-server[1094]: 2026/09/20 12:47:00 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4104lb # [ 54.656355] niks3-server[1094]: 2026/09/20 12:47:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4105lb # [ 54.661740] niks3-server[1094]: 2026/09/20 12:47:00 INFO Signed narinfos id=2 count=14106worker1 # [ 55.004432] nix-grpc-daemon[868]: time=2026-09-20T12:47:00.816Z level=INFO msg="Uploading 1 narinfos"4107lb # [ 54.663702] niks3-server[1094]: 2026/09/20 12:47:00 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4108lb # [ 54.672132] niks3-server[1094]: 2026/09/20 12:47:00 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4109lb # [ 54.678020] niks3-server[1094]: 2026/09/20 12:47:00 INFO Completed upload id=24110worker1 # [ 55.020762] nix-grpc-daemon[868]: time=2026-09-20T12:47:00.834Z level=INFO msg="Upload complete. (312ms)"4111lb # [ 54.679976] niks3-server[1094]: 2026/09/20 12:47:00 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4112worker1 # [ 55.040429] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14113worker1 # [ 55.051547] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04114worker1 # [ 55.054105] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14115worker1 # [ 55.057161] nix-daemon[856]: reaped child process 891, status = succeeded4116worker1 # [ 55.088842] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584117worker1 # [ 55.104561] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584118worker1 # [ 55.120202] nix-grpc-daemon[868]: time=2026-09-20T12:47:00.933Z level=INFO msg="All 1 paths already cached"4119client # copying 1 paths to 'lb:50051'...4120worker1 # [ 55.133577] nix-grpc-daemon[786]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584121client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4122worker1 # [ 55.161195] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4123worker1 # [ 55.391193] nix-daemon[856]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4124worker1 # [ 55.417261] nix-daemon[856]: reaped child process 921, status = succeeded4125worker1 # [ 55.633619] nix-daemon[856]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4126worker1 # [ 55.656978] nix-daemon[856]: reaped child process 926, status = succeeded4127lb # [ 55.323169] niks3-server[1094]: 2026/09/20 12:47:01 INFO Received uploads request method=POST path=/api/pending_closures4128worker1 # [ 55.679888] nix-grpc-daemon[868]: time=2026-09-20T12:47:01.492Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4129worker1 # [ 55.681815] nix-grpc-daemon[868]: time=2026-09-20T12:47:01.492Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4130lb # [ 55.369612] niks3-server[1094]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4131lb # [ 55.387434] niks3-server[1094]: 2026/09/20 12:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4132worker1 # [ 55.740253] nix-grpc-daemon[868]: time=2026-09-20T12:47:01.553Z level=INFO msg="Uploading 1 narinfos"4133lb # [ 55.402143] niks3-server[1094]: 2026/09/20 12:47:01 INFO Signed narinfos id=3 count=14134lb # [ 55.408435] niks3-server[1094]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4135lb # [ 55.424391] niks3-server[1094]: 2026/09/20 12:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4136worker1 # [ 55.774477] nix-grpc-daemon[868]: time=2026-09-20T12:47:01.587Z level=INFO msg="Upload complete. (592ms)"4137lb # [ 55.431738] niks3-server[1094]: 2026/09/20 12:47:01 INFO Completed upload id=34138worker1 # [ 55.779456] nix-grpc-daemon[786]: ts=2026-09-20T12:47:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=6244139lb # [ 55.436810] niks3-server[1094]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4140worker1 # [ 55.785043] nix-grpc-daemon[786]: ts=2026-09-20T12:47:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584141worker1 # [ 55.798880] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4142worker1 # [ 55.801258] nix-daemon[856]: reaped child process 916, status = succeeded4143client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4144worker1 # [ 55.813628] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4145client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4146client # farm-b> LOG-farm-b-t14147worker1 # [ 56.639218] nix-daemon[856]: reaped child process 930, status = succeeded4148worker1 # [ 56.832646] nix-daemon[856]: accepted connection from pid 945, user nix-grpc-daemon (trusted)4149worker1 # [ 56.857318] nix-daemon[856]: reaped child process 948, status = succeeded4150lb # [ 56.530691] niks3-server[1094]: 2026/09/20 12:47:02 INFO Received uploads request method=POST path=/api/pending_closures4151worker1 # [ 56.888128] nix-grpc-daemon[868]: time=2026-09-20T12:47:02.700Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4152worker1 # [ 56.889687] nix-grpc-daemon[868]: time=2026-09-20T12:47:02.700Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4153lb # [ 56.599346] niks3-server[1094]: 2026/09/20 12:47:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4154worker1 # [ 56.949795] nix-grpc-daemon[868]: time=2026-09-20T12:47:02.762Z level=INFO msg="Uploading 1 narinfos"4155lb # [ 56.608545] niks3-server[1094]: 2026/09/20 12:47:02 INFO Signed narinfos id=4 count=14156lb # [ 56.624394] niks3-server[1094]: 2026/09/20 12:47:02 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4157lb # [ 56.690989] niks3-server[1094]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4158lb # [ 56.708840] niks3-server[1094]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4159lb # [ 56.715833] niks3-server[1094]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4160lb # [ 56.729023] niks3-server[1094]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4161worker1 # [ 57.083000] nix-grpc-daemon[868]: time=2026-09-20T12:47:02.896Z level=INFO msg="Upload complete. (459ms)"4162lb # [ 56.740248] niks3-server[1094]: 2026/09/20 12:47:02 INFO Completed upload id=44163worker1 # [ 57.092953] nix-grpc-daemon[786]: ts=2026-09-20T12:47:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14164worker1 # [ 57.105174] nix-grpc-daemon[786]: ts=2026-09-20T12:47:02Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04165worker1 # [ 57.109629] nix-grpc-daemon[786]: ts=2026-09-20T12:47:02Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14166worker1 # [ 57.116196] nix-daemon[856]: reaped child process 928, status = succeeded4167worker1 # [ 57.153221] nix-grpc-daemon[786]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584168worker1 # [ 57.173273] nix-grpc-daemon[786]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584169worker1 # [ 57.196957] nix-grpc-daemon[868]: time=2026-09-20T12:47:03.010Z level=INFO msg="All 2 paths already cached"4170client # copying 1 paths to 'lb:50051'...4171client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4172worker1 # [ 57.210623] nix-grpc-daemon[786]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584173worker1 # [ 57.232214] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4174worker1 # [ 57.463947] nix-daemon[856]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4175worker1 # [ 57.488639] nix-daemon[856]: reaped child process 957, status = succeeded4176lb # [ 57.244860] envoy[909]: conn peer=[2001:db8:1::1]:35874 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=27144177lb # [ 57.250544] envoy[909]: conn peer=[2001:db8:1::1]:45828 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=784178lb # [ 57.256884] envoy[909]: conn peer=[2001:db8:1::1]:45830 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114179lb # [ 57.262012] envoy[909]: conn peer=[2001:db8:1::1]:45842 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=174180worker1 # [ 57.712265] nix-daemon[856]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4181worker1 # [ 57.739785] nix-daemon[856]: reaped child process 962, status = succeeded4182lb # [ 57.403163] niks3-server[1094]: 2026/09/20 12:47:03 INFO Received uploads request method=POST path=/api/pending_closures4183worker1 # [ 57.760503] nix-grpc-daemon[868]: time=2026-09-20T12:47:03.573Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4184worker1 # [ 57.762181] nix-grpc-daemon[868]: time=2026-09-20T12:47:03.573Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4185lb # [ 57.458925] niks3-server[1094]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4186worker1 # [ 57.818107] nix-grpc-daemon[868]: time=2026-09-20T12:47:03.631Z level=INFO msg="Uploading 1 narinfos"4187lb # [ 57.479136] niks3-server[1094]: 2026/09/20 12:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4188lb # [ 57.489698] niks3-server[1094]: 2026/09/20 12:47:03 INFO Signed narinfos id=5 count=14189lb # [ 57.495191] niks3-server[1094]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4190lb # [ 57.506015] niks3-server[1094]: 2026/09/20 12:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4191lb # [ 57.514934] niks3-server[1094]: 2026/09/20 12:47:03 INFO Completed upload id=54192worker1 # [ 57.858155] nix-grpc-daemon[868]: time=2026-09-20T12:47:03.671Z level=INFO msg="Upload complete. (605ms)"4193lb # [ 57.518426] niks3-server[1094]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4194worker1 # [ 57.872381] nix-grpc-daemon[786]: ts=2026-09-20T12:47:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=7524195worker1 # [ 57.885848] nix-grpc-daemon[786]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584196worker1 # [ 57.901357] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4197worker1 # [ 57.912459] nix-daemon[856]: reaped child process 952, status = succeeded4198client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4199worker1 # [ 57.928716] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4200client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4201client # farm-top> LOG-farm-top-t14202worker1 # [ 59.162262] nix-daemon[856]: reaped child process 966, status = succeeded4203worker1 # [ 59.382466] nix-daemon[856]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4204worker1 # [ 59.412451] nix-daemon[856]: reaped child process 984, status = succeeded4205lb # [ 59.081902] niks3-server[1094]: 2026/09/20 12:47:05 INFO Received uploads request method=POST path=/api/pending_closures4206worker1 # [ 59.435432] nix-grpc-daemon[868]: time=2026-09-20T12:47:05.246Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4207worker1 # [ 59.437409] nix-grpc-daemon[868]: time=2026-09-20T12:47:05.246Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4208lb # [ 59.147092] niks3-server[1094]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4209lb # [ 59.156466] niks3-server[1094]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4210worker1 # [ 59.510797] nix-grpc-daemon[868]: time=2026-09-20T12:47:05.323Z level=INFO msg="Uploading 1 narinfos"4211lb # [ 59.168526] niks3-server[1094]: 2026/09/20 12:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4212lb # [ 59.171555] niks3-server[1094]: 2026/09/20 12:47:05 INFO Signed narinfos id=6 count=14213lb # [ 59.182129] niks3-server[1094]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4214lb # [ 59.192208] niks3-server[1094]: 2026/09/20 12:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4215lb # [ 59.199536] niks3-server[1094]: 2026/09/20 12:47:05 INFO Completed upload id=64216worker1 # [ 59.543090] nix-grpc-daemon[868]: time=2026-09-20T12:47:05.356Z level=INFO msg="Upload complete. (398ms)"4217lb # [ 59.202393] niks3-server[1094]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4218worker1 # [ 59.557065] nix-grpc-daemon[786]: ts=2026-09-20T12:47:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14219worker1 # [ 59.577891] nix-daemon[856]: reaped child process 964, status = succeeded4220client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.87 seconds)4221client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4222client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.30 seconds)4223client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4224client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4225client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4226client # copying 3 paths...4227client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4228client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4229client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4230client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.46 seconds)4231(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.78 seconds)4232subtest: repeat is answered Cached by the scheduler without building4233worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4234worker1 # finding garbage collector roots...4235worker1 # deleting specified paths...4236worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4237worker1 # deleting unused links...4238worker1 # note: hard linking is currently saving 0.0 KiB4239worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.37 seconds)4240worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4241worker2 # finding garbage collector roots...4242worker2 # deleting specified paths...4243worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4244worker2 # deleting unused links...4245worker2 # note: hard linking is currently saving 0.0 KiB4246worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.38 seconds)4247client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24248worker1 # [ 61.603487] nix-grpc-daemon[786]: ts=2026-09-20T12:47:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415584249client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4250client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4251worker2 # [ 61.615605] nix-grpc-daemon[776]: ts=2026-09-20T12:47:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704252worker1 # [ 61.682367] nix-grpc-daemon[786]: ts=2026-09-20T12:47:07Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14253client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.44 seconds)4254client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4255client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.27 seconds)4256lb: must succeed: curl -sf localhost:9901/clusters4257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4258lb: must succeed: curl -sf localhost:9901/clusters4259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4260worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4261worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4262(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.84 seconds)4263subtest: inputs already in the cache are not sent to the scheduler4264lb: must succeed: curl -sf localhost:9901/clusters4265lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4266lb: must succeed: curl -sf localhost:9901/clusters4267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4268worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4269worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.08 seconds)4270client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24271worker2 # [ 62.905762] nix-grpc-daemon[776]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587704272client # don't know how to build these paths:4273client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4274worker1 # [ 62.946444] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584275worker1 # [ 62.961710] nix-grpc-daemon[868]: time=2026-09-20T12:47:08.772Z level=INFO msg="All 2 paths already cached"4276worker1 # [ 62.978068] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04277worker1 # [ 62.986414] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14278worker1 # [ 63.025361] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584279worker1 # [ 63.049260] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584280worker1 # [ 63.065728] nix-grpc-daemon[868]: time=2026-09-20T12:47:08.878Z level=INFO msg="All 2 paths already cached"4281client # copying 1 paths to 'lb:50051'...4282client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4283worker1 # [ 63.075491] nix-grpc-daemon[786]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584284worker1 # [ 63.093794] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4285worker1 # [ 63.332299] nix-daemon[856]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4286worker1 # [ 63.360299] nix-daemon[856]: reaped child process 1016, status = succeeded4287worker1 # [ 63.584729] nix-daemon[856]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)4288worker1 # [ 63.608126] nix-daemon[856]: reaped child process 1021, status = succeeded4289lb # [ 63.283329] niks3-server[1094]: 2026/09/20 12:47:09 INFO Received uploads request method=POST path=/api/pending_closures4290worker1 # [ 63.642460] nix-grpc-daemon[868]: time=2026-09-20T12:47:09.455Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4291worker1 # [ 63.644477] nix-grpc-daemon[868]: time=2026-09-20T12:47:09.455Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4292lb # [ 63.329193] niks3-server[1094]: 2026/09/20 12:47:09 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4293worker1 # [ 63.691132] nix-grpc-daemon[868]: time=2026-09-20T12:47:09.504Z level=INFO msg="Uploading 1 narinfos"4294lb # [ 63.347929] niks3-server[1094]: 2026/09/20 12:47:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4295lb # [ 63.361086] niks3-server[1094]: 2026/09/20 12:47:09 INFO Signed narinfos id=7 count=14296lb # [ 63.367808] niks3-server[1094]: 2026/09/20 12:47:09 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4297lb # [ 63.384185] niks3-server[1094]: 2026/09/20 12:47:09 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4298worker1 # [ 63.731443] nix-grpc-daemon[868]: time=2026-09-20T12:47:09.544Z level=INFO msg="Upload complete. (619ms)"4299lb # [ 63.389667] niks3-server[1094]: 2026/09/20 12:47:09 INFO Completed upload id=74300lb # [ 63.394203] niks3-server[1094]: 2026/09/20 12:47:09 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4301worker1 # [ 63.740396] nix-grpc-daemon[786]: ts=2026-09-20T12:47:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=7524302worker1 # [ 63.749000] nix-grpc-daemon[786]: ts=2026-09-20T12:47:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584303worker1 # [ 63.760982] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4304worker1 # [ 63.766254] nix-daemon[856]: reaped child process 1011, status = succeeded4305client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4306worker1 # [ 63.780529] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4307client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4308worker1 # [ 65.413273] nix-daemon[856]: reaped child process 1025, status = succeeded4309client # farm-top> LOG-farm-top-t1b4310worker1 # [ 65.614503] nix-daemon[856]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4311worker1 # [ 65.638102] nix-daemon[856]: reaped child process 1043, status = succeeded4312lb # [ 65.309422] niks3-server[1094]: 2026/09/20 12:47:11 INFO Received uploads request method=POST path=/api/pending_closures4313worker1 # [ 65.660652] nix-grpc-daemon[868]: time=2026-09-20T12:47:11.471Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4314worker1 # [ 65.662197] nix-grpc-daemon[868]: time=2026-09-20T12:47:11.471Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4315lb # [ 65.371440] niks3-server[1094]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4316lb # [ 65.381588] niks3-server[1094]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4317worker1 # [ 65.733936] nix-grpc-daemon[868]: time=2026-09-20T12:47:11.547Z level=INFO msg="Uploading 1 narinfos"4318lb # [ 65.391317] niks3-server[1094]: 2026/09/20 12:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4319lb # [ 65.399803] niks3-server[1094]: 2026/09/20 12:47:11 INFO Signed narinfos id=8 count=14320lb # [ 65.404648] niks3-server[1094]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4321lb # [ 65.416197] niks3-server[1094]: 2026/09/20 12:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4322lb # [ 65.424287] niks3-server[1094]: 2026/09/20 12:47:11 INFO Completed upload id=84323worker1 # [ 65.766622] nix-grpc-daemon[868]: time=2026-09-20T12:47:11.580Z level=INFO msg="Upload complete. (371ms)"4324lb # [ 65.426596] niks3-server[1094]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4325worker1 # [ 65.785567] nix-grpc-daemon[786]: ts=2026-09-20T12:47:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=2 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14326worker1 # [ 65.803387] nix-daemon[856]: reaped child process 1023, status = succeeded4327client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 3.23 seconds)4328client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4329client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.25 seconds)4330lb: must succeed: curl -sf localhost:9901/clusters4331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4332lb: must succeed: curl -sf localhost:9901/clusters4333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4334worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4335worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4336(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.93 seconds)4337subtest: two clients wanting the same drv build it once4338client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4339client # [ 66.394022] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4340client # Running as unit: dedup1.service; invocation ID: a1550392f0564925b37483ccb9a38b9c4341client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.12 seconds)4342client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4343worker2 # [ 66.687771] nix-grpc-daemon[776]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587704344client # [ 66.711676] nix[885]: don't know how to build these paths:4345client # [ 66.713162] nix[885]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4346worker1 # [ 66.735465] nix-grpc-daemon[786]: ts=2026-09-20T12:47:12Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04347worker1 # [ 66.735837] nix-grpc-daemon[786]: ts=2026-09-20T12:47:12Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14348worker2 # [ 66.762622] nix-grpc-daemon[776]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704349worker2 # [ 66.783750] nix-grpc-daemon[776]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704350client # [ 66.812223] nix[885]: copying 1 paths to 'lb:50051'...4351worker2 # [ 66.799473] nix-grpc-daemon[776]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587704352client # [ 66.820238] nix[885]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4353worker2 # [ 66.818267] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4354worker2 # [ 66.836412] nix-grpc-daemon[776]: ts=2026-09-20T12:47:12Z level=info event=niks3_push_started pid=8654355worker2 # [ 67.123338] nix-grpc-daemon[865]: time=2026-09-20T12:47:13.058Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4356worker2 # [ 67.337740] nix-daemon[841]: accepted connection from pid 878, user nix-grpc-daemon (trusted)4357worker2 # [ 67.362566] nix-daemon[841]: reaped child process 881, status = succeeded4358client # [ 67.525140] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4359client # Running as unit: dedup2.service; invocation ID: 2d8ad4eec4ee4d2a9b0797acd096817b4360client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4361lb # [ 67.245212] envoy[909]: conn peer=[2001:db8:1::1]:45852 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=3861 ms=75844362lb # [ 67.250507] envoy[909]: conn peer=[2001:db8:1::1]:50540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=503 ms=1064363lb # [ 67.257100] envoy[909]: conn peer=[2001:db8:1::1]:50544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1729 ms=28694364worker2 # [ 67.583033] nix-daemon[841]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4365worker2 # [ 67.606577] nix-daemon[841]: reaped child process 886, status = succeeded4366lb # [ 67.298546] niks3-server[1094]: 2026/09/20 12:47:13 INFO Received uploads request method=POST path=/api/pending_closures4367worker2 # [ 67.626175] nix-grpc-daemon[865]: time=2026-09-20T12:47:13.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4368worker2 # [ 67.630337] nix-grpc-daemon[865]: time=2026-09-20T12:47:13.565Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4369lb # [ 67.342472] niks3-server[1094]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4370lb # [ 67.360872] niks3-server[1094]: 2026/09/20 12:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4371worker2 # [ 67.688323] nix-grpc-daemon[865]: time=2026-09-20T12:47:13.624Z level=INFO msg="Uploading 1 narinfos"4372lb # [ 67.371605] niks3-server[1094]: 2026/09/20 12:47:13 INFO Signed narinfos id=9 count=14373lb # [ 67.382021] niks3-server[1094]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4374lb # [ 67.397655] niks3-server[1094]: 2026/09/20 12:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4375worker2 # [ 67.728120] nix-grpc-daemon[865]: time=2026-09-20T12:47:13.665Z level=INFO msg="Upload complete. (595ms)"4376lb # [ 67.407312] niks3-server[1094]: 2026/09/20 12:47:13 INFO Completed upload id=94377lb # [ 67.411504] niks3-server[1094]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4378worker2 # [ 67.736654] nix-grpc-daemon[776]: ts=2026-09-20T12:47:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=0 paths=1 nar_bytes_in=4324379worker2 # [ 67.739932] nix-grpc-daemon[776]: ts=2026-09-20T12:47:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704380worker2 # [ 67.749209] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4381worker2 # [ 67.753370] nix-daemon[841]: reaped child process 863, status = succeeded4382client # [ 67.777610] nix[885]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4383worker2 # [ 67.764969] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4384worker1 # [ 67.900814] nix-grpc-daemon[786]: ts=2026-09-20T12:47:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415584385client # [ 67.900211] nix[901]: don't know how to build these paths:4386client # [ 67.901173] nix[901]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4387worker1 # [ 67.924506] nix-grpc-daemon[786]: ts=2026-09-20T12:47:13Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04388worker2 # [ 67.917140] nix-grpc-daemon[776]: ts=2026-09-20T12:47:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704389client # [ 67.940781] nix[901]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4390client: must succeed: sleep 24391client: (finished: must succeed: sleep 2, in 2.09 seconds)4392??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4393 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394394client: waiting for success: ! systemctl is-active dedup1 dedup24395??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4396 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394397worker2 # [ 71.192579] nix-daemon[841]: reaped child process 894, status = succeeded4398worker2 # [ 71.375346] nix-daemon[841]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4399worker2 # [ 71.400730] nix-daemon[841]: reaped child process 922, status = succeeded4400lb # [ 71.093601] niks3-server[1094]: 2026/09/20 12:47:17 INFO Received uploads request method=POST path=/api/pending_closures4401worker2 # [ 71.427859] nix-grpc-daemon[865]: time=2026-09-20T12:47:17.363Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4402worker2 # [ 71.430096] nix-grpc-daemon[865]: time=2026-09-20T12:47:17.363Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4403lb # [ 71.158204] niks3-server[1094]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4404lb # [ 71.170164] niks3-server[1094]: 2026/09/20 12:47:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4405worker2 # [ 71.492588] nix-grpc-daemon[865]: time=2026-09-20T12:47:17.427Z level=INFO msg="Uploading 1 narinfos"4406lb # [ 71.178004] niks3-server[1094]: 2026/09/20 12:47:17 INFO Signed narinfos id=10 count=14407lb # [ 71.185934] niks3-server[1094]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4408lb # [ 71.197431] niks3-server[1094]: 2026/09/20 12:47:17 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4409lb # [ 71.208856] niks3-server[1094]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4410lb # [ 71.216238] niks3-server[1094]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4411lb # [ 71.220426] niks3-server[1094]: 2026/09/20 12:47:17 INFO Completed upload id=104412worker2 # [ 71.541231] nix-grpc-daemon[865]: time=2026-09-20T12:47:17.478Z level=INFO msg="Upload complete. (364ms)"4413worker2 # [ 71.557870] nix-grpc-daemon[776]: ts=2026-09-20T12:47:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14414worker2 # [ 71.576759] nix-grpc-daemon[776]: ts=2026-09-20T12:47:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14415worker2 # [ 71.587222] nix-daemon[841]: reaped child process 892, status = succeeded4416client # [ 71.629938] systemd[1]: dedup2.service: Deactivated successfully.4417client # [ 71.641181] systemd[1]: dedup2.service: Consumed 212ms CPU time over 4.109s wall clock time, 31M memory peak, 7.4K incoming IP traffic, 6.5K outgoing IP traffic.4418client # [ 71.680321] systemd[1]: dedup1.service: Deactivated successfully.4419client # [ 71.682760] systemd[1]: dedup1.service: Consumed 205ms CPU time over 5.291s wall clock time, 37.2M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.4420client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.19 seconds)4421client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4422client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.08 seconds)4423worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4424worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.09 seconds)4425worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4426worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4427worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4428worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4429worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4430worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4431(finished: subtest: two clients wanting the same drv build it once, in 6.44 seconds)4432subtest: build hook: nix-daemon with builders = grpc://lb4433client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4434client # [ 73.045028] systemd[1]: Started Nix Daemon.4435client # [ 73.270412] nix-daemon[943]: accepted connection from pid 940, user root (trusted)4436worker2 # [ 73.505287] nix-grpc-daemon[776]: ts=2026-09-20T12:47:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587704437worker1 # [ 73.538930] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584438worker1 # [ 73.554353] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04439worker1 # [ 73.559190] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14440worker1 # [ 73.600257] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584441worker1 # [ 73.618253] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584442worker1 # [ 73.635329] nix-grpc-daemon[786]: ts=2026-09-20T12:47:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584443worker1 # [ 73.667954] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4444worker1 # [ 73.924275] nix-daemon[856]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4445worker1 # [ 73.948229] nix-daemon[856]: reaped child process 1082, status = succeeded4446worker1 # [ 74.142017] nix-daemon[856]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4447worker1 # [ 74.164877] nix-daemon[856]: reaped child process 1087, status = succeeded4448lb # [ 73.831505] niks3-server[1094]: 2026/09/20 12:47:19 INFO Received uploads request method=POST path=/api/pending_closures4449worker1 # [ 74.193656] nix-grpc-daemon[868]: time=2026-09-20T12:47:20.006Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4450worker1 # [ 74.195387] nix-grpc-daemon[868]: time=2026-09-20T12:47:20.006Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4451lb # [ 73.879611] niks3-server[1094]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4452worker1 # [ 74.238773] nix-grpc-daemon[868]: time=2026-09-20T12:47:20.052Z level=INFO msg="Uploading 1 narinfos"4453lb # [ 73.899136] niks3-server[1094]: 2026/09/20 12:47:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4454lb # [ 73.909546] niks3-server[1094]: 2026/09/20 12:47:19 INFO Signed narinfos id=11 count=14455lb # [ 73.913854] niks3-server[1094]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4456lb # [ 73.922350] niks3-server[1094]: 2026/09/20 12:47:19 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4457lb # [ 73.928069] niks3-server[1094]: 2026/09/20 12:47:20 INFO Completed upload id=114458worker1 # [ 74.270120] nix-grpc-daemon[868]: time=2026-09-20T12:47:20.083Z level=INFO msg="Upload complete. (577ms)"4459lb # [ 73.931086] niks3-server[1094]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4460worker1 # [ 74.280403] nix-grpc-daemon[786]: ts=2026-09-20T12:47:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=5124461worker1 # [ 74.296602] nix-grpc-daemon[786]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584462worker1 # [ 74.304447] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4463worker1 # [ 74.310402] nix-daemon[856]: reaped child process 1077, status = succeeded4464worker1 # [ 74.324570] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4465worker1 # [ 75.383165] nix-daemon[856]: reaped child process 1091, status = succeeded4466worker1 # [ 75.569307] nix-daemon[856]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4467worker1 # [ 75.595759] nix-daemon[856]: reaped child process 1109, status = succeeded4468lb # [ 75.265468] niks3-server[1094]: 2026/09/20 12:47:21 INFO Received uploads request method=POST path=/api/pending_closures4469worker1 # [ 75.614903] nix-grpc-daemon[868]: time=2026-09-20T12:47:21.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4470worker1 # [ 75.621288] nix-grpc-daemon[868]: time=2026-09-20T12:47:21.430Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4471lb # [ 75.317428] niks3-server[1094]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4472lb # [ 75.324976] niks3-server[1094]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4473lb # [ 75.334498] niks3-server[1094]: 2026/09/20 12:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4474worker1 # [ 75.681952] nix-grpc-daemon[868]: time=2026-09-20T12:47:21.494Z level=INFO msg="Uploading 1 narinfos"4475lb # [ 75.340712] niks3-server[1094]: 2026/09/20 12:47:21 INFO Signed narinfos id=12 count=14476lb # [ 75.344777] niks3-server[1094]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4477lb # [ 75.353044] niks3-server[1094]: 2026/09/20 12:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4478lb # [ 75.358770] niks3-server[1094]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4479lb # [ 75.362973] niks3-server[1094]: 2026/09/20 12:47:21 INFO Completed upload id=124480worker1 # [ 75.705771] nix-grpc-daemon[868]: time=2026-09-20T12:47:21.519Z level=INFO msg="Upload complete. (360ms)"4481worker1 # [ 75.717085] nix-grpc-daemon[786]: ts=2026-09-20T12:47:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14482worker2 # [ 75.698623] nix-grpc-daemon[776]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:587704483worker1 # [ 75.730369] nix-daemon[856]: reaped child process 1089, status = succeeded4484worker2 # [ 75.716920] nix-grpc-daemon[776]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4485worker2 # [ 75.765652] nix-grpc-daemon[776]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:587704486worker2 # [ 75.773053] nix-grpc-daemon[776]: ts=2026-09-20T12:47:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=0 paths=1 nar_bytes_out=1284487worker2 # [ 76.092689] nix-grpc-daemon[776]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587704488worker2 # [ 76.101259] nix-grpc-daemon[776]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704489worker2 # [ 76.109497] nix-grpc-daemon[865]: time=2026-09-20T12:47:22.044Z level=INFO msg="All 1 paths already cached"4490worker1 # [ 76.146636] nix-grpc-daemon[786]: ts=2026-09-20T12:47:21Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04491worker1 # [ 76.154457] nix-grpc-daemon[786]: ts=2026-09-20T12:47:21Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14492worker1 # [ 76.165222] nix-grpc-daemon[786]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584493worker1 # [ 76.177687] nix-grpc-daemon[786]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584494worker1 # [ 76.192950] nix-grpc-daemon[868]: time=2026-09-20T12:47:22.006Z level=INFO msg="All 1 paths already cached"4495worker1 # [ 76.201547] nix-grpc-daemon[786]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584496worker1 # [ 76.217486] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4497worker1 # [ 76.442854] nix-daemon[856]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4498worker1 # [ 76.466932] nix-daemon[856]: reaped child process 1118, status = succeeded4499worker1 # [ 76.651207] nix-daemon[856]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4500worker1 # [ 76.671654] nix-daemon[856]: reaped child process 1123, status = succeeded4501lb # [ 76.342567] niks3-server[1094]: 2026/09/20 12:47:22 INFO Received uploads request method=POST path=/api/pending_closures4502worker1 # [ 76.734125] nix-grpc-daemon[868]: time=2026-09-20T12:47:22.547Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4503worker1 # [ 76.737438] nix-grpc-daemon[868]: time=2026-09-20T12:47:22.549Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4504lb # [ 76.414592] niks3-server[1094]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4505lb # [ 76.428573] niks3-server[1094]: 2026/09/20 12:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4506worker1 # [ 76.776917] nix-grpc-daemon[868]: time=2026-09-20T12:47:22.590Z level=INFO msg="Uploading 1 narinfos"4507lb # [ 76.438909] niks3-server[1094]: 2026/09/20 12:47:22 INFO Signed narinfos id=13 count=14508lb # [ 76.445593] niks3-server[1094]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4509lb # [ 76.454425] niks3-server[1094]: 2026/09/20 12:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4510lb # [ 76.467460] niks3-server[1094]: 2026/09/20 12:47:22 INFO Completed upload id=134511worker1 # [ 76.810611] nix-grpc-daemon[868]: time=2026-09-20T12:47:22.623Z level=INFO msg="Upload complete. (574ms)"4512worker1 # [ 76.828490] nix-grpc-daemon[786]: ts=2026-09-20T12:47:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=6244513worker1 # [ 76.843159] nix-grpc-daemon[786]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584514worker1 # [ 76.854490] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4515worker1 # [ 76.865887] nix-daemon[856]: reaped child process 1113, status = succeeded4516worker1 # [ 76.878208] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4517lb # [ 76.556284] niks3-server[1094]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4518lb # [ 77.245749] envoy[909]: conn peer=[2001:db8:1::1]:50558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1086 ms=37044519lb # [ 77.251310] envoy[909]: conn peer=[2001:db8:1::1]:50554 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1374 ms=49394520lb # [ 77.257582] envoy[909]: conn peer=[2001:db8:1::1]:35012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3404 tx=1771 ms=23344521lb # [ 77.263676] envoy[909]: conn peer=[2001:db8:1::1]:35028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=754522worker1 # [ 77.704434] nix-daemon[856]: reaped child process 1127, status = succeeded4523worker1 # [ 77.870470] nix-daemon[856]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4524worker1 # [ 77.893708] nix-daemon[856]: reaped child process 1145, status = succeeded4525lb # [ 77.563689] niks3-server[1094]: 2026/09/20 12:47:23 INFO Received uploads request method=POST path=/api/pending_closures4526worker1 # [ 77.916755] nix-grpc-daemon[868]: time=2026-09-20T12:47:23.729Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4527worker1 # [ 77.919984] nix-grpc-daemon[868]: time=2026-09-20T12:47:23.732Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4528worker1 # [ 77.961134] nix-grpc-daemon[868]: time=2026-09-20T12:47:23.774Z level=INFO msg="Uploading 1 narinfos"4529lb # [ 77.611234] niks3-server[1094]: 2026/09/20 12:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4530lb # [ 77.630786] niks3-server[1094]: 2026/09/20 12:47:23 INFO Signed narinfos id=14 count=14531lb # [ 77.639422] niks3-server[1094]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4532lb # [ 77.649082] niks3-server[1094]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4533worker1 # [ 78.000879] nix-grpc-daemon[868]: time=2026-09-20T12:47:23.814Z level=INFO msg="Upload complete. (310ms)"4534lb # [ 77.660352] niks3-server[1094]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4535lb # [ 77.665709] niks3-server[1094]: 2026/09/20 12:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4536lb # [ 77.669611] niks3-server[1094]: 2026/09/20 12:47:23 INFO Completed upload id=144537worker1 # [ 78.013650] nix-grpc-daemon[786]: ts=2026-09-20T12:47:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14538lb # [ 77.673162] niks3-server[1094]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4539worker1 # [ 78.027507] nix-grpc-daemon[786]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584540worker1 # [ 78.030974] nix-daemon[856]: reaped child process 1125, status = succeeded4541worker1 # [ 78.048203] nix-grpc-daemon[786]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:415584542worker1 # [ 78.054399] nix-grpc-daemon[786]: ts=2026-09-20T12:47:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_out=1764543worker1 # [ 78.395057] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:415584544worker2 # [ 78.381207] nix-grpc-daemon[776]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704545worker2 # [ 78.407880] nix-grpc-daemon[865]: time=2026-09-20T12:47:24.343Z level=INFO msg="All 1 paths already cached"4546worker1 # [ 78.453181] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04547worker1 # [ 78.461725] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14548worker1 # [ 78.500986] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584549worker1 # [ 78.520925] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584550worker1 # [ 78.775745] nix-grpc-daemon[868]: time=2026-09-20T12:47:24.589Z level=INFO msg="All 2 paths already cached"4551worker1 # [ 78.788916] nix-grpc-daemon[786]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584552worker1 # [ 78.805299] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4553worker1 # [ 79.038772] nix-daemon[856]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4554worker1 # [ 79.062991] nix-daemon[856]: reaped child process 1154, status = succeeded4555worker1 # [ 79.253946] nix-daemon[856]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4556worker1 # [ 79.277392] nix-daemon[856]: reaped child process 1159, status = succeeded4557lb # [ 78.953451] niks3-server[1094]: 2026/09/20 12:47:25 INFO Received uploads request method=POST path=/api/pending_closures4558worker1 # [ 79.369165] nix-grpc-daemon[868]: time=2026-09-20T12:47:25.181Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4559worker1 # [ 79.373791] nix-grpc-daemon[868]: time=2026-09-20T12:47:25.181Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4560lb # [ 79.051668] niks3-server[1094]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4561lb # [ 79.066069] niks3-server[1094]: 2026/09/20 12:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4562worker1 # [ 79.416338] nix-grpc-daemon[868]: time=2026-09-20T12:47:25.229Z level=INFO msg="Uploading 1 narinfos"4563lb # [ 79.078900] niks3-server[1094]: 2026/09/20 12:47:25 INFO Signed narinfos id=15 count=14564lb # [ 79.085041] niks3-server[1094]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4565lb # [ 79.097067] niks3-server[1094]: 2026/09/20 12:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4566lb # [ 79.103636] niks3-server[1094]: 2026/09/20 12:47:25 INFO Completed upload id=154567worker1 # [ 79.445615] nix-grpc-daemon[868]: time=2026-09-20T12:47:25.259Z level=INFO msg="Upload complete. (616ms)"4568worker1 # [ 79.461641] nix-grpc-daemon[786]: ts=2026-09-20T12:47:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_in=7524569worker1 # [ 79.480873] nix-grpc-daemon[786]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584570worker1 # [ 79.489211] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4571worker1 # [ 79.496556] nix-daemon[856]: reaped child process 1149, status = succeeded4572worker1 # [ 79.518291] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4573lb # [ 79.188290] niks3-server[1094]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4574worker1 # [ 81.373521] nix-daemon[856]: reaped child process 1163, status = succeeded4575worker1 # [ 81.548398] nix-daemon[856]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4576worker1 # [ 81.571424] nix-daemon[856]: reaped child process 1181, status = succeeded4577lb # [ 81.242946] niks3-server[1094]: 2026/09/20 12:47:27 INFO Received uploads request method=POST path=/api/pending_closures4578worker1 # [ 81.600706] nix-grpc-daemon[868]: time=2026-09-20T12:47:27.412Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4579worker1 # [ 81.603360] nix-grpc-daemon[868]: time=2026-09-20T12:47:27.412Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4580lb # [ 81.301869] niks3-server[1094]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4581worker1 # [ 81.660099] nix-grpc-daemon[868]: time=2026-09-20T12:47:27.473Z level=INFO msg="Uploading 1 narinfos"4582lb # [ 81.317731] niks3-server[1094]: 2026/09/20 12:47:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4583lb # [ 81.329121] niks3-server[1094]: 2026/09/20 12:47:27 INFO Signed narinfos id=16 count=14584lb # [ 81.334604] niks3-server[1094]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4585lb # [ 81.340499] niks3-server[1094]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4586lb # [ 81.347504] niks3-server[1094]: 2026/09/20 12:47:27 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4587lb # [ 81.355515] niks3-server[1094]: 2026/09/20 12:47:27 INFO Completed upload id=164588worker1 # [ 81.698398] nix-grpc-daemon[868]: time=2026-09-20T12:47:27.511Z level=INFO msg="Upload complete. (363ms)"4589lb # [ 81.358320] niks3-server[1094]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4590worker1 # [ 81.714579] nix-grpc-daemon[786]: ts=2026-09-20T12:47:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=2 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14591worker1 # [ 81.732333] nix-grpc-daemon[786]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584592worker1 # [ 81.737136] nix-daemon[856]: reaped child process 1161, status = succeeded4593worker1 # [ 81.742767] nix-grpc-daemon[786]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:415584594worker1 # [ 81.746828] nix-grpc-daemon[786]: ts=2026-09-20T12:47:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_out=2324595client # [ 81.870968] nix-daemon[943]: reaped child process 945, status = succeeded4596client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 9.14 seconds)4597client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4598client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4599client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4600client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4601client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4602client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4603(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 9.28 seconds)4604subtest: an input only one worker has still reaches the builder4605client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4606client # this derivation will be built:4607client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4608client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4609client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.82 seconds)4610client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4611client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.25 seconds)4612worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4613worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.28 seconds)4614worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4615worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.05 seconds)4616client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24617client # [ 83.682302] nix-daemon[943]: accepted connection from pid 1026, user root (trusted)4618client # this derivation will be built:4619client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4620worker2 # [ 83.883401] nix-grpc-daemon[776]: ts=2026-09-20T12:47:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587704621client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4622worker1 # [ 83.910601] nix-grpc-daemon[786]: ts=2026-09-20T12:47:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584623worker1 # [ 84.140391] nix-daemon[856]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)4624worker1 # [ 84.164368] nix-daemon[856]: reaped child process 1191, status = succeeded4625lb # [ 83.834010] niks3-server[1094]: 2026/09/20 12:47:29 INFO Received uploads request method=POST path=/api/pending_closures4626worker1 # [ 84.197018] nix-grpc-daemon[868]: time=2026-09-20T12:47:30.009Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4627worker1 # [ 84.199794] nix-grpc-daemon[868]: time=2026-09-20T12:47:30.009Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4628lb # [ 83.881381] niks3-server[1094]: 2026/09/20 12:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4629worker1 # [ 84.231854] nix-grpc-daemon[868]: time=2026-09-20T12:47:30.045Z level=INFO msg="Uploading 1 narinfos"4630lb # [ 83.891642] niks3-server[1094]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4631lb # [ 83.902373] niks3-server[1094]: 2026/09/20 12:47:29 INFO Signed narinfos id=17 count=14632lb # [ 83.910566] niks3-server[1094]: 2026/09/20 12:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4633lb # [ 84.462943] niks3-server[1094]: 2026/09/20 12:47:30 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4634lb # [ 84.553130] niks3-server[1094]: 2026/09/20 12:47:30 INFO Completed upload id=174635worker1 # [ 84.896351] nix-grpc-daemon[868]: time=2026-09-20T12:47:30.709Z level=INFO msg="Upload complete. (979ms)"4636lb # [ 84.578810] niks3-server[1094]: 2026/09/20 12:47:30 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4637worker1 # [ 84.922524] nix-grpc-daemon[786]: ts=2026-09-20T12:47:30Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04638worker1 # [ 84.930468] nix-grpc-daemon[786]: ts=2026-09-20T12:47:30Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14639worker2 # [ 84.953844] nix-grpc-daemon[776]: ts=2026-09-20T12:47:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704640worker2 # [ 84.972482] nix-grpc-daemon[776]: ts=2026-09-20T12:47:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704641client # copying 2 paths to 'lb:50051'...4642client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4643worker2 # [ 84.994345] nix-grpc-daemon[776]: ts=2026-09-20T12:47:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587704644client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4645worker2 # [ 85.007808] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4646worker2 # [ 85.245646] nix-daemon[841]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4647worker2 # [ 85.268952] nix-daemon[841]: reaped child process 942, status = succeeded4648worker2 # [ 85.449586] nix-daemon[841]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4649worker2 # [ 85.471786] nix-daemon[841]: reaped child process 947, status = succeeded4650lb # [ 85.165455] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received uploads request method=POST path=/api/pending_closures4651lb # [ 85.172292] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received uploads request method=POST path=/api/pending_closures4652worker2 # [ 85.502339] nix-grpc-daemon[865]: time=2026-09-20T12:47:31.437Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4653worker2 # [ 85.503924] nix-grpc-daemon[865]: time=2026-09-20T12:47:31.438Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4654worker2 # [ 85.505982] nix-grpc-daemon[865]: time=2026-09-20T12:47:31.438Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4655lb # [ 85.213613] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4656lb # [ 85.223026] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4657lb # [ 85.233726] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4658worker2 # [ 85.561800] nix-grpc-daemon[865]: time=2026-09-20T12:47:31.498Z level=INFO msg="Uploading 3 narinfos"4659lb # [ 85.243366] niks3-server[1094]: 2026/09/20 12:47:31 INFO Signed narinfos id=18 count=14660lb # [ 85.248400] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4661lb # [ 85.251300] niks3-server[1094]: 2026/09/20 12:47:31 INFO Signed narinfos id=19 count=24662lb # [ 85.257955] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4663lb # [ 85.263019] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4664lb # [ 85.276438] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4665lb # [ 85.286521] niks3-server[1094]: 2026/09/20 12:47:31 INFO Completed upload id=184666lb # [ 85.289413] niks3-server[1094]: 2026/09/20 12:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4667lb # [ 85.293403] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4668lb # [ 85.295516] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4669lb # [ 85.297785] niks3-server[1094]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4670lb # [ 85.300619] niks3-server[1094]: 2026/09/20 12:47:31 INFO Completed upload id=194671worker2 # [ 85.622156] nix-grpc-daemon[865]: time=2026-09-20T12:47:31.559Z level=INFO msg="Upload complete. (586ms)"4672worker2 # [ 85.628303] nix-grpc-daemon[776]: ts=2026-09-20T12:47:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=0 paths=2 nar_bytes_in=9844673worker2 # [ 85.637429] nix-grpc-daemon[776]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704674worker2 # [ 85.641896] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4675worker2 # [ 85.645862] nix-daemon[841]: reaped child process 937, status = succeeded4676worker2 # [ 85.657332] nix-grpc-daemon[776]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4677worker2 # [ 85.670848] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4678client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4679client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4680worker2 # [ 86.896435] nix-daemon[841]: reaped child process 952, status = succeeded4681worker2 # [ 87.071448] nix-daemon[841]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4682worker2 # [ 87.092408] nix-daemon[841]: reaped child process 970, status = succeeded4683lb # [ 86.786530] niks3-server[1094]: 2026/09/20 12:47:32 INFO Received uploads request method=POST path=/api/pending_closures4684worker2 # [ 87.116353] nix-grpc-daemon[865]: time=2026-09-20T12:47:33.052Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4685lb # [ 86.809332] niks3-server[1094]: 2026/09/20 12:47:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4686worker2 # [ 87.133843] nix-grpc-daemon[865]: time=2026-09-20T12:47:33.070Z level=INFO msg="Uploading 1 narinfos"4687lb # [ 86.816822] niks3-server[1094]: 2026/09/20 12:47:32 INFO Signed narinfos id=20 count=14688lb # [ 86.820278] niks3-server[1094]: 2026/09/20 12:47:32 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4689lb # [ 86.822128] niks3-server[1094]: 2026/09/20 12:47:32 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4690worker2 # [ 87.151993] nix-grpc-daemon[865]: time=2026-09-20T12:47:33.089Z level=INFO msg="Upload complete. (268ms)"4691lb # [ 86.831785] niks3-server[1094]: 2026/09/20 12:47:32 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4692lb # [ 86.838154] niks3-server[1094]: 2026/09/20 12:47:32 INFO Completed upload id=204693lb # [ 86.839296] niks3-server[1094]: 2026/09/20 12:47:32 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4694worker2 # [ 87.158981] nix-grpc-daemon[776]: ts=2026-09-20T12:47:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14695worker2 # [ 87.169396] nix-daemon[841]: reaped child process 949, status = succeeded4696client # copying 1 paths...4697worker2 # [ 87.206986] nix-grpc-daemon[776]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:587704698client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4699worker1 # [ 87.265138] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:415584700worker1 # [ 87.282790] nix-grpc-daemon[786]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4701worker1 # [ 87.305097] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_out=1204702client # [ 87.434275] nix-daemon[943]: reaped child process 1029, status = succeeded4703client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 4.02 seconds)4704client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24705lb # [ 87.249287] envoy[909]: conn peer=[2001:db8:1::1]:35042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3875 tx=1875 ms=20014706lb # [ 87.255496] envoy[909]: conn peer=[2001:db8:1::1]:35058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=724707lb # [ 87.260865] envoy[909]: conn peer=[2001:db8:1::1]:35064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4356 tx=2127 ms=34084708lb # [ 87.268317] envoy[909]: conn peer=[2001:db8:1::1]:46212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=674709lb # [ 87.273563] envoy[909]: conn peer=[2001:db8:1::1]:46216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3864 tx=1766 ms=34614710lb # [ 87.279341] envoy[909]: conn peer=[2001:db8:1::1]:46218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1054711client # [ 87.664133] nix-daemon[943]: accepted connection from pid 1048, user root (trusted)4712client # this derivation will be built:4713client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4714worker1 # [ 87.889591] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:415584715client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4716worker2 # [ 87.872768] nix-grpc-daemon[776]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704717worker2 # [ 87.884516] nix-grpc-daemon[865]: time=2026-09-20T12:47:33.821Z level=INFO msg="All 1 paths already cached"4718worker1 # [ 87.928285] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04719worker1 # [ 87.936710] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14720worker1 # [ 87.946442] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584721worker1 # [ 87.960257] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584722client # copying 1 paths to 'lb:50051'...4723client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4724worker1 # [ 87.992928] nix-grpc-daemon[786]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415584725worker1 # [ 88.009389] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4726worker1 # [ 88.032987] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)4727worker1 # [ 88.061498] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)4728worker1 # [ 88.207645] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 334 ms (attempt 3/5)4729worker1 # [ 88.550386] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 648 ms (attempt 4/5)4730worker1 # [ 89.203468] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4731worker1 # [ 89.211859] nix-grpc-daemon[786]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4732worker1 # [ 89.471475] nix-daemon[856]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)4733worker1 # [ 89.492179] nix-daemon[856]: reaped child process 1207, status = succeeded4734worker1 # [ 89.680245] nix-daemon[856]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)4735worker1 # [ 89.699574] nix-daemon[856]: reaped child process 1212, status = succeeded4736lb # [ 89.371544] niks3-server[1094]: 2026/09/20 12:47:35 INFO Received uploads request method=POST path=/api/pending_closures4737worker1 # [ 89.724481] nix-grpc-daemon[868]: time=2026-09-20T12:47:35.537Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4738worker1 # [ 89.725992] nix-grpc-daemon[868]: time=2026-09-20T12:47:35.537Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4739lb # [ 89.402608] niks3-server[1094]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4740worker1 # [ 89.766632] nix-grpc-daemon[868]: time=2026-09-20T12:47:35.579Z level=INFO msg="Uploading 1 narinfos"4741lb # [ 89.423200] niks3-server[1094]: 2026/09/20 12:47:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4742lb # [ 89.431905] niks3-server[1094]: 2026/09/20 12:47:35 INFO Signed narinfos id=21 count=14743lb # [ 89.434174] niks3-server[1094]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4744lb # [ 89.444148] niks3-server[1094]: 2026/09/20 12:47:35 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4745lb # [ 89.449892] niks3-server[1094]: 2026/09/20 12:47:35 INFO Completed upload id=214746worker1 # [ 89.791877] nix-grpc-daemon[868]: time=2026-09-20T12:47:35.605Z level=INFO msg="Upload complete. (542ms)"4747lb # [ 89.452515] niks3-server[1094]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4748worker1 # [ 89.801038] nix-grpc-daemon[786]: ts=2026-09-20T12:47:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 paths=1 nar_bytes_in=5684749worker1 # [ 89.815705] nix-grpc-daemon[786]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415584750worker1 # [ 89.822430] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4751worker1 # [ 89.828286] nix-daemon[856]: reaped child process 1198, status = succeeded4752client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4753worker1 # [ 89.843827] nix-daemon[856]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4754client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4755worker1 # [ 90.894548] nix-daemon[856]: reaped child process 1216, status = succeeded4756worker1 # [ 91.078941] nix-daemon[856]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)4757worker1 # [ 91.099425] nix-daemon[856]: reaped child process 1234, status = succeeded4758lb # [ 90.771589] niks3-server[1094]: 2026/09/20 12:47:36 INFO Received uploads request method=POST path=/api/pending_closures4759worker1 # [ 91.124655] nix-grpc-daemon[868]: time=2026-09-20T12:47:36.937Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4760worker1 # [ 91.162763] nix-grpc-daemon[868]: time=2026-09-20T12:47:36.975Z level=INFO msg="Uploading 1 narinfos"4761lb # [ 90.814315] niks3-server[1094]: 2026/09/20 12:47:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4762lb # [ 90.823456] niks3-server[1094]: 2026/09/20 12:47:36 INFO Signed narinfos id=22 count=14763lb # [ 90.834650] niks3-server[1094]: 2026/09/20 12:47:36 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4764lb # [ 90.862793] niks3-server[1094]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4765lb # [ 90.885183] niks3-server[1094]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4766lb # [ 90.895529] niks3-server[1094]: 2026/09/20 12:47:36 INFO Completed upload id=224767worker1 # [ 91.238460] nix-grpc-daemon[868]: time=2026-09-20T12:47:37.051Z level=INFO msg="Upload complete. (382ms)"4768lb # [ 90.900320] niks3-server[1094]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4769worker1 # [ 91.251423] nix-grpc-daemon[786]: ts=2026-09-20T12:47:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14770worker1 # [ 91.272969] nix-daemon[856]: reaped child process 1214, status = succeeded4771client # copying 1 paths...4772worker1 # [ 91.301516] nix-grpc-daemon[786]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584773client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4774worker2 # [ 91.318402] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:587704775worker2 # [ 91.334342] nix-grpc-daemon[776]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4776worker2 # [ 91.354812] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=0 paths=1 nar_bytes_out=1204777client # [ 91.504855] nix-daemon[943]: reaped child process 1051, status = succeeded4778client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 4.07 seconds)4779client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24780client # [ 91.727530] nix-daemon[943]: accepted connection from pid 1070, user root (trusted)4781client # this derivation will be built:4782client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4783worker2 # [ 91.918219] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587704784client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4785worker1 # [ 91.947579] nix-grpc-daemon[786]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415584786worker1 # [ 91.958772] nix-grpc-daemon[868]: time=2026-09-20T12:47:37.772Z level=INFO msg="All 1 paths already cached"4787worker1 # [ 91.979201] nix-grpc-daemon[786]: ts=2026-09-20T12:47:37Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04788worker1 # [ 91.986355] nix-grpc-daemon[786]: ts=2026-09-20T12:47:37Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14789worker2 # [ 91.968071] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704790worker2 # [ 91.985335] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587704791worker2 # [ 92.002859] nix-grpc-daemon[865]: time=2026-09-20T12:47:37.939Z level=INFO msg="All 1 paths already cached"4792client # copying 1 paths to 'lb:50051'...4793client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4794worker2 # [ 92.014549] nix-grpc-daemon[776]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587704795worker2 # [ 92.030642] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4796worker2 # [ 92.247671] nix-daemon[841]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4797worker2 # [ 92.271294] nix-daemon[841]: reaped child process 978, status = succeeded4798worker2 # [ 92.461520] nix-daemon[841]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4799worker2 # [ 92.482886] nix-daemon[841]: reaped child process 983, status = succeeded4800lb # [ 92.175004] niks3-server[1094]: 2026/09/20 12:47:38 INFO Received uploads request method=POST path=/api/pending_closures4801worker2 # [ 92.506215] nix-grpc-daemon[865]: time=2026-09-20T12:47:38.443Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4802worker2 # [ 92.507812] nix-grpc-daemon[865]: time=2026-09-20T12:47:38.443Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4803lb # [ 92.207604] niks3-server[1094]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4804lb # [ 92.225535] niks3-server[1094]: 2026/09/20 12:47:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4805worker2 # [ 92.551529] nix-grpc-daemon[865]: time=2026-09-20T12:47:38.488Z level=INFO msg="Uploading 1 narinfos"4806lb # [ 92.237730] niks3-server[1094]: 2026/09/20 12:47:38 INFO Signed narinfos id=23 count=14807lb # [ 92.244332] niks3-server[1094]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4808lb # [ 92.254630] niks3-server[1094]: 2026/09/20 12:47:38 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4809worker2 # [ 92.581769] nix-grpc-daemon[865]: time=2026-09-20T12:47:38.518Z level=INFO msg="Upload complete. (530ms)"4810lb # [ 92.263389] niks3-server[1094]: 2026/09/20 12:47:38 INFO Completed upload id=234811lb # [ 92.267543] niks3-server[1094]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4812worker2 # [ 92.593061] nix-grpc-daemon[776]: ts=2026-09-20T12:47:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=0 paths=1 nar_bytes_in=5684813worker2 # [ 92.606880] nix-grpc-daemon[776]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587704814worker2 # [ 92.614147] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4815worker2 # [ 92.619676] nix-daemon[841]: reaped child process 973, status = succeeded4816worker2 # [ 92.633533] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4817client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4818client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4819worker2 # [ 93.464788] nix-daemon[841]: reaped child process 987, status = succeeded4820worker2 # [ 93.634437] nix-daemon[841]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4821worker2 # [ 93.656453] nix-daemon[841]: reaped child process 1005, status = succeeded4822lb # [ 93.348795] niks3-server[1094]: 2026/09/20 12:47:39 INFO Received uploads request method=POST path=/api/pending_closures4823worker2 # [ 93.680464] nix-grpc-daemon[865]: time=2026-09-20T12:47:39.616Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4824lb # [ 93.381735] niks3-server[1094]: 2026/09/20 12:47:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4825worker2 # [ 93.714179] nix-grpc-daemon[865]: time=2026-09-20T12:47:39.651Z level=INFO msg="Uploading 1 narinfos"4826lb # [ 93.397882] niks3-server[1094]: 2026/09/20 12:47:39 INFO Signed narinfos id=24 count=14827lb # [ 93.404909] niks3-server[1094]: 2026/09/20 12:47:39 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4828lb # [ 93.412628] niks3-server[1094]: 2026/09/20 12:47:39 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4829worker2 # [ 93.742765] nix-grpc-daemon[865]: time=2026-09-20T12:47:39.679Z level=INFO msg="Upload complete. (295ms)"4830lb # [ 93.423207] niks3-server[1094]: 2026/09/20 12:47:39 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4831lb # [ 93.428479] niks3-server[1094]: 2026/09/20 12:47:39 INFO Completed upload id=244832lb # [ 93.431250] niks3-server[1094]: 2026/09/20 12:47:39 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4833worker2 # [ 93.752938] nix-grpc-daemon[776]: ts=2026-09-20T12:47:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58770 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14834worker2 # [ 93.766291] nix-daemon[841]: reaped child process 985, status = succeeded4835client # copying 1 paths...4836worker2 # [ 93.799335] nix-grpc-daemon[776]: ts=2026-09-20T12:47:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:587704837client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4838worker1 # [ 93.853990] nix-grpc-daemon[786]: ts=2026-09-20T12:47:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:415584839worker1 # [ 93.867492] nix-grpc-daemon[786]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4840worker1 # [ 93.886538] nix-grpc-daemon[786]: ts=2026-09-20T12:47:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41558 duration_s=0 paths=1 nar_bytes_out=1204841client # [ 94.009750] nix-daemon[943]: reaped child process 1073, status = succeeded4842client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.50 seconds)4843(finished: subtest: an input only one worker has still reaches the builder, in 11.99 seconds)4844subtest: upload whose reference the worker lacks is completed from the cache4845client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4846client # this derivation will be built:4847client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4848client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4849client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.47 seconds)4850client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4851client # this derivation will be built:4852client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4853client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4854client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.46 seconds)4855client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4856client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.21 seconds)4857worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4858worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.26 seconds)4859worker2 # [ 95.475633] systemd[1]: Stopping Nix worker-protocol over gRPC...4860worker2 # [ 96.050410] nix-grpc-daemon[776]: ts=2026-09-20T12:47:41Z level=info event=signal_exit4861worker2 # [ 97.249050] nix-grpc-daemon[776]: ts=2026-09-20T12:47:43Z level=info event=scheduler_restarting4862worker1 # [ 97.291116] nix-grpc-daemon[786]: ts=2026-09-20T12:47:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415444863worker1 # [ 97.296051] nix-grpc-daemon[786]: ts=2026-09-20T12:47:43Z level=info event=worker_gone addr=192.168.1.4:500514864worker2 # [ 97.285278] nix-grpc-daemon[776]: ts=2026-09-20T12:47:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4865worker2 # [ 97.318276] nix-daemon[841]: reaped child process 843, status = succeeded4866lb # [ 97.247888] envoy[909]: conn peer=[2001:db8:1::1]:46226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1804 ms=35484867lb # [ 97.253537] envoy[909]: conn peer=[2001:db8:1::1]:44886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1044868lb # [ 97.258334] envoy[909]: conn peer=[2001:db8:1::1]:44900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1779 ms=20044869lb # [ 97.263163] envoy[909]: conn peer=[2001:db8:1::1]:44904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=984870lb # [ 97.267987] envoy[909]: conn peer=[2001:db8:1::4]:34036 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=68710 tx=3371 ms=692384871worker2 # [ 98.223120] nix-grpc-daemon[776]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4872worker2 # [ 98.223530] nix-grpc-daemon[776]: E0000 00:00:1789908464.159891 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4873worker2 # [ 98.223712] nix-grpc-daemon[776]: E0000 00:00:1789908464.160203 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4874worker2 # [ 98.795649] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4875worker2 # [ 98.803293] systemd[1]: Stopped Nix worker-protocol over gRPC.4876worker2 # [ 98.810438] systemd[1]: nix-grpc-daemon.service: Consumed 1.997s CPU time over 1min 11.084s wall clock time, 48.1M memory peak, 53.2K incoming IP traffic, 153.6K outgoing IP traffic.4877worker2 # [ 98.826015] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4878worker2 # [ 98.831535] systemd[1]: Closed Nix worker-protocol over gRPC.4879worker2 # [ 98.834950] systemd[1]: nix-grpc-daemon.socket: Consumed 789us CPU time over 1min 19.975s wall clock time, 256K memory peak, 64K incoming IP traffic, 54.7K outgoing IP traffic.4880lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4881lb: must succeed: curl -sf localhost:9901/clusters4882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4883lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4884client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24885worker1 # [ 99.194642] nix-grpc-daemon[786]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415584886client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4887worker1 # [ 99.202832] nix-grpc-daemon[786]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:415584888client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.28 seconds)4889worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4890worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4891worker2 # [ 99.321920] systemd[1]: Listening on Nix worker-protocol over gRPC.4892worker1 # [ 99.423777] systemd[1]: Stopping Nix worker-protocol over gRPC...4893worker2 # [ 99.784278] systemd[1]: Starting scheduler-token.service...4894worker2 # [ 99.890159] systemd[1]: scheduler-token.service: Deactivated successfully.4895worker2 # [ 99.896151] systemd[1]: Finished scheduler-token.service.4896worker2 # [ 99.897550] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 108ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4897worker2 # [ 99.905128] systemd[1]: Starting Nix worker-protocol over gRPC...4898worker1 # [ 99.973356] nix-grpc-daemon[786]: ts=2026-09-20T12:47:45Z level=info event=signal_exit4899worker2 # [ 100.090087] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4900worker2 # [ 100.092785] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4901worker2 # [ 100.096992] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14902worker2 # [ 100.105533] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4903worker2 # [ 100.107365] systemd[1]: Started Nix worker-protocol over gRPC.4904worker2 # [ 100.134331] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514905worker1 # [ 100.159714] nix-grpc-daemon[786]: ts=2026-09-20T12:47:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:415444906worker1 # [ 100.165684] nix-grpc-daemon[786]: ts=2026-09-20T12:47:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04907worker1 # [ 100.239179] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_yield4908lb # [ 99.897639] niks3-server[1094]: 2026/09/20 12:47:45 INFO lead: released remote=192.168.1.3:353104909worker2 # [ 100.220129] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_restarting addr=lb:500514910worker1 # [ 100.248356] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415444911worker1 # [ 100.255687] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=worker_gone addr=192.168.1.4:500514912worker1 # [ 100.259932] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_restarting addr=lb:500514913worker1 # [ 100.264322] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415444914worker1 # [ 100.268705] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=worker_gone addr=192.168.1.3:500514915worker2 # [ 100.421856] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514916worker2 # [ 100.427579] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4917worker1 # [ 100.456995] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514918worker1 # [ 100.462435] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4919worker2 # [ 100.629161] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514920worker2 # [ 100.634947] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4921worker1 # [ 100.663368] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514922worker1 # [ 100.668736] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4923worker2 # [ 100.836077] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514924worker2 # [ 100.841426] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4925worker1 # [ 100.869632] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514926worker1 # [ 100.874998] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4927worker2 # [ 101.042418] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514928worker2 # [ 101.047668] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4929worker1 # [ 101.075949] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500514930worker1 # [ 101.081354] nix-grpc-daemon[786]: ts=2026-09-20T12:47:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4931worker1 # [ 101.245313] nix-grpc-daemon[786]: ts=2026-09-20T12:47:47Z level=info event=scheduler_restarting4932worker2 # [ 101.250811] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_connected addr=lb:500514933worker2 # [ 101.255120] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4934worker1 # [ 101.287138] nix-daemon[856]: reaped child process 858, status = succeeded4935worker1 # [ 101.307499] nix-grpc-daemon[786]: ts=2026-09-20T12:47:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4936worker2 # [ 101.451604] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_connected addr=lb:500514937worker2 # [ 101.457017] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4938worker2 # [ 101.658025] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_connected addr=lb:500514939worker2 # [ 101.663211] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4940worker2 # [ 101.864343] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_connected addr=lb:500514941worker2 # [ 101.869592] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4942worker2 # [ 102.070905] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_connected addr=lb:500514943worker2 # [ 102.076483] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4944worker1 # [ 102.135200] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4945worker1 # [ 102.135685] nix-grpc-daemon[786]: E0000 00:00:1789908467.948385 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4946worker1 # [ 102.211554] nix-grpc-daemon[786]: E0000 00:00:1789908468.024821 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4947worker2 # [ 102.279327] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_connected addr=lb:500514948worker2 # [ 102.283694] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4949worker2 # [ 102.480385] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_connected addr=lb:500514950worker2 # [ 102.486114] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4951worker2 # [ 102.687449] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_connected addr=lb:500514952worker2 # [ 102.692935] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4953worker1 # [ 102.749376] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4954worker1 # [ 102.755659] systemd[1]: Stopped Nix worker-protocol over gRPC.4955worker1 # [ 102.762656] systemd[1]: nix-grpc-daemon.service: Consumed 4.370s CPU time over 1min 15.155s wall clock time, 63.8M memory peak, 98.5K incoming IP traffic, 187.7K outgoing IP traffic.4956worker1 # [ 102.777431] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4957worker1 # [ 102.780913] systemd[1]: Closed Nix worker-protocol over gRPC.4958worker1 # [ 102.782476] systemd[1]: nix-grpc-daemon.socket: Consumed 798us CPU time over 1min 23.877s wall clock time, 256K memory peak, 104.5K incoming IP traffic, 84.7K outgoing IP traffic.4959lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4960lb: must succeed: curl -sf localhost:9901/clusters4961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4962lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4963client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24964worker2 # [ 102.894231] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_connected addr=lb:500514965worker2 # [ 102.899809] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4966worker2 # [ 103.100137] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400664967worker2 # [ 103.113567] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_connected addr=lb:500514968worker2 # [ 103.124298] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4969worker2 # [ 103.141217] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4970worker2 # [ 103.157051] nix-grpc-daemon[1029]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)4971worker2 # [ 103.197167] nix-grpc-daemon[1029]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)4972worker2 # [ 103.225879] nix-grpc-daemon[1029]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 3/5)4973worker2 # [ 103.312069] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_connected addr=lb:500514974worker2 # [ 103.316202] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4975worker2 # [ 103.478410] nix-grpc-daemon[1029]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 4/5)4976worker2 # [ 103.513253] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_connected addr=lb:500514977worker2 # [ 103.518571] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4978worker2 # [ 103.719697] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_connected addr=lb:500514979worker2 # [ 103.725149] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4980worker2 # [ 103.761054] nix-grpc-daemon[1029]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4981client # copying 2 paths...4982client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4983worker2 # [ 103.790041] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400664984client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4985worker2 # [ 103.805457] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4986worker2 # [ 103.841878] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=niks3_push_started pid=10854987worker2 # [ 103.928687] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_connected addr=lb:500514988worker2 # [ 103.938975] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker2 # [ 104.072635] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.009Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4990worker2 # [ 104.132174] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500514991worker2 # [ 104.148587] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker2 # [ 104.260999] nix-daemon[841]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4993worker2 # [ 104.280234] nix-daemon[841]: reaped child process 1101, status = succeeded4994lb # [ 103.973679] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received uploads request method=POST path=/api/pending_closures4995lb # [ 103.984161] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received uploads request method=POST path=/api/pending_closures4996worker2 # [ 104.316087] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.251Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4997worker2 # [ 104.317631] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.251Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4998worker2 # [ 104.319290] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.251Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4999worker2 # [ 104.335291] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500515000worker2 # [ 104.337346] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5001lb # [ 104.038428] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5002lb # [ 104.050588] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5003lb # [ 104.062974] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5004lb # [ 104.073792] niks3-server[1094]: 2026/09/20 12:47:50 INFO Signed narinfos id=25 count=15005worker2 # [ 104.394382] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.329Z level=INFO msg="Uploading 3 narinfos"5006lb # [ 104.075757] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5007lb # [ 104.085722] niks3-server[1094]: 2026/09/20 12:47:50 INFO Signed narinfos id=26 count=25008lb # [ 104.087665] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5009lb # [ 104.094926] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5010lb # [ 104.109313] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5011lb # [ 104.117730] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5012lb # [ 104.121420] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5013lb # [ 104.124285] niks3-server[1094]: 2026/09/20 12:47:50 INFO Completed upload id=255014lb # [ 104.125630] niks3-server[1094]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5015worker2 # [ 104.447568] nix-grpc-daemon[1085]: time=2026-09-20T12:47:50.382Z level=INFO msg="Upload complete. (367ms)"5016lb # [ 104.128939] niks3-server[1094]: 2026/09/20 12:47:50 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5017lb # [ 104.131797] niks3-server[1094]: 2026/09/20 12:47:50 INFO Completed upload id=265018worker2 # [ 104.456488] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 paths=2 nar_bytes_in=3125019worker2 # [ 104.474488] nix-daemon[841]: reaped child process 1083, status = succeeded5020client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.64 seconds)5021worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5022worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)5023worker2 # [ 104.538583] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500515024worker2 # [ 104.543147] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5025worker1 # [ 104.642949] systemd[1]: Listening on Nix worker-protocol over gRPC.5026worker1 # [ 104.654180] systemd[1]: Starting Nix worker-protocol over gRPC...5027worker2 # [ 104.744498] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500515028worker2 # [ 104.749798] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker1 # [ 104.852540] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5030worker1 # [ 104.855428] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5031worker1 # [ 104.859762] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15032worker1 # [ 104.868073] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5033worker1 # [ 104.884847] systemd[1]: Started Nix worker-protocol over gRPC.5034lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5035lb: must succeed: curl -sf localhost:9901/clusters5036lb # [ 104.543392] niks3-server[1094]: 2026/09/20 12:47:50 INFO lead: acquired remote=192.168.1.3:549205037worker1 # [ 104.890852] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=scheduler_take_over5038worker1 # [ 104.916211] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500515039worker1 # [ 104.921710] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:558205040worker1 # [ 104.923658] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05041worker2 # [ 104.950973] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500515042worker1 # [ 104.984130] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=oidc_jwks provider=mock keys=15043worker1 # [ 104.988380] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:558205044lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5045lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.11 seconds)5046(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.97 seconds)5047subtest: scheduler down: the other node takes the lock and keeps it5048lb: must succeed: curl -sf localhost:9901/clusters5049worker1 # [ 104.992668] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05050lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5051lb: must succeed: curl -sf localhost:9901/clusters5052lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5053lb: must succeed: curl -sf localhost:9901/clusters5054lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5055lb: must succeed: curl -sf localhost:9901/clusters5056lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5057worker1 # [ 105.279677] systemd[1]: Stopping Nix worker-protocol over gRPC...5058worker1 # [ 105.876387] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=signal_exit5059worker1 # [ 105.883906] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=scheduler_yield5060lb # [ 105.545449] niks3-server[1094]: 2026/09/20 12:47:51 INFO lead: released remote=192.168.1.3:549205061worker2 # [ 105.869082] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:51Z level=info event=scheduler_restarting addr=lb:500515062worker1 # [ 105.894177] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:558205063worker1 # [ 105.901411] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=worker_gone addr=192.168.1.4:500515064worker1 # [ 105.905601] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=scheduler_restarting addr=lb:500515065worker1 # [ 105.909622] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:558205066worker1 # [ 105.915807] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=worker_gone addr=192.168.1.3:500515067worker2 # [ 106.069520] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515068worker2 # [ 106.074890] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5069worker1 # [ 106.101900] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500515070worker1 # [ 106.107175] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5071worker2 # [ 106.276320] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515072worker2 # [ 106.281676] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5073worker1 # [ 106.308068] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515074worker1 # [ 106.313314] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5075worker2 # [ 106.482869] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515076worker2 # [ 106.488240] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5077worker1 # [ 106.514435] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515078worker1 # [ 106.519934] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5079worker2 # [ 106.689425] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515080worker2 # [ 106.694738] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5081worker1 # [ 106.721266] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515082worker1 # [ 106.726694] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5083worker1 # [ 106.893463] nix-grpc-daemon[1260]: ts=2026-09-20T12:47:52Z level=info event=scheduler_restarting5084worker1 # [ 106.916755] nix-grpc-daemon[1260]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5085worker2 # [ 106.898060] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500515086worker1 # [ 106.922132] nix-grpc-daemon[1260]: E0000 00:00:1789908472.730070 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5087worker2 # [ 106.902550] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5088worker1 # [ 106.930015] nix-grpc-daemon[1260]: E0000 00:00:1789908472.735419 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5089worker2 # [ 107.099167] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500515090worker2 # [ 107.104833] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5091worker2 # [ 107.305939] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500515092worker2 # [ 107.311515] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5093worker2 # [ 107.513058] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500515094worker2 # [ 107.518875] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5095lb # [ 107.251709] envoy[909]: conn peer=[2001:db8:1::1]:44914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=265096lb # [ 107.256809] envoy[909]: conn peer=192.168.1.3:33734 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13527 tx=3728 ms=724725097lb # [ 107.261850] envoy[909]: conn peer=[2001:db8:1::1]:53558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=282 ms=13705098lb # [ 107.269225] envoy[909]: conn peer=[2001:db8:1::3]:37612 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1028 tx=349 ms=20335099worker2 # [ 107.720428] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500515100worker2 # [ 107.725990] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101worker2 # [ 107.929022] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500515102worker2 # [ 107.934344] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5103worker2 # [ 108.135256] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500515104worker2 # [ 108.141038] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5105worker2 # [ 108.342322] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500515106worker2 # [ 108.348262] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5107worker2 # [ 108.549574] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500515108worker2 # [ 108.555609] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5109worker2 # [ 108.756903] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500515110worker2 # [ 108.762745] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5111worker1 # [ 108.899927] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5112worker1 # [ 108.907031] systemd[1]: Stopped Nix worker-protocol over gRPC.5113worker1 # [ 108.913400] systemd[1]: nix-grpc-daemon.service: Consumed 179ms CPU time over 4.245s wall clock time, 10.8M memory peak, 9.2K incoming IP traffic, 7.9K outgoing IP traffic.5114worker1 # [ 108.926140] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5115worker1 # [ 108.930925] systemd[1]: Closed Nix worker-protocol over gRPC.5116worker1 # [ 108.935205] systemd[1]: nix-grpc-daemon.socket: Consumed 947us CPU time over 4.294s wall clock time, 256K memory peak, 12.9K incoming IP traffic, 9.9K outgoing IP traffic.5117worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5118worker2 # [ 108.967315] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500515119worker2 # [ 108.975897] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5120worker2 # [ 109.167520] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500515121worker2 # [ 109.173434] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5122worker2 # [ 109.374547] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500515123worker2 # [ 109.380338] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5124worker2 # [ 109.581702] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500515125worker2 # [ 109.587458] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5126worker2 # [ 109.788814] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500515127worker2 # [ 109.794531] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5128worker2 # [ 109.998138] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500515129worker2 # [ 110.002501] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5130worker2 # [ 110.123983] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:56Z level=info event=scheduler_take_over5131lb # [ 109.804692] niks3-server[1094]: 2026/09/20 12:47:55 INFO lead: acquired remote=192.168.1.4:601665132worker2 # [ 110.199231] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500515133worker2 # [ 110.232200] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:56Z level=info event=oidc_jwks provider=mock keys=15134worker2 # [ 110.238072] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400805135worker2 # [ 110.243716] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05136worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 2.29 seconds)5137worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5138worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5139client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25140worker2 # [ 111.533802] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400665141client # don't know how to build these paths:5142client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5143worker2 # [ 111.545059] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400665144worker2 # [ 111.577161] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05145worker2 # [ 111.585172] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15146worker2 # [ 111.625988] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665147worker2 # [ 111.644810] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400665148client # copying 1 paths to 'lb:50051'...5149client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5150worker2 # [ 111.663360] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400665151worker2 # [ 111.679849] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5152worker2 # [ 111.889773] nix-daemon[841]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)5153worker2 # [ 111.911510] nix-daemon[841]: reaped child process 1134, status = succeeded5154worker2 # [ 112.101127] nix-daemon[841]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)5155worker2 # [ 112.120711] nix-daemon[841]: reaped child process 1139, status = succeeded5156lb # [ 111.813099] niks3-server[1094]: 2026/09/20 12:47:57 INFO Received uploads request method=POST path=/api/pending_closures5157worker2 # [ 112.148172] nix-grpc-daemon[1085]: time=2026-09-20T12:47:58.084Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5158worker2 # [ 112.149747] nix-grpc-daemon[1085]: time=2026-09-20T12:47:58.084Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5159lb # [ 111.857556] niks3-server[1094]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5160lb # [ 111.879131] niks3-server[1094]: 2026/09/20 12:47:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5161lb # [ 111.887792] niks3-server[1094]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5162lb # [ 111.896251] niks3-server[1094]: 2026/09/20 12:47:57 INFO Signed narinfos id=27 count=15163worker2 # [ 112.217314] nix-grpc-daemon[1085]: time=2026-09-20T12:47:58.154Z level=INFO msg="Uploading 1 narinfos"5164lb # [ 111.905435] niks3-server[1094]: 2026/09/20 12:47:57 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5165lb # [ 111.916598] niks3-server[1094]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5166lb # [ 111.926274] niks3-server[1094]: 2026/09/20 12:47:58 INFO Completed upload id=275167worker2 # [ 112.247055] nix-grpc-daemon[1085]: time=2026-09-20T12:47:58.184Z level=INFO msg="Upload complete. (549ms)"5168worker2 # [ 112.258333] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 paths=1 nar_bytes_in=5205169worker2 # [ 112.273177] nix-grpc-daemon[1029]: ts=2026-09-20T12:47:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665170worker2 # [ 112.286604] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5171worker2 # [ 112.293450] nix-daemon[841]: reaped child process 1129, status = succeeded5172client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5173worker2 # [ 112.305089] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5174client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5175client # farm-a> LOG-farm-a-on-standby5176worker2 # [ 113.747720] nix-daemon[841]: reaped child process 1143, status = succeeded5177worker2 # [ 113.932102] nix-daemon[841]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)5178worker2 # [ 113.953193] nix-daemon[841]: reaped child process 1161, status = succeeded5179lb # [ 113.645619] niks3-server[1094]: 2026/09/20 12:47:59 INFO Received uploads request method=POST path=/api/pending_closures5180worker2 # [ 113.984450] nix-grpc-daemon[1085]: time=2026-09-20T12:47:59.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5181worker2 # [ 113.986009] nix-grpc-daemon[1085]: time=2026-09-20T12:47:59.919Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5182lb # [ 113.698133] niks3-server[1094]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5183lb # [ 113.709266] niks3-server[1094]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5184lb # [ 113.717690] niks3-server[1094]: 2026/09/20 12:47:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5185lb # [ 113.723424] niks3-server[1094]: 2026/09/20 12:47:59 INFO Signed narinfos id=28 count=15186worker2 # [ 114.045132] nix-grpc-daemon[1085]: time=2026-09-20T12:47:59.981Z level=INFO msg="Uploading 1 narinfos"5187lb # [ 113.727804] niks3-server[1094]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5188lb # [ 113.740341] niks3-server[1094]: 2026/09/20 12:47:59 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5189lb # [ 113.747222] niks3-server[1094]: 2026/09/20 12:47:59 INFO Completed upload id=285190worker2 # [ 114.067673] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.004Z level=INFO msg="Upload complete. (346ms)"5191lb # [ 113.750266] niks3-server[1094]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5192worker2 # [ 114.091106] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15193worker2 # [ 114.107947] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05194worker2 # [ 114.111517] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15195worker2 # [ 114.116217] nix-daemon[841]: reaped child process 1141, status = succeeded5196worker2 # [ 114.138294] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665197worker2 # [ 114.156168] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400665198worker2 # [ 114.175326] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.112Z level=INFO msg="All 1 paths already cached"5199client # copying 1 paths to 'lb:50051'...5200client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5201worker2 # [ 114.187023] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400665202worker2 # [ 114.202075] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5203worker2 # [ 114.414529] nix-daemon[841]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5204worker2 # [ 114.434291] nix-daemon[841]: reaped child process 1170, status = succeeded5205worker2 # [ 114.620642] nix-daemon[841]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5206worker2 # [ 114.639631] nix-daemon[841]: reaped child process 1175, status = succeeded5207lb # [ 114.331640] niks3-server[1094]: 2026/09/20 12:48:00 INFO Received uploads request method=POST path=/api/pending_closures5208worker2 # [ 114.662787] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.599Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5209worker2 # [ 114.665062] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.601Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5210lb # [ 114.368111] niks3-server[1094]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5211worker2 # [ 114.704537] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.641Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 114.388834] niks3-server[1094]: 2026/09/20 12:48:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5213lb # [ 114.398135] niks3-server[1094]: 2026/09/20 12:48:00 INFO Signed narinfos id=29 count=15214lb # [ 114.403716] niks3-server[1094]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5215lb # [ 114.413619] niks3-server[1094]: 2026/09/20 12:48:00 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5216worker2 # [ 114.738550] nix-grpc-daemon[1085]: time=2026-09-20T12:48:00.675Z level=INFO msg="Upload complete. (517ms)"5217lb # [ 114.418718] niks3-server[1094]: 2026/09/20 12:48:00 INFO Completed upload id=295218lb # [ 114.421772] niks3-server[1094]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5219worker2 # [ 114.748328] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 paths=1 nar_bytes_in=6405220worker2 # [ 114.764528] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665221worker2 # [ 114.771829] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5222worker2 # [ 114.777933] nix-daemon[841]: reaped child process 1165, status = succeeded5223client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5224client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5225worker2 # [ 114.791229] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5226client # farm-b> LOG-farm-b-on-standby5227worker2 # [ 115.847832] nix-daemon[841]: reaped child process 1179, status = succeeded5228worker2 # [ 116.033063] nix-daemon[841]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5229worker2 # [ 116.053938] nix-daemon[841]: reaped child process 1197, status = succeeded5230lb # [ 115.749725] niks3-server[1094]: 2026/09/20 12:48:01 INFO Received uploads request method=POST path=/api/pending_closures5231worker2 # [ 116.081789] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5232worker2 # [ 116.083582] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.017Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5233lb # [ 115.792407] niks3-server[1094]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5234lb # [ 115.805022] niks3-server[1094]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5235lb # [ 115.815828] niks3-server[1094]: 2026/09/20 12:48:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5236worker2 # [ 116.142043] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.078Z level=INFO msg="Uploading 1 narinfos"5237lb # [ 115.821624] niks3-server[1094]: 2026/09/20 12:48:01 INFO Signed narinfos id=30 count=15238lb # [ 115.828267] niks3-server[1094]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5239lb # [ 115.836993] niks3-server[1094]: 2026/09/20 12:48:01 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5240lb # [ 115.843789] niks3-server[1094]: 2026/09/20 12:48:01 INFO Completed upload id=305241lb # [ 115.847104] niks3-server[1094]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5242worker2 # [ 116.163940] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.101Z level=INFO msg="Upload complete. (351ms)"5243worker2 # [ 116.188837] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15244worker2 # [ 116.200303] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05245worker2 # [ 116.203870] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15246worker2 # [ 116.210188] nix-daemon[841]: reaped child process 1177, status = succeeded5247worker2 # [ 116.230904] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665248worker2 # [ 116.248674] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400665249worker2 # [ 116.265481] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.202Z level=INFO msg="All 2 paths already cached"5250client # copying 1 paths to 'lb:50051'...5251client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5252worker2 # [ 116.277138] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400665253worker2 # [ 116.292126] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5254worker2 # [ 116.507463] nix-daemon[841]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5255worker2 # [ 116.527331] nix-daemon[841]: reaped child process 1206, status = succeeded5256worker2 # [ 116.711193] nix-daemon[841]: accepted connection from pid 1208, user nix-grpc-daemon (trusted)5257worker2 # [ 116.729723] nix-daemon[841]: reaped child process 1211, status = succeeded5258lb # [ 116.423212] niks3-server[1094]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures5259worker2 # [ 116.754744] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.691Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5260worker2 # [ 116.756441] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.691Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5261lb # [ 116.455239] niks3-server[1094]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5262lb # [ 116.468717] niks3-server[1094]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5263worker2 # [ 116.796177] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.733Z level=INFO msg="Uploading 1 narinfos"5264lb # [ 116.479920] niks3-server[1094]: 2026/09/20 12:48:02 INFO Signed narinfos id=31 count=15265lb # [ 116.482832] niks3-server[1094]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5266lb # [ 116.494442] niks3-server[1094]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5267lb # [ 116.500978] niks3-server[1094]: 2026/09/20 12:48:02 INFO Completed upload id=315268worker2 # [ 116.821588] nix-grpc-daemon[1085]: time=2026-09-20T12:48:02.758Z level=INFO msg="Upload complete. (511ms)"5269lb # [ 116.503513] niks3-server[1094]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5270worker2 # [ 116.832058] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 paths=1 nar_bytes_in=7685271worker2 # [ 116.843199] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665272worker2 # [ 116.851102] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5273worker2 # [ 116.856182] nix-daemon[841]: reaped child process 1201, status = succeeded5274client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5275worker2 # [ 116.872205] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5276client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5277worker2 # [ 117.493508] nix-daemon[841]: reaped child process 1215, status = succeeded5278client # farm-top> LOG-farm-top-on-standby5279worker2 # [ 117.659957] nix-daemon[841]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5280worker2 # [ 117.682473] nix-daemon[841]: reaped child process 1233, status = succeeded5281lb # [ 117.374090] niks3-server[1094]: 2026/09/20 12:48:03 INFO Received uploads request method=POST path=/api/pending_closures5282worker2 # [ 117.707091] nix-grpc-daemon[1085]: time=2026-09-20T12:48:03.642Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5283worker2 # [ 117.708828] nix-grpc-daemon[1085]: time=2026-09-20T12:48:03.642Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5284lb # [ 117.408115] niks3-server[1094]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5285lb # [ 117.428330] niks3-server[1094]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5286lb # [ 117.438565] niks3-server[1094]: 2026/09/20 12:48:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5287worker2 # [ 117.764725] nix-grpc-daemon[1085]: time=2026-09-20T12:48:03.701Z level=INFO msg="Uploading 1 narinfos"5288lb # [ 117.446413] niks3-server[1094]: 2026/09/20 12:48:03 INFO Signed narinfos id=32 count=15289lb # [ 117.451814] niks3-server[1094]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5290lb # [ 117.460860] niks3-server[1094]: 2026/09/20 12:48:03 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5291lb # [ 117.468384] niks3-server[1094]: 2026/09/20 12:48:03 INFO Completed upload id=325292worker2 # [ 117.789042] nix-grpc-daemon[1085]: time=2026-09-20T12:48:03.726Z level=INFO msg="Upload complete. (307ms)"5293lb # [ 117.471417] niks3-server[1094]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5294worker2 # [ 117.801292] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15295worker2 # [ 117.821296] nix-daemon[841]: reaped child process 1213, status = succeeded5296client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.59 seconds)5297client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5298client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5299worker1 # [ 118.223315] systemd[1]: Listening on Nix worker-protocol over gRPC.5300worker1 # [ 118.235437] systemd[1]: Starting Nix worker-protocol over gRPC...5301worker1 # [ 118.442892] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5302worker1 # [ 118.445618] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5303worker1 # [ 118.449993] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15304worker1 # [ 118.458569] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5305worker1 # [ 118.464623] systemd[1]: Started Nix worker-protocol over gRPC.5306lb: must succeed: curl -sf localhost:9901/clusters5307worker1 # [ 118.486581] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:04Z level=info event=scheduler_connected addr=lb:500515308worker2 # [ 118.469144] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400805309worker2 # [ 118.474958] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05310lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)5311worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5312worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5313lb: must succeed: curl -sf localhost:9901/clusters5314lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5315lb: must succeed: curl -sf localhost:9901/clusters5316lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5317worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5318worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5319(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.88 seconds)5320subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5321client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5322client # [ 118.940711] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5323client # Running as unit: rs.service; invocation ID: ac9e6062999748f7998528de59adecc35324client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5325worker1 # [ 119.206916] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:405465326worker1 # [ 119.223326] nix-daemon[856]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5327client # [ 119.235644] nix[1174]: don't know how to build these paths:5328client # [ 119.237367] nix[1174]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5329worker2 # [ 119.238613] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05330worker2 # [ 119.244496] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15331worker2 # [ 119.284045] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665332worker2 # [ 119.300987] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400665333client # [ 119.332253] nix[1174]: copying 1 paths to 'lb:50051'...5334worker2 # [ 119.317528] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400665335client # [ 119.334701] nix[1174]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5336worker2 # [ 119.335946] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5337worker2 # [ 119.560698] nix-daemon[841]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5338worker2 # [ 119.580713] nix-daemon[841]: reaped child process 1249, status = succeeded5339worker2 # [ 119.762751] nix-daemon[841]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5340worker2 # [ 119.781607] nix-daemon[841]: reaped child process 1254, status = succeeded5341lb # [ 119.477894] niks3-server[1094]: 2026/09/20 12:48:05 INFO Received uploads request method=POST path=/api/pending_closures5342worker2 # [ 119.808449] nix-grpc-daemon[1085]: time=2026-09-20T12:48:05.744Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5343worker2 # [ 119.810771] nix-grpc-daemon[1085]: time=2026-09-20T12:48:05.744Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5344lb # [ 119.516482] niks3-server[1094]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5345lb # [ 119.529703] niks3-server[1094]: 2026/09/20 12:48:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5346worker2 # [ 119.856208] nix-grpc-daemon[1085]: time=2026-09-20T12:48:05.793Z level=INFO msg="Uploading 1 narinfos"5347lb # [ 119.541479] niks3-server[1094]: 2026/09/20 12:48:05 INFO Signed narinfos id=33 count=15348lb # [ 119.546792] niks3-server[1094]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5349lb # [ 119.555149] niks3-server[1094]: 2026/09/20 12:48:05 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5350lb # [ 119.561506] niks3-server[1094]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5351lb # [ 119.567151] niks3-server[1094]: 2026/09/20 12:48:05 INFO Completed upload id=335352worker2 # [ 119.887831] nix-grpc-daemon[1085]: time=2026-09-20T12:48:05.824Z level=INFO msg="Upload complete. (533ms)"5353worker2 # [ 119.896527] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=0 paths=1 nar_bytes_in=4165354worker2 # [ 119.916619] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400665355worker2 # [ 119.923761] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5356worker2 # [ 119.929594] nix-daemon[841]: reaped child process 1244, status = succeeded5357client # [ 119.955680] nix[1174]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5358worker2 # [ 119.945070] nix-daemon[841]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5359lb: must succeed: curl -sf localhost:9901/clusters5360lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5361lb: must succeed: curl -sf localhost:9901/clusters5362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5363lb: must succeed: curl -sf localhost:9901/clusters5364lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5365lb: must succeed: curl -sf localhost:9901/clusters5366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5367worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5368worker1 # [ 122.011422] systemd-journald[275]: Received client request to rotate journal, rotating.5369worker1 # [ 122.034452] systemd-journald[275]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e3c409fba0da4c218fe23f411d35f86f.5370worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.11 seconds)5371worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5372worker2 # [ 122.032441] systemd[1]: Stopping Nix worker-protocol over gRPC...5373worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5374??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5375 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735376client: must succeed: sleep 5.05377worker2 # [ 122.122478] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:08Z level=info event=draining5378client: (finished: must succeed: sleep 5.0, in 5.07 seconds)5379worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5380lb # [ 127.254117] envoy[909]: conn peer=[2001:db8:1::1]:46038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9593 tx=3869 ms=62835381worker2 # [ 127.595901] nix-daemon[841]: reaped child process 1258, status = succeeded5382worker2 # [ 127.777022] nix-daemon[841]: accepted connection from pid 1298, user nix-grpc-daemon (trusted)5383worker2 # [ 127.797396] nix-daemon[841]: reaped child process 1301, status = succeeded5384lb # [ 127.491622] niks3-server[1094]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures5385worker2 # [ 127.822811] nix-grpc-daemon[1085]: time=2026-09-20T12:48:13.759Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5386worker2 # [ 127.828364] nix-grpc-daemon[1085]: time=2026-09-20T12:48:13.761Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5387lb # [ 127.540160] niks3-server[1094]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5388lb # [ 127.552385] niks3-server[1094]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5389lb # [ 127.560732] niks3-server[1094]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5390worker2 # [ 127.885962] nix-grpc-daemon[1085]: time=2026-09-20T12:48:13.822Z level=INFO msg="Uploading 1 narinfos"5391lb # [ 127.569032] niks3-server[1094]: 2026/09/20 12:48:13 INFO Signed narinfos id=34 count=15392lb # [ 127.574696] niks3-server[1094]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5393lb # [ 127.583753] niks3-server[1094]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5394lb # [ 127.589017] niks3-server[1094]: 2026/09/20 12:48:13 INFO Completed upload id=345395worker2 # [ 127.909061] nix-grpc-daemon[1085]: time=2026-09-20T12:48:13.846Z level=INFO msg="Upload complete. (346ms)"5396lb # [ 127.591440] niks3-server[1094]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5397worker2 # [ 127.925312] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40066 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15398worker2 # [ 127.941953] nix-daemon[841]: reaped child process 1256, status = succeeded5399client # [ 127.979930] systemd[1]: rs.service: Deactivated successfully.5400client # [ 127.985464] systemd[1]: rs.service: Consumed 170ms CPU time over 9.042s wall clock time, 37M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5401worker2 # [ 128.123234] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=signal_exit5402worker2 # [ 128.148703] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_yield5403worker1 # [ 128.178539] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:13Z level=info event=scheduler_restarting addr=lb:500515404lb # [ 127.833243] niks3-server[1094]: 2026/09/20 12:48:13 INFO lead: released remote=192.168.1.4:601665405worker2 # [ 128.159988] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400805406worker2 # [ 128.166689] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=worker_gone addr=192.168.1.3:500515407worker2 # [ 128.170957] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_restarting addr=lb:500515408worker2 # [ 128.174984] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400805409worker2 # [ 128.179785] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=worker_gone addr=192.168.1.4:500515410worker1 # [ 128.378724] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515411worker2 # [ 128.362503] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 128.384460] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker2 # [ 128.368138] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414lb # [ 128.129270] niks3-server[1094]: 2026/09/20 12:48:14 INFO lead: acquired remote=192.168.1.3:592305415worker1 # [ 128.476394] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_take_over5416worker1 # [ 128.585460] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515417worker2 # [ 128.569255] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515418worker1 # [ 128.594150] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419worker2 # [ 128.574556] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.17 seconds)5421client: waiting for success: ! systemctl is-active rs5422client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5423client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5424client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5425client: must fail: journalctl -u rs | grep -q 'reconnecting'5426client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5427lb: must succeed: curl -sf localhost:9901/clusters5428worker1 # [ 128.793820] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515429worker2 # [ 128.775810] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 128.799558] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker2 # [ 128.781272] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5432lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5433worker1 # [ 129.000418] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515434worker2 # [ 128.984574] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_connected addr=lb:500515435worker1 # [ 129.005868] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker2 # [ 128.988745] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker2 # [ 129.154686] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:15Z level=info event=scheduler_restarting5438worker1 # [ 129.208976] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_connected addr=lb:500515439worker1 # [ 129.213360] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker2 # [ 129.203915] nix-grpc-daemon[1029]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5441worker2 # [ 129.212882] nix-grpc-daemon[1029]: E0000 00:00:1789908495.139996 1072 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5442worker2 # [ 129.219802] nix-grpc-daemon[1029]: E0000 00:00:1789908495.140562 1072 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5443worker2 # [ 129.226140] nix-grpc-daemon[1029]: ts=2026-09-20T12:48:15Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5444worker2 # [ 129.231177] nix-daemon[841]: reaped child process 1078, status = succeeded5445worker1 # [ 129.410200] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_connected addr=lb:500515446worker1 # [ 129.415654] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker1 # [ 129.616748] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_connected addr=lb:500515448worker1 # [ 129.622223] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5449worker1 # [ 129.823155] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_connected addr=lb:500515450worker1 # [ 129.828692] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5451lb: must succeed: curl -sf localhost:9901/clusters5452lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5453worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5454worker1 # [ 130.029727] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=scheduler_connected addr=lb:500515455worker1 # [ 130.039254] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:608425456worker1 # [ 130.043153] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05457worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5458worker2 # [ 130.698484] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5459worker2 # [ 130.701669] systemd[1]: Stopped Nix worker-protocol over gRPC.5460worker2 # [ 130.707401] systemd[1]: nix-grpc-daemon.service: Consumed 2.103s CPU time over 30.791s wall clock time, 34.6M memory peak, 62.6K incoming IP traffic, 146.2K outgoing IP traffic.5461worker2 # [ 130.725034] systemd[1]: Starting scheduler-token.service...5462worker2 # [ 130.825396] systemd[1]: scheduler-token.service: Deactivated successfully.5463worker2 # [ 130.832526] systemd[1]: Finished scheduler-token.service.5464worker2 # [ 130.834929] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 104ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5465worker2 # [ 130.840701] systemd[1]: Starting Nix worker-protocol over gRPC...5466worker2 # [ 131.020253] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5467worker2 # [ 131.022840] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5468lb: must succeed: curl -sf localhost:9901/clusters5469worker2 # [ 131.026981] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15470worker2 # [ 131.035082] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5471worker2 # [ 131.039080] systemd[1]: Started Nix worker-protocol over gRPC.5472worker2 # [ 131.062769] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:16Z level=info event=scheduler_connected addr=lb:500515473worker1 # [ 131.099397] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:16Z level=info event=oidc_jwks provider=mock keys=15474worker1 # [ 131.104118] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:608425475worker1 # [ 131.109428] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5477worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5478worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5479(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.34 seconds)5480subtest: scheduler restart mid-build: build finishes, no double build5481client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5482client # [ 131.279614] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5483client # Running as unit: mid.service; invocation ID: 1046ae0041ca4d73b8ea2397abc6c5b75484client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5485worker1 # [ 131.545118] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:405465486client # [ 131.547339] nix[1210]: don't know how to build these paths:5487client # [ 131.548691] nix[1210]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5488worker1 # [ 131.564241] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:17Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05489worker1 # [ 131.571781] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:17Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15490worker1 # [ 131.612910] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405465491worker1 # [ 131.631480] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5492worker1 # [ 131.657317] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)5493worker1 # [ 131.696967] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 264 ms (attempt 3/5)5494worker1 # [ 131.968902] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 737 ms (attempt 4/5)5495worker1 # [ 132.715398] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5496worker1 # [ 132.740919] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405465497client # [ 132.749393] nix[1210]: copying 1 paths to 'lb:50051'...5498worker1 # [ 132.759733] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405465499client # [ 132.754524] nix[1210]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5500worker1 # [ 132.779114] nix-daemon[856]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5501worker1 # [ 132.808514] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:18Z level=info event=niks3_push_started pid=14405502worker1 # [ 133.031566] nix-grpc-daemon[1440]: time=2026-09-20T12:48:18.844Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5503worker1 # [ 133.214472] nix-daemon[856]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)5504worker1 # [ 133.233437] nix-daemon[856]: reaped child process 1456, status = succeeded5505worker1 # [ 133.414219] nix-daemon[856]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5506worker1 # [ 133.432718] nix-daemon[856]: reaped child process 1461, status = succeeded5507lb # [ 133.102059] niks3-server[1094]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures5508worker1 # [ 133.456093] nix-grpc-daemon[1440]: time=2026-09-20T12:48:19.268Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5509worker1 # [ 133.457606] nix-grpc-daemon[1440]: time=2026-09-20T12:48:19.268Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5510lb # [ 133.137657] niks3-server[1094]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5511lb # [ 133.150695] niks3-server[1094]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5512worker1 # [ 133.500328] nix-grpc-daemon[1440]: time=2026-09-20T12:48:19.313Z level=INFO msg="Uploading 1 narinfos"5513lb # [ 133.164865] niks3-server[1094]: 2026/09/20 12:48:19 INFO Signed narinfos id=35 count=15514lb # [ 133.171885] niks3-server[1094]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5515lb # [ 133.182598] niks3-server[1094]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5516worker1 # [ 133.532088] nix-grpc-daemon[1440]: time=2026-09-20T12:48:19.345Z level=INFO msg="Upload complete. (495ms)"5517lb # [ 133.191585] niks3-server[1094]: 2026/09/20 12:48:19 INFO Completed upload id=355518worker1 # [ 133.541261] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40546 duration_s=0 paths=1 nar_bytes_in=4245519worker1 # [ 133.549875] nix-grpc-daemon[1324]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405465520worker1 # [ 133.558235] nix-grpc-daemon[1440]: time=2026-09-20T12:48:19.363Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n"5521worker1 # [ 133.568295] nix-daemon[856]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5522worker1 # [ 133.570594] nix-daemon[856]: reaped child process 1438, status = succeeded5523worker1 # [ 133.580910] nix-daemon[856]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5524client # [ 133.573254] nix[1210]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5525lb: must succeed: curl -sf localhost:9901/clusters5526lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5527lb: must succeed: curl -sf localhost:9901/clusters5528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5529worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5530worker1 # [ 135.316994] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1324 (nix-grpc-daemon) on client request.5531worker1 # [ 135.318661] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5532lb # [ 134.997311] niks3-server[1094]: 2026/09/20 12:48:21 INFO lead: released remote=192.168.1.3:592305533client # [ 135.342023] nix[1210]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5534worker2 # [ 135.326642] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5535client # [ 135.347470] nix[1210]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5536worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5537client: must succeed: sleep 5.05538worker1 # [ 135.362121] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5539worker1 # [ 135.363639] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5540worker1 # [ 135.374562] systemd[1]: nix-grpc-daemon.service: Consumed 606ms CPU time over 17.125s wall clock time, 31.6M memory peak, 16.6K incoming IP traffic, 23K outgoing IP traffic.5541worker1 # [ 135.383553] nix-daemon[856]: reaped child process 1463, status = succeeded5542worker1 # [ 135.385355] nix-daemon[856]: reaped child process 1380, status = succeeded5543worker1 # [ 135.391772] nix-daemon[1465]: unexpected Nix daemon error: error: interrupted by the user5544worker1 # [ 135.401775] nix-daemon[856]: reaped child process 1465, status = failed with exit code 15545worker1 # [ 135.511781] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5546worker1 # [ 135.522447] systemd[1]: Starting Nix worker-protocol over gRPC...5547worker2 # [ 135.527203] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:21Z level=info event=scheduler_connected addr=lb:500515548worker1 # [ 135.732578] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5549worker1 # [ 135.734584] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5550worker1 # [ 135.738839] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15551worker1 # [ 135.747124] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5552worker1 # [ 135.755767] systemd[1]: Started Nix worker-protocol over gRPC.5553worker1 # [ 135.761038] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=scheduler_take_over5554lb # [ 135.418506] niks3-server[1094]: 2026/09/20 12:48:21 INFO lead: acquired remote=192.168.1.3:541325555worker1 # [ 135.777701] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=scheduler_connected addr=lb:500515556worker1 # [ 135.781387] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:524065557worker1 # [ 135.783047] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05558worker1 # [ 135.784864] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=oidc_jwks provider=mock keys=15559worker1 # [ 135.786083] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:524065560worker1 # [ 135.787716] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05561worker1 # [ 135.858062] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05562worker1 # [ 135.863361] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15563worker1 # [ 135.917880] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524225564worker1 # [ 135.935935] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5565worker1 # [ 135.952278] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5566client # [ 135.963867] nix[1210]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5567worker1 # [ 135.975559] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5568lb # [ 137.254634] envoy[909]: conn peer=[2001:db8:1::1]:46042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2940 tx=1352 ms=87545569lb # [ 137.257254] envoy[909]: conn peer=[2001:db8:1::4]:41264 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=38535 tx=2207 ms=290635570lb # [ 137.259621] envoy[909]: conn peer=[2001:db8:1::3]:56144 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=618 ms=168725571client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5572client: waiting for success: ! systemctl is-active mid5573worker1 # [ 140.807044] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:26Z level=info event=niks3_push_started pid=15805574worker1 # [ 140.844955] nix-daemon[856]: reaped child process 1556, status = succeeded5575worker1 # [ 141.041484] nix-grpc-daemon[1580]: time=2026-09-20T12:48:26.854Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5576worker1 # [ 141.221455] nix-daemon[856]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)5577worker1 # [ 141.241640] nix-daemon[856]: reaped child process 1596, status = succeeded5578lb # [ 140.914039] niks3-server[1094]: 2026/09/20 12:48:26 INFO Received uploads request method=POST path=/api/pending_closures5579worker1 # [ 141.266152] nix-grpc-daemon[1580]: time=2026-09-20T12:48:27.077Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5580worker1 # [ 141.267714] nix-grpc-daemon[1580]: time=2026-09-20T12:48:27.077Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5581lb # [ 140.964665] niks3-server[1094]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5582lb # [ 140.977457] niks3-server[1094]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5583lb # [ 140.986139] niks3-server[1094]: 2026/09/20 12:48:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5584worker1 # [ 141.335705] nix-grpc-daemon[1580]: time=2026-09-20T12:48:27.147Z level=INFO msg="Uploading 1 narinfos"5585lb # [ 140.996260] niks3-server[1094]: 2026/09/20 12:48:27 INFO Signed narinfos id=36 count=15586lb # [ 141.012201] niks3-server[1094]: 2026/09/20 12:48:27 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5587lb # [ 141.103338] niks3-server[1094]: 2026/09/20 12:48:27 INFO Completed upload id=365588worker1 # [ 141.446277] nix-grpc-daemon[1580]: time=2026-09-20T12:48:27.259Z level=INFO msg="Upload complete. (400ms)"5589worker1 # [ 141.464455] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52422 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15590lb # [ 141.126518] niks3-server[1094]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5591lb # [ 141.134085] niks3-server[1094]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5592worker1 # [ 141.483994] nix-daemon[856]: reaped child process 1554, status = succeeded5593client # [ 141.501587] systemd[1]: mid.service: Deactivated successfully.5594client # [ 141.505977] systemd[1]: mid.service: Consumed 190ms CPU time over 10.225s wall clock time, 37M memory peak, 9.7K incoming IP traffic, 10.8K outgoing IP traffic.5595client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5596client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5597client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5598(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.73 seconds)5599subtest: interrupting the client stops the build on the worker5600client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5601client # [ 141.980420] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5602client # Running as unit: intr.service; invocation ID: cd6d1edc8ac5412aa05fd6e93a4359eb5603client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5604worker2 # [ 142.209324] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400525605worker2 # [ 142.224354] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5606client # [ 142.258829] nix[1246]: don't know how to build these paths:5607client # [ 142.261016] nix[1246]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5608worker1 # [ 142.281718] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:28Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05609worker1 # [ 142.287780] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:28Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15610worker2 # [ 142.305809] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525611worker2 # [ 142.324492] nix-grpc-daemon[1311]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5612worker2 # [ 142.362053] nix-grpc-daemon[1311]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5)5613worker2 # [ 142.523661] nix-grpc-daemon[1311]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 3/5)5614worker2 # [ 142.805500] nix-grpc-daemon[1311]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 484 ms (attempt 4/5)5615worker2 # [ 143.299541] nix-grpc-daemon[1311]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5616worker2 # [ 143.318417] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400525617worker2 # [ 143.331409] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400525618client # [ 143.348609] nix[1246]: copying 1 paths to 'lb:50051'...5619client # [ 143.348863] nix[1246]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5620worker2 # [ 143.340931] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5621worker2 # [ 143.362706] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:29Z level=info event=niks3_push_started pid=14035622worker2 # [ 143.568655] nix-grpc-daemon[1403]: time=2026-09-20T12:48:29.505Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5623worker2 # [ 143.753032] nix-daemon[841]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)5624worker2 # [ 143.772745] nix-daemon[841]: reaped child process 1419, status = succeeded5625worker2 # [ 143.954796] nix-daemon[841]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5626worker2 # [ 143.973271] nix-daemon[841]: reaped child process 1424, status = succeeded5627lb # [ 143.664915] niks3-server[1094]: 2026/09/20 12:48:29 INFO Received uploads request method=POST path=/api/pending_closures5628worker2 # [ 144.000076] nix-grpc-daemon[1403]: time=2026-09-20T12:48:29.936Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5629worker2 # [ 144.001610] nix-grpc-daemon[1403]: time=2026-09-20T12:48:29.936Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5630lb # [ 143.710226] niks3-server[1094]: 2026/09/20 12:48:29 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5631lb # [ 143.721699] niks3-server[1094]: 2026/09/20 12:48:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5632worker2 # [ 144.049987] nix-grpc-daemon[1403]: time=2026-09-20T12:48:29.986Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 143.734864] niks3-server[1094]: 2026/09/20 12:48:29 INFO Signed narinfos id=37 count=15634lb # [ 143.742394] niks3-server[1094]: 2026/09/20 12:48:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5635lb # [ 143.754154] niks3-server[1094]: 2026/09/20 12:48:29 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5636lb # [ 143.764482] niks3-server[1094]: 2026/09/20 12:48:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5637lb # [ 143.768883] niks3-server[1094]: 2026/09/20 12:48:29 INFO Completed upload id=375638worker2 # [ 144.091816] nix-grpc-daemon[1403]: time=2026-09-20T12:48:30.028Z level=INFO msg="Upload complete. (518ms)"5639worker2 # [ 144.101837] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=4245640worker2 # [ 144.116501] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525641worker2 # [ 144.122952] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5642worker2 # [ 144.132322] nix-daemon[841]: reaped child process 1401, status = succeeded5643client # [ 144.153078] nix[1246]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5644worker2 # [ 144.141071] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5645client: must succeed: systemctl kill -s INT intr5646worker2 # [ 145.745607] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:31Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5647client # [ 145.758849] systemd[1]: intr.service: Sent signal SIGINT to main process 1246 (nix) on client request.5648client # [ 145.769823] nix[1246]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5649client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5650client # [ 145.780304] nix[1246]: error: interrupted by the user5651worker2 # [ 145.766875] nix-daemon[841]: reaped child process 1426, status = succeeded5652worker2 # [ 145.770700] nix-daemon[1428]: unexpected Nix daemon error: error: interrupted by the user5653worker2 # [ 145.779117] nix-daemon[841]: reaped child process 1428, status = failed with exit code 15654client # [ 145.809707] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5655client # [ 145.812083] systemd[1]: intr.service: Failed with result 'exit-code'.5656client # [ 145.815571] systemd[1]: intr.service: Consumed 147ms CPU time over 3.833s wall clock time, 39.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.5657(finished: subtest: interrupting the client stops the build on the worker, in 3.98 seconds)5658subtest: a deploy mid-build drains: the build finishes, the new generation takes over5659client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5660client # [ 145.991749] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5661client # Running as unit: sw.service; invocation ID: e89006fc8ac74148b0a369c80aab2edd5662client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5663client # [ 146.239976] nix[1266]: don't know how to build these paths:5664client # [ 146.241036] nix[1266]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5665worker1 # [ 146.245797] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524225666worker1 # [ 146.262373] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05667worker1 # [ 146.270116] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15668worker1 # [ 146.307316] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524225669worker1 # [ 146.324584] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)5670worker1 # [ 146.332475] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 2/5)5671worker1 # [ 146.458525] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 3/5)5672worker1 # [ 146.559131] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 206 ms (attempt 4/5)5673worker1 # [ 146.773424] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5674worker1 # [ 146.795627] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524225675client # [ 146.804412] nix[1266]: copying 1 paths to 'lb:50051'...5676worker1 # [ 146.812799] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524225677client # [ 146.808502] nix[1266]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5678worker1 # [ 146.832134] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5679worker1 # [ 147.042743] nix-daemon[856]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)5680worker1 # [ 147.065162] nix-daemon[856]: reaped child process 1634, status = succeeded5681worker1 # [ 147.248848] nix-daemon[856]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)5682worker1 # [ 147.277043] nix-daemon[856]: reaped child process 1642, status = succeeded5683lb # [ 146.946013] niks3-server[1094]: 2026/09/20 12:48:33 INFO Received uploads request method=POST path=/api/pending_closures5684worker1 # [ 147.305159] nix-grpc-daemon[1580]: time=2026-09-20T12:48:33.117Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5685worker1 # [ 147.306686] nix-grpc-daemon[1580]: time=2026-09-20T12:48:33.117Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5686lb # [ 146.984902] niks3-server[1094]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5687worker1 # [ 147.341319] nix-grpc-daemon[1580]: time=2026-09-20T12:48:33.154Z level=INFO msg="Uploading 1 narinfos"5688lb # [ 146.998580] niks3-server[1094]: 2026/09/20 12:48:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5689lb # [ 147.009376] niks3-server[1094]: 2026/09/20 12:48:33 INFO Signed narinfos id=38 count=15690lb # [ 147.016188] niks3-server[1094]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5691lb # [ 147.024233] niks3-server[1094]: 2026/09/20 12:48:33 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5692lb # [ 147.037143] niks3-server[1094]: 2026/09/20 12:48:33 INFO Completed upload id=385693worker1 # [ 147.379788] nix-grpc-daemon[1580]: time=2026-09-20T12:48:33.193Z level=INFO msg="Upload complete. (529ms)"5694lb # [ 147.040090] niks3-server[1094]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5695worker1 # [ 147.388759] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52422 duration_s=0 paths=1 nar_bytes_in=4165696worker1 # [ 147.399440] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524225697worker1 # [ 147.405821] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5698worker1 # [ 147.410736] nix-daemon[856]: reaped child process 1629, status = succeeded5699worker1 # [ 147.421291] nix-daemon[856]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5700client # [ 147.414799] nix[1266]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5701lb # [ 147.254305] envoy[909]: conn peer=[2001:db8:1::1]:54934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3858 tx=1675 ms=99325702lb # [ 147.259954] envoy[909]: conn peer=[2001:db8:1::1]:45214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=757 ms=35835703worker1: must succeed: systemctl reload nix-grpc-daemon.service5704worker1 # [ 149.932368] systemd[1]: Reloading Nix worker-protocol over gRPC...5705worker1 # [ 149.985287] systemd[1]: Reloaded Nix worker-protocol over gRPC.5706worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5707worker1: must succeed: pgrep -f 'read -t [9]0 x'5708worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5709client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25710client # don't know how to build these paths:5711client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5712worker2 # [ 150.281852] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400525713worker2 # [ 150.289905] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400525714worker1 # [ 150.336904] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:36Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05715worker1 # [ 150.338825] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:36Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15716worker2 # [ 150.365406] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525717worker2 # [ 150.382393] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400525718client # copying 1 paths to 'lb:50051'...5719worker2 # [ 150.400146] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400525720client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5721worker2 # [ 150.425575] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5722worker2 # [ 150.637947] nix-daemon[841]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)5723worker2 # [ 150.659734] nix-daemon[841]: reaped child process 1481, status = succeeded5724worker1 # [ 150.755289] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:36Z level=info event=draining5725worker2 # [ 150.841991] nix-daemon[841]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5726worker2 # [ 150.862384] nix-daemon[841]: reaped child process 1486, status = succeeded5727lb # [ 150.553390] niks3-server[1094]: 2026/09/20 12:48:36 INFO Received uploads request method=POST path=/api/pending_closures5728worker2 # [ 150.885492] nix-grpc-daemon[1403]: time=2026-09-20T12:48:36.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5729worker2 # [ 150.887161] nix-grpc-daemon[1403]: time=2026-09-20T12:48:36.821Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5730lb # [ 150.583536] niks3-server[1094]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5731lb # [ 150.601189] niks3-server[1094]: 2026/09/20 12:48:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5732worker2 # [ 150.928956] nix-grpc-daemon[1403]: time=2026-09-20T12:48:36.865Z level=INFO msg="Uploading 1 narinfos"5733lb # [ 150.615239] niks3-server[1094]: 2026/09/20 12:48:36 INFO Signed narinfos id=39 count=15734lb # [ 150.623627] niks3-server[1094]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5735lb # [ 150.632522] niks3-server[1094]: 2026/09/20 12:48:36 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5736lb # [ 150.648790] niks3-server[1094]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5737lb # [ 150.664326] niks3-server[1094]: 2026/09/20 12:48:36 INFO Completed upload id=395738worker2 # [ 150.985003] nix-grpc-daemon[1403]: time=2026-09-20T12:48:36.922Z level=INFO msg="Upload complete. (540ms)"5739worker2 # [ 150.995577] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=5285740worker2 # [ 151.012767] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525741worker2 # [ 151.022662] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5742worker2 # [ 151.032756] nix-daemon[841]: reaped child process 1476, status = succeeded5743client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5744worker2 # [ 151.043973] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5745client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5746client # farm-a> LOG-farm-a-during-drain5747worker2 # [ 152.079022] nix-daemon[841]: reaped child process 1490, status = succeeded5748worker2 # [ 152.248829] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)5749worker2 # [ 152.270736] nix-daemon[841]: reaped child process 1508, status = succeeded5750lb # [ 151.966179] niks3-server[1094]: 2026/09/20 12:48:38 INFO Received uploads request method=POST path=/api/pending_closures5751worker2 # [ 152.320452] nix-grpc-daemon[1403]: time=2026-09-20T12:48:38.253Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5752worker2 # [ 152.322165] nix-grpc-daemon[1403]: time=2026-09-20T12:48:38.253Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5753lb # [ 152.029245] niks3-server[1094]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5754lb # [ 152.043405] niks3-server[1094]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5755lb # [ 152.053416] niks3-server[1094]: 2026/09/20 12:48:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5756worker2 # [ 152.381985] nix-grpc-daemon[1403]: time=2026-09-20T12:48:38.317Z level=INFO msg="Uploading 1 narinfos"5757lb # [ 152.063392] niks3-server[1094]: 2026/09/20 12:48:38 INFO Signed narinfos id=40 count=15758lb # [ 152.071261] niks3-server[1094]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5759lb # [ 152.083328] niks3-server[1094]: 2026/09/20 12:48:38 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5760lb # [ 152.329169] niks3-server[1094]: 2026/09/20 12:48:38 INFO Completed upload id=405761worker2 # [ 152.653605] nix-grpc-daemon[1403]: time=2026-09-20T12:48:38.590Z level=INFO msg="Upload complete. (593ms)"5762lb # [ 152.339440] niks3-server[1094]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5763worker2 # [ 152.665355] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15764worker1 # [ 152.695246] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:38Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05765worker1 # [ 152.697676] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:38Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15766worker2 # [ 152.688142] nix-daemon[841]: reaped child process 1488, status = succeeded5767worker2 # [ 152.727316] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525768worker2 # [ 152.744517] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400525769worker2 # [ 152.762697] nix-grpc-daemon[1403]: time=2026-09-20T12:48:38.699Z level=INFO msg="All 1 paths already cached"5770client # copying 1 paths to 'lb:50051'...5771worker2 # [ 152.772103] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400525772client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5773worker2 # [ 152.798555] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5774worker2 # [ 153.016104] nix-daemon[841]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)5775worker2 # [ 153.035310] nix-daemon[841]: reaped child process 1515, status = succeeded5776worker2 # [ 153.218301] nix-daemon[841]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5777worker2 # [ 153.237901] nix-daemon[841]: reaped child process 1520, status = succeeded5778lb # [ 152.927385] niks3-server[1094]: 2026/09/20 12:48:39 INFO Received uploads request method=POST path=/api/pending_closures5779worker2 # [ 153.263850] nix-grpc-daemon[1403]: time=2026-09-20T12:48:39.200Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5780worker2 # [ 153.265607] nix-grpc-daemon[1403]: time=2026-09-20T12:48:39.200Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5781lb # [ 152.968936] niks3-server[1094]: 2026/09/20 12:48:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5782worker2 # [ 153.296250] nix-grpc-daemon[1403]: time=2026-09-20T12:48:39.233Z level=INFO msg="Uploading 1 narinfos"5783lb # [ 152.982184] niks3-server[1094]: 2026/09/20 12:48:39 INFO Signed narinfos id=41 count=15784lb # [ 152.991654] niks3-server[1094]: 2026/09/20 12:48:39 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5785lb # [ 153.028400] niks3-server[1094]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5786lb # [ 153.044272] niks3-server[1094]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5787lb # [ 153.051108] niks3-server[1094]: 2026/09/20 12:48:39 INFO Completed upload id=415788worker2 # [ 153.365500] nix-grpc-daemon[1403]: time=2026-09-20T12:48:39.301Z level=INFO msg="Upload complete. (547ms)"5789lb # [ 153.055911] niks3-server[1094]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5790worker2 # [ 153.376571] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=6405791worker2 # [ 153.395962] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525792worker2 # [ 153.407189] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5793worker2 # [ 153.410036] nix-daemon[841]: reaped child process 1510, status = succeeded5794client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5795client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5796worker2 # [ 153.426308] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5797worker2 # [ 154.650279] nix-daemon[841]: reaped child process 1524, status = succeeded5798client # farm-b> LOG-farm-b-during-drain5799worker2 # [ 154.817989] nix-daemon[841]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)5800worker2 # [ 154.839632] nix-daemon[841]: reaped child process 1542, status = succeeded5801lb # [ 154.534945] niks3-server[1094]: 2026/09/20 12:48:40 INFO Received uploads request method=POST path=/api/pending_closures5802worker2 # [ 155.247855] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.182Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5803worker2 # [ 155.252712] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.182Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5804lb # [ 154.962408] niks3-server[1094]: 2026/09/20 12:48:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5805worker2 # [ 155.291238] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.227Z level=INFO msg="Uploading 1 narinfos"5806lb # [ 154.974605] niks3-server[1094]: 2026/09/20 12:48:41 INFO Signed narinfos id=42 count=15807lb # [ 154.980995] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5808lb # [ 154.994028] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5809lb # [ 155.002363] niks3-server[1094]: 2026/09/20 12:48:41 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5810lb # [ 155.018076] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5811lb # [ 155.029652] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5812lb # [ 155.046042] niks3-server[1094]: 2026/09/20 12:48:41 INFO Completed upload id=425813worker2 # [ 155.366644] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.303Z level=INFO msg="Upload complete. (728ms)"5814worker1 # [ 155.402484] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:41Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05815worker1 # [ 155.404832] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:41Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15816worker2 # [ 155.380893] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15817worker2 # [ 155.400538] nix-daemon[841]: reaped child process 1522, status = succeeded5818worker2 # [ 155.429558] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525819worker2 # [ 155.447014] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400525820worker2 # [ 155.462144] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.399Z level=INFO msg="All 2 paths already cached"5821client # copying 1 paths to 'lb:50051'...5822worker2 # [ 155.478479] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400525823client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5824worker2 # [ 155.502062] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5825worker2 # [ 155.711956] nix-daemon[841]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)5826worker2 # [ 155.734225] nix-daemon[841]: reaped child process 1549, status = succeeded5827worker2 # [ 155.908131] nix-daemon[841]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)5828worker2 # [ 155.928862] nix-daemon[841]: reaped child process 1554, status = succeeded5829lb # [ 155.625934] niks3-server[1094]: 2026/09/20 12:48:41 INFO Received uploads request method=POST path=/api/pending_closures5830worker2 # [ 155.961928] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.898Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5831worker2 # [ 155.963533] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.898Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5832lb # [ 155.660713] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5833lb # [ 155.675661] niks3-server[1094]: 2026/09/20 12:48:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5834worker2 # [ 156.002695] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.939Z level=INFO msg="Uploading 1 narinfos"5835lb # [ 155.687740] niks3-server[1094]: 2026/09/20 12:48:41 INFO Signed narinfos id=43 count=15836lb # [ 155.694283] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5837lb # [ 155.704481] niks3-server[1094]: 2026/09/20 12:48:41 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5838lb # [ 155.712568] niks3-server[1094]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5839lb # [ 155.717244] niks3-server[1094]: 2026/09/20 12:48:41 INFO Completed upload id=435840worker2 # [ 156.037416] nix-grpc-daemon[1403]: time=2026-09-20T12:48:41.974Z level=INFO msg="Upload complete. (517ms)"5841worker2 # [ 156.044351] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=7685842worker2 # [ 156.060612] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400525843worker2 # [ 156.070155] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5844worker2 # [ 156.083490] nix-daemon[841]: reaped child process 1544, status = succeeded5845client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5846worker2 # [ 156.092181] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5847client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5848client # farm-top> LOG-farm-top-during-drain5849worker2 # [ 156.929823] nix-daemon[841]: reaped child process 1558, status = succeeded5850worker2 # [ 157.111274] nix-daemon[841]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)5851worker2 # [ 157.134387] nix-daemon[841]: reaped child process 1576, status = succeeded5852lb # [ 156.825894] niks3-server[1094]: 2026/09/20 12:48:42 INFO Received uploads request method=POST path=/api/pending_closures5853worker2 # [ 157.160519] nix-grpc-daemon[1403]: time=2026-09-20T12:48:43.095Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5854worker2 # [ 157.162515] nix-grpc-daemon[1403]: time=2026-09-20T12:48:43.095Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5855lb # [ 156.867265] niks3-server[1094]: 2026/09/20 12:48:42 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5856worker2 # [ 157.211839] nix-grpc-daemon[1403]: time=2026-09-20T12:48:43.148Z level=INFO msg="Uploading 1 narinfos"5857lb # [ 156.886208] niks3-server[1094]: 2026/09/20 12:48:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5858lb # [ 156.896980] niks3-server[1094]: 2026/09/20 12:48:42 INFO Signed narinfos id=44 count=15859lb # [ 156.899019] niks3-server[1094]: 2026/09/20 12:48:42 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5860lb # [ 156.909097] niks3-server[1094]: 2026/09/20 12:48:42 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5861lb # [ 156.917466] niks3-server[1094]: 2026/09/20 12:48:42 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5862lb # [ 156.923439] niks3-server[1094]: 2026/09/20 12:48:42 INFO Completed upload id=445863worker2 # [ 157.243183] nix-grpc-daemon[1403]: time=2026-09-20T12:48:43.180Z level=INFO msg="Upload complete. (338ms)"5864lb # [ 156.926251] niks3-server[1094]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5865worker2 # [ 157.255869] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15866worker2 # [ 157.276870] nix-daemon[841]: reaped child process 1556, status = succeeded5867client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 7.29 seconds)5868client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5869client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.25 seconds)5870worker1: must succeed: pkill -f 'read -t [9]0 x'5871lb # [ 157.256849] envoy[909]: conn peer=[2001:db8:1::1]:45224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9612 tx=3874 ms=69935872worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5873client: waiting for success: ! systemctl is-active sw5874worker1 # [ 157.889116] nix-daemon[856]: reaped child process 1647, status = succeeded5875worker1 # [ 158.078839] nix-daemon[856]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)5876worker1 # [ 158.103494] nix-daemon[856]: reaped child process 1696, status = succeeded5877lb # [ 157.772178] niks3-server[1094]: 2026/09/20 12:48:43 INFO Received uploads request method=POST path=/api/pending_closures5878worker1 # [ 158.140677] nix-grpc-daemon[1580]: time=2026-09-20T12:48:43.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5879worker1 # [ 158.142612] nix-grpc-daemon[1580]: time=2026-09-20T12:48:43.952Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5880lb # [ 157.830234] niks3-server[1094]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5881lb # [ 157.846849] niks3-server[1094]: 2026/09/20 12:48:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5882worker1 # [ 158.196254] nix-grpc-daemon[1580]: time=2026-09-20T12:48:44.008Z level=INFO msg="Uploading 1 narinfos"5883lb # [ 157.859224] niks3-server[1094]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5884lb # [ 157.867859] niks3-server[1094]: 2026/09/20 12:48:43 INFO Signed narinfos id=45 count=15885lb # [ 157.872588] niks3-server[1094]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5886lb # [ 157.884108] niks3-server[1094]: 2026/09/20 12:48:43 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5887lb # [ 157.889824] niks3-server[1094]: 2026/09/20 12:48:43 INFO Completed upload id=455888worker1 # [ 158.232755] nix-grpc-daemon[1580]: time=2026-09-20T12:48:44.046Z level=INFO msg="Upload complete. (386ms)"5889lb # [ 157.894375] niks3-server[1094]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5890worker1 # [ 158.249387] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52422 duration_s=10 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15891worker1 # [ 158.268647] nix-daemon[856]: reaped child process 1645, status = succeeded5892client # [ 158.285501] systemd[1]: sw.service: Deactivated successfully.5893client # [ 158.290185] systemd[1]: sw.service: Consumed 168ms CPU time over 12.292s wall clock time, 39.2M memory peak, 8.7K incoming IP traffic, 9K outgoing IP traffic.5894worker1 # [ 158.761255] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=signal_exit5895client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5896worker1 # [ 158.781187] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=scheduler_yield5897client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5898lb # [ 158.440252] niks3-server[1094]: 2026/09/20 12:48:44 INFO lead: released remote=192.168.1.3:541325899worker1 # [ 158.788662] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=scheduler_restarting addr=lb:500515900worker2 # [ 158.767712] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:44Z level=info event=scheduler_restarting addr=lb:500515901worker1 # [ 158.794412] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:524065902worker1 # [ 158.799384] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=worker_gone addr=192.168.1.3:500515903worker1 # [ 158.803624] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:524065904worker1 # [ 158.808328] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=worker_gone addr=192.168.1.4:500515905client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5906worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5907worker1 # [ 158.989825] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=scheduler_connected addr=lb:500515908worker2 # [ 158.968159] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:44Z level=info event=scheduler_connected addr=lb:500515909worker1 # [ 158.995642] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5910worker2 # [ 158.973951] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5911worker1 # [ 159.196631] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515912worker2 # [ 159.175361] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515913worker1 # [ 159.202604] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker2 # [ 159.181346] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5915worker2 # [ 159.382892] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515916worker1 # [ 159.406009] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515917worker1 # [ 159.410533] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918worker2 # [ 159.388734] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5919worker1 # [ 159.606731] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515920worker2 # [ 159.590052] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515921worker1 # [ 159.612525] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker2 # [ 159.595667] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker1 # [ 159.786431] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=scheduler_restarting5924worker1 # [ 159.808179] nix-daemon[856]: reaped child process 1552, status = succeeded5925worker1 # [ 159.822768] nix-grpc-daemon[1501]: ts=2026-09-20T12:48:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5926worker2 # [ 159.798758] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 159.803473] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker1 # [ 159.968436] nix-grpc-daemon[1501]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5929worker1 # [ 159.973105] nix-grpc-daemon[1501]: E0000 00:00:1789908525.781545 1544 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5930worker2 # [ 160.000149] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_connected addr=lb:500515931worker2 # [ 160.005669] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932worker1 # [ 160.033884] nix-grpc-daemon[1501]: E0000 00:00:1789908525.847219 1544 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5933worker2 # [ 160.206766] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_connected addr=lb:500515934worker2 # [ 160.212630] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5935worker2 # [ 160.414011] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_connected addr=lb:500515936worker2 # [ 160.419748] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5937worker2 # [ 160.621045] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 160.626678] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5939worker2 # [ 160.829964] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_connected addr=lb:500515940worker2 # [ 160.834437] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5941worker2 # [ 161.031028] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_connected addr=lb:500515942worker2 # [ 161.036713] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5943lb # [ 160.727828] niks3-server[1094]: 2026/09/20 12:48:46 INFO lead: acquired remote=192.168.1.4:427945944worker2 # [ 161.049261] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:46Z level=info event=scheduler_take_over5945worker2 # [ 161.238147] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_connected addr=lb:500515946worker2 # [ 161.243755] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5947worker2 # [ 161.444945] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_connected addr=lb:500515948worker2 # [ 161.450418] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5949worker2 # [ 161.652050] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_connected addr=lb:500515950worker2 # [ 161.658024] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5951worker1 # [ 161.805016] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5952worker1 # [ 161.810246] systemd[1]: nix-grpc-daemon.service: Consumed 952ms CPU time over 26.443s wall clock time, 32M memory peak, 26K incoming IP traffic, 33.1K outgoing IP traffic.5953worker1 # [ 161.834606] systemd[1]: Starting Nix worker-protocol over gRPC...5954worker2 # [ 161.861034] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=scheduler_connected addr=lb:500515955worker2 # [ 161.891178] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=oidc_jwks provider=mock keys=15956worker2 # [ 161.896131] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:575025957worker2 # [ 161.901623] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05958worker1 # [ 162.039741] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5959worker1 # [ 162.042438] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5960worker1 # [ 162.046980] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15961worker1 # [ 162.055186] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5962worker1 # [ 162.064685] systemd[1]: Started Nix worker-protocol over gRPC.5963worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.24 seconds)5964lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5965lb: must succeed: curl -sf localhost:9901/clusters5966worker1 # [ 162.080965] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:47Z level=info event=scheduler_connected addr=lb:500515967worker2 # [ 162.061366] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:575025968worker2 # [ 162.067692] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5970lb: must succeed: curl -sf localhost:9901/clusters5971lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5972lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)5973lb: must succeed: curl -sf localhost:9901/clusters5974lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5975worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5976worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5977(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.47 seconds)5978subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5979client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15980worker1 # [ 163.659515] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561005981worker1 # [ 163.674349] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)5982worker1 # [ 163.695937] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561005983worker1 # [ 163.712107] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)5984worker1 # [ 163.747729] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)5985worker1 # [ 163.893779] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 3/5)5986worker1 # [ 164.180666] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 791 ms (attempt 4/5)5987worker1 # [ 164.981074] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5988worker2 # [ 165.014113] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:50Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05989worker2 # [ 165.020122] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:50Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15990worker2 # [ 165.025988] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:50Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05991worker2 # [ 165.031295] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:50Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05992worker1 # [ 165.083786] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561005993worker1 # [ 165.100268] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561005994worker1 # [ 165.115135] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561005995worker1 # [ 165.131443] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)5996worker1 # [ 165.158519] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:50Z level=info event=niks3_push_started pid=17665997worker1 # [ 165.375611] nix-grpc-daemon[1766]: time=2026-09-20T12:48:51.188Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5998worker1 # [ 165.557391] nix-daemon[856]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)5999worker1 # [ 165.578170] nix-daemon[856]: reaped child process 1782, status = succeeded6000worker1 # [ 165.762906] nix-daemon[856]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6001worker1 # [ 165.781467] nix-daemon[856]: reaped child process 1787, status = succeeded6002lb # [ 165.450249] niks3-server[1094]: 2026/09/20 12:48:51 INFO Received uploads request method=POST path=/api/pending_closures6003worker1 # [ 165.804090] nix-grpc-daemon[1766]: time=2026-09-20T12:48:51.616Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6004worker1 # [ 165.805685] nix-grpc-daemon[1766]: time=2026-09-20T12:48:51.616Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6005lb # [ 165.490223] niks3-server[1094]: 2026/09/20 12:48:51 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6006worker1 # [ 165.847565] nix-grpc-daemon[1766]: time=2026-09-20T12:48:51.660Z level=INFO msg="Uploading 1 narinfos"6007lb # [ 165.503986] niks3-server[1094]: 2026/09/20 12:48:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6008lb # [ 165.512378] niks3-server[1094]: 2026/09/20 12:48:51 INFO Signed narinfos id=46 count=16009lb # [ 165.522103] niks3-server[1094]: 2026/09/20 12:48:51 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6010lb # [ 165.535104] niks3-server[1094]: 2026/09/20 12:48:51 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6011lb # [ 165.544779] niks3-server[1094]: 2026/09/20 12:48:51 INFO Completed upload id=466012worker1 # [ 165.887573] nix-grpc-daemon[1766]: time=2026-09-20T12:48:51.700Z level=INFO msg="Upload complete. (508ms)"6013lb # [ 165.548142] niks3-server[1094]: 2026/09/20 12:48:51 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6014worker1 # [ 165.896415] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=5126015worker1 # [ 165.912705] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006016worker1 # [ 165.919257] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6017worker1 # [ 165.925175] nix-daemon[856]: reaped child process 1764, status = succeeded6018worker1 # [ 165.940394] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6019worker1 # [ 166.964486] nix-daemon[856]: reaped child process 1791, status = succeeded6020worker1 # [ 167.124300] nix-daemon[856]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6021worker1 # [ 167.144437] nix-daemon[856]: reaped child process 1809, status = succeeded6022lb # [ 166.813958] niks3-server[1094]: 2026/09/20 12:48:52 INFO Received uploads request method=POST path=/api/pending_closures6023worker1 # [ 167.167167] nix-grpc-daemon[1766]: time=2026-09-20T12:48:52.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6024worker1 # [ 167.172445] nix-grpc-daemon[1766]: time=2026-09-20T12:48:52.982Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6025lb # [ 166.852392] niks3-server[1094]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6026lb # [ 166.863613] niks3-server[1094]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6027worker1 # [ 167.222683] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.035Z level=INFO msg="Uploading 1 narinfos"6028lb # [ 166.878721] niks3-server[1094]: 2026/09/20 12:48:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6029lb # [ 166.888119] niks3-server[1094]: 2026/09/20 12:48:52 INFO Signed narinfos id=47 count=16030lb # [ 166.894109] niks3-server[1094]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6031lb # [ 166.902062] niks3-server[1094]: 2026/09/20 12:48:52 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6032lb # [ 166.911969] niks3-server[1094]: 2026/09/20 12:48:52 INFO Completed upload id=476033worker1 # [ 167.254765] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.068Z level=INFO msg="Upload complete. (303ms)"6034lb # [ 166.915276] niks3-server[1094]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6035worker1 # [ 167.265059] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16036worker2 # [ 167.247853] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:53Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16037worker1 # [ 167.283603] nix-daemon[856]: reaped child process 1789, status = succeeded6038worker2 # [ 167.260291] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:53Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06039worker1 # [ 167.315384] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006040worker1 # [ 167.332163] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006041worker1 # [ 167.347555] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006042worker1 # [ 167.361880] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6043worker1 # [ 167.568719] nix-daemon[856]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6044worker1 # [ 167.589523] nix-daemon[856]: reaped child process 1816, status = succeeded6045lb # [ 167.253714] envoy[909]: conn peer=[2001:db8:1::1]:45222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1375 ms=120166046lb # [ 167.258860] envoy[909]: conn peer=[2001:db8:1::3]:45398 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1426 tx=539 ms=240496047worker1 # [ 167.761778] nix-daemon[856]: accepted connection from pid 1818, user nix-grpc-daemon (trusted)6048worker1 # [ 167.782180] nix-daemon[856]: reaped child process 1821, status = succeeded6049lb # [ 167.452649] niks3-server[1094]: 2026/09/20 12:48:53 INFO Received uploads request method=POST path=/api/pending_closures6050worker1 # [ 167.805082] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.618Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6051worker1 # [ 167.806602] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.618Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6052lb # [ 167.485795] niks3-server[1094]: 2026/09/20 12:48:53 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6053worker1 # [ 167.839557] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.652Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 167.501539] niks3-server[1094]: 2026/09/20 12:48:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6055lb # [ 167.511362] niks3-server[1094]: 2026/09/20 12:48:53 INFO Signed narinfos id=48 count=16056worker1 # [ 167.864345] nix-grpc-daemon[1766]: time=2026-09-20T12:48:53.677Z level=INFO msg="Upload complete. (486ms)"6057lb # [ 167.521577] niks3-server[1094]: 2026/09/20 12:48:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6058lb # [ 167.525409] niks3-server[1094]: 2026/09/20 12:48:53 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6059lb # [ 167.529853] niks3-server[1094]: 2026/09/20 12:48:53 INFO Completed upload id=486060worker1 # [ 167.872231] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=5126061lb # [ 167.532846] niks3-server[1094]: 2026/09/20 12:48:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6062worker1 # [ 167.880699] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006063worker1 # [ 167.891538] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6064worker1 # [ 167.896304] nix-daemon[856]: reaped child process 1811, status = succeeded6065worker1 # [ 167.909399] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6066worker1 # [ 169.354209] nix-daemon[856]: reaped child process 1825, status = succeeded6067worker1 # [ 169.536715] nix-daemon[856]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)6068worker1 # [ 169.557269] nix-daemon[856]: reaped child process 1843, status = succeeded6069lb # [ 169.227038] niks3-server[1094]: 2026/09/20 12:48:55 INFO Received uploads request method=POST path=/api/pending_closures6070worker1 # [ 169.580316] nix-grpc-daemon[1766]: time=2026-09-20T12:48:55.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6071worker1 # [ 169.581806] nix-grpc-daemon[1766]: time=2026-09-20T12:48:55.392Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6072lb # [ 169.263363] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6073lb # [ 169.277227] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6074worker1 # [ 169.627584] nix-grpc-daemon[1766]: time=2026-09-20T12:48:55.440Z level=INFO msg="Uploading 1 narinfos"6075lb # [ 169.288052] niks3-server[1094]: 2026/09/20 12:48:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6076lb # [ 169.298687] niks3-server[1094]: 2026/09/20 12:48:55 INFO Signed narinfos id=49 count=16077lb # [ 169.302872] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6078lb # [ 169.312312] niks3-server[1094]: 2026/09/20 12:48:55 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6079lb # [ 169.316869] niks3-server[1094]: 2026/09/20 12:48:55 INFO Completed upload id=496080worker1 # [ 169.659632] nix-grpc-daemon[1766]: time=2026-09-20T12:48:55.473Z level=INFO msg="Upload complete. (338ms)"6081lb # [ 169.319384] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6082worker2 # [ 169.646442] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:55Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16083worker1 # [ 169.669063] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16084worker2 # [ 169.659471] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:55Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06085worker1 # [ 169.687805] nix-daemon[856]: reaped child process 1823, status = succeeded6086worker1 # [ 169.723172] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006087worker1 # [ 169.739365] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006088worker1 # [ 169.754574] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006089worker1 # [ 169.768687] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6090worker1 # [ 169.979755] nix-daemon[856]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6091worker1 # [ 169.998266] nix-daemon[856]: reaped child process 1850, status = succeeded6092worker1 # [ 170.178659] nix-daemon[856]: accepted connection from pid 1852, user nix-grpc-daemon (trusted)6093worker1 # [ 170.196971] nix-daemon[856]: reaped child process 1855, status = succeeded6094lb # [ 169.868098] niks3-server[1094]: 2026/09/20 12:48:55 INFO Received uploads request method=POST path=/api/pending_closures6095worker1 # [ 170.219768] nix-grpc-daemon[1766]: time=2026-09-20T12:48:56.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6096worker1 # [ 170.221500] nix-grpc-daemon[1766]: time=2026-09-20T12:48:56.033Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6097lb # [ 169.895494] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6098worker1 # [ 170.251632] nix-grpc-daemon[1766]: time=2026-09-20T12:48:56.064Z level=INFO msg="Uploading 1 narinfos"6099lb # [ 169.911284] niks3-server[1094]: 2026/09/20 12:48:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6100lb # [ 169.923917] niks3-server[1094]: 2026/09/20 12:48:55 INFO Signed narinfos id=50 count=16101lb # [ 169.932821] niks3-server[1094]: 2026/09/20 12:48:55 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6102worker1 # [ 170.282971] nix-grpc-daemon[1766]: time=2026-09-20T12:48:56.096Z level=INFO msg="Upload complete. (497ms)"6103lb # [ 169.941208] niks3-server[1094]: 2026/09/20 12:48:56 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6104lb # [ 169.947177] niks3-server[1094]: 2026/09/20 12:48:56 INFO Completed upload id=506105worker1 # [ 170.289759] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=5126106lb # [ 169.950854] niks3-server[1094]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6107worker1 # [ 170.302954] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006108worker1 # [ 170.308963] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6109worker1 # [ 170.313828] nix-daemon[856]: reaped child process 1845, status = succeeded6110worker1 # [ 170.327284] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6111worker1 # [ 171.562656] nix-daemon[856]: reaped child process 1859, status = succeeded6112worker1 # [ 171.731899] nix-daemon[856]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6113worker1 # [ 171.752355] nix-daemon[856]: reaped child process 1877, status = succeeded6114lb # [ 171.422786] niks3-server[1094]: 2026/09/20 12:48:57 INFO Received uploads request method=POST path=/api/pending_closures6115worker1 # [ 171.776425] nix-grpc-daemon[1766]: time=2026-09-20T12:48:57.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6116worker1 # [ 171.777993] nix-grpc-daemon[1766]: time=2026-09-20T12:48:57.588Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6117lb # [ 171.456848] niks3-server[1094]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6118lb # [ 171.470392] niks3-server[1094]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6119lb # [ 171.483537] niks3-server[1094]: 2026/09/20 12:48:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6120worker1 # [ 171.831765] nix-grpc-daemon[1766]: time=2026-09-20T12:48:57.644Z level=INFO msg="Uploading 1 narinfos"6121lb # [ 171.491976] niks3-server[1094]: 2026/09/20 12:48:57 INFO Signed narinfos id=51 count=16122lb # [ 171.494489] niks3-server[1094]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6123lb # [ 171.505605] niks3-server[1094]: 2026/09/20 12:48:57 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6124lb # [ 171.512391] niks3-server[1094]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6125lb # [ 171.518058] niks3-server[1094]: 2026/09/20 12:48:57 INFO Completed upload id=516126worker1 # [ 171.860815] nix-grpc-daemon[1766]: time=2026-09-20T12:48:57.674Z level=INFO msg="Upload complete. (323ms)"6127worker2 # [ 171.846374] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:57Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16128worker1 # [ 171.873130] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16129worker2 # [ 171.860492] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:57Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06130worker1 # [ 171.888389] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006131worker1 # [ 171.899147] nix-daemon[856]: reaped child process 1857, status = succeeded6132worker1 # [ 171.903280] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006133worker1 # [ 171.912009] nix-grpc-daemon[1766]: time=2026-09-20T12:48:57.725Z level=INFO msg="All 1 paths already cached"6134worker1 # [ 171.917824] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006135worker1 # [ 171.929510] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6136worker1 # [ 172.121102] nix-daemon[856]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6137worker1 # [ 172.139800] nix-daemon[856]: reaped child process 1884, status = succeeded6138worker1 # [ 172.312964] nix-daemon[856]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6139worker1 # [ 172.331439] nix-daemon[856]: reaped child process 1889, status = succeeded6140lb # [ 172.000905] niks3-server[1094]: 2026/09/20 12:48:58 INFO Received uploads request method=POST path=/api/pending_closures6141worker1 # [ 172.354406] nix-grpc-daemon[1766]: time=2026-09-20T12:48:58.167Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6142worker1 # [ 172.356111] nix-grpc-daemon[1766]: time=2026-09-20T12:48:58.167Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6143lb # [ 172.029646] niks3-server[1094]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6144lb # [ 172.043330] niks3-server[1094]: 2026/09/20 12:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6145worker1 # [ 172.392882] nix-grpc-daemon[1766]: time=2026-09-20T12:48:58.206Z level=INFO msg="Uploading 1 narinfos"6146lb # [ 172.056442] niks3-server[1094]: 2026/09/20 12:48:58 INFO Signed narinfos id=52 count=16147lb # [ 172.059328] niks3-server[1094]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6148lb # [ 172.068265] niks3-server[1094]: 2026/09/20 12:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6149lb # [ 172.075068] niks3-server[1094]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6150lb # [ 172.079368] niks3-server[1094]: 2026/09/20 12:48:58 INFO Completed upload id=526151worker1 # [ 172.421681] nix-grpc-daemon[1766]: time=2026-09-20T12:48:58.235Z level=INFO msg="Upload complete. (479ms)"6152worker1 # [ 172.429377] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=6246153worker1 # [ 172.443402] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006154worker1 # [ 172.450144] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6155worker1 # [ 172.455020] nix-daemon[856]: reaped child process 1879, status = succeeded6156worker1 # [ 172.466770] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6157worker1 # [ 173.303713] nix-daemon[856]: reaped child process 1893, status = succeeded6158worker1 # [ 173.479583] nix-daemon[856]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6159worker1 # [ 173.501203] nix-daemon[856]: reaped child process 1911, status = succeeded6160lb # [ 173.170674] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received uploads request method=POST path=/api/pending_closures6161worker1 # [ 173.523193] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.336Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6162worker1 # [ 173.527831] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.338Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6163lb # [ 173.212546] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6164lb # [ 173.224974] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6165lb # [ 173.232505] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6166worker1 # [ 173.583362] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.396Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 173.244371] niks3-server[1094]: 2026/09/20 12:48:59 INFO Signed narinfos id=53 count=16168lb # [ 173.246946] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6169lb # [ 173.256300] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6170lb # [ 173.264761] niks3-server[1094]: 2026/09/20 12:48:59 INFO Completed upload id=536171worker1 # [ 173.607488] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.420Z level=INFO msg="Upload complete. (328ms)"6172lb # [ 173.267664] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6173worker2 # [ 173.595368] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:59Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16174worker1 # [ 173.621037] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16175worker2 # [ 173.608129] nix-grpc-daemon[1311]: ts=2026-09-20T12:48:59Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06176worker1 # [ 173.640287] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006177worker1 # [ 173.646553] nix-daemon[856]: reaped child process 1891, status = succeeded6178worker1 # [ 173.650334] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006179worker1 # [ 173.662926] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.476Z level=INFO msg="All 1 paths already cached"6180worker1 # [ 173.671502] nix-grpc-daemon[1710]: ts=2026-09-20T12:48:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006181worker1 # [ 173.683288] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6182worker1 # [ 173.884193] nix-daemon[856]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6183worker1 # [ 173.903855] nix-daemon[856]: reaped child process 1918, status = succeeded6184worker1 # [ 174.081034] nix-daemon[856]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6185worker1 # [ 174.099761] nix-daemon[856]: reaped child process 1923, status = succeeded6186lb # [ 173.768404] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received uploads request method=POST path=/api/pending_closures6187worker1 # [ 174.120191] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.933Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6188worker1 # [ 174.122163] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.935Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6189lb # [ 173.804394] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6190lb # [ 173.815974] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6191worker1 # [ 174.164868] nix-grpc-daemon[1766]: time=2026-09-20T12:48:59.978Z level=INFO msg="Uploading 1 narinfos"6192lb # [ 173.827325] niks3-server[1094]: 2026/09/20 12:48:59 INFO Signed narinfos id=54 count=16193lb # [ 173.833087] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6194lb # [ 173.840876] niks3-server[1094]: 2026/09/20 12:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6195lb # [ 173.847821] niks3-server[1094]: 2026/09/20 12:48:59 INFO Completed upload id=546196worker1 # [ 174.190498] nix-grpc-daemon[1766]: time=2026-09-20T12:49:00.004Z level=INFO msg="Upload complete. (494ms)"6197lb # [ 173.850768] niks3-server[1094]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6198worker1 # [ 174.200954] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=6246199worker1 # [ 174.212133] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006200worker1 # [ 174.219367] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6201worker1 # [ 174.224957] nix-daemon[856]: reaped child process 1913, status = succeeded6202worker1 # [ 174.239050] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6203worker1 # [ 175.280567] nix-daemon[856]: reaped child process 1927, status = succeeded6204worker1 # [ 175.454328] nix-daemon[856]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6205worker1 # [ 175.473063] nix-daemon[856]: reaped child process 1945, status = succeeded6206lb # [ 175.142826] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received uploads request method=POST path=/api/pending_closures6207worker1 # [ 175.496426] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.308Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6208worker1 # [ 175.497950] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.308Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6209lb # [ 175.177053] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6210lb # [ 175.203450] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6211lb # [ 175.215050] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6212worker1 # [ 175.561930] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.375Z level=INFO msg="Uploading 1 narinfos"6213lb # [ 175.224334] niks3-server[1094]: 2026/09/20 12:49:01 INFO Signed narinfos id=55 count=16214lb # [ 175.230091] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6215lb # [ 175.236807] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6216lb # [ 175.240474] niks3-server[1094]: 2026/09/20 12:49:01 INFO Completed upload id=556217worker1 # [ 175.583043] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.396Z level=INFO msg="Upload complete. (332ms)"6218lb # [ 175.242640] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6219worker2 # [ 175.569346] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:01Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16220worker2 # [ 175.582113] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:01Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06221worker1 # [ 175.593148] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16222worker1 # [ 175.613139] nix-daemon[856]: reaped child process 1925, status = succeeded6223worker1 # [ 175.646869] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006224worker1 # [ 175.663368] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006225worker1 # [ 175.678718] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.492Z level=INFO msg="All 1 paths already cached"6226worker1 # [ 175.687820] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006227worker1 # [ 175.702117] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6228worker1 # [ 175.900691] nix-daemon[856]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6229worker1 # [ 175.921779] nix-daemon[856]: reaped child process 1952, status = succeeded6230worker1 # [ 176.096088] nix-daemon[856]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6231worker1 # [ 176.113992] nix-daemon[856]: reaped child process 1957, status = succeeded6232lb # [ 175.783429] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received uploads request method=POST path=/api/pending_closures6233worker1 # [ 176.135145] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.948Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6234worker1 # [ 176.136831] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.948Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6235lb # [ 175.814631] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6236worker1 # [ 176.166351] nix-grpc-daemon[1766]: time=2026-09-20T12:49:01.979Z level=INFO msg="Uploading 1 narinfos"6237lb # [ 175.829202] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6238lb # [ 175.843173] niks3-server[1094]: 2026/09/20 12:49:01 INFO Signed narinfos id=56 count=16239worker1 # [ 176.188838] nix-grpc-daemon[1766]: time=2026-09-20T12:49:02.002Z level=INFO msg="Upload complete. (470ms)"6240lb # [ 175.847729] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6241lb # [ 175.852532] niks3-server[1094]: 2026/09/20 12:49:01 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6242worker1 # [ 176.194960] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=6246243lb # [ 175.856757] niks3-server[1094]: 2026/09/20 12:49:01 INFO Completed upload id=566244lb # [ 175.858774] niks3-server[1094]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6245worker1 # [ 176.206280] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006246worker1 # [ 176.212569] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6247worker1 # [ 176.216702] nix-daemon[856]: reaped child process 1947, status = succeeded6248worker1 # [ 176.228621] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6249worker1 # [ 177.671903] nix-daemon[856]: reaped child process 1961, status = succeeded6250worker1 # [ 177.845043] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6251worker1 # [ 177.865995] nix-daemon[856]: reaped child process 1979, status = succeeded6252lb # [ 177.538559] niks3-server[1094]: 2026/09/20 12:49:03 INFO Received uploads request method=POST path=/api/pending_closures6253worker1 # [ 177.891935] nix-grpc-daemon[1766]: time=2026-09-20T12:49:03.704Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6254worker1 # [ 177.893676] nix-grpc-daemon[1766]: time=2026-09-20T12:49:03.704Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6255lb # [ 177.574995] niks3-server[1094]: 2026/09/20 12:49:03 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6256lb # [ 177.586586] niks3-server[1094]: 2026/09/20 12:49:03 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6257worker1 # [ 177.941513] nix-grpc-daemon[1766]: time=2026-09-20T12:49:03.754Z level=INFO msg="Uploading 1 narinfos"6258lb # [ 177.598808] niks3-server[1094]: 2026/09/20 12:49:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6259lb # [ 177.608627] niks3-server[1094]: 2026/09/20 12:49:03 INFO Signed narinfos id=57 count=16260lb # [ 177.613304] niks3-server[1094]: 2026/09/20 12:49:03 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6261worker1 # [ 177.960157] nix-grpc-daemon[1766]: time=2026-09-20T12:49:03.773Z level=INFO msg="Upload complete. (321ms)"6262lb # [ 177.618258] niks3-server[1094]: 2026/09/20 12:49:03 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6263lb # [ 177.622055] niks3-server[1094]: 2026/09/20 12:49:03 INFO Completed upload id=576264lb # [ 177.624968] niks3-server[1094]: 2026/09/20 12:49:03 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6265worker2 # [ 177.947362] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:03Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16266worker1 # [ 177.972905] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16267worker2 # [ 177.961342] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:03Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06268worker1 # [ 177.988363] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006269worker1 # [ 177.995234] nix-daemon[856]: reaped child process 1959, status = succeeded6270worker1 # [ 177.999642] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006271worker1 # [ 178.008573] nix-grpc-daemon[1766]: time=2026-09-20T12:49:03.822Z level=INFO msg="All 2 paths already cached"6272worker1 # [ 178.016469] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006273worker1 # [ 178.027164] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6274worker1 # [ 178.216188] nix-daemon[856]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)6275worker1 # [ 178.236392] nix-daemon[856]: reaped child process 1986, status = succeeded6276worker1 # [ 178.414189] nix-daemon[856]: accepted connection from pid 1988, user nix-grpc-daemon (trusted)6277worker1 # [ 178.433288] nix-daemon[856]: reaped child process 1991, status = succeeded6278lb # [ 178.103178] niks3-server[1094]: 2026/09/20 12:49:04 INFO Received uploads request method=POST path=/api/pending_closures6279worker1 # [ 178.455412] nix-grpc-daemon[1766]: time=2026-09-20T12:49:04.268Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6280worker1 # [ 178.457060] nix-grpc-daemon[1766]: time=2026-09-20T12:49:04.268Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6281lb # [ 178.135871] niks3-server[1094]: 2026/09/20 12:49:04 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6282worker1 # [ 178.487961] nix-grpc-daemon[1766]: time=2026-09-20T12:49:04.301Z level=INFO msg="Uploading 1 narinfos"6283lb # [ 178.150901] niks3-server[1094]: 2026/09/20 12:49:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6284lb # [ 178.160573] niks3-server[1094]: 2026/09/20 12:49:04 INFO Signed narinfos id=58 count=16285lb # [ 178.163321] niks3-server[1094]: 2026/09/20 12:49:04 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6286lb # [ 178.173603] niks3-server[1094]: 2026/09/20 12:49:04 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6287worker1 # [ 178.521535] nix-grpc-daemon[1766]: time=2026-09-20T12:49:04.335Z level=INFO msg="Upload complete. (481ms)"6288lb # [ 178.179208] niks3-server[1094]: 2026/09/20 12:49:04 INFO Completed upload id=586289lb # [ 178.183658] niks3-server[1094]: 2026/09/20 12:49:04 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6290worker1 # [ 178.528365] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=7526291worker1 # [ 178.543187] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006292worker1 # [ 178.550231] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6293worker1 # [ 178.555124] nix-daemon[856]: reaped child process 1981, status = succeeded6294worker1 # [ 178.566402] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6295worker1 # [ 179.598887] nix-daemon[856]: reaped child process 1995, status = succeeded6296worker1 # [ 179.773110] nix-daemon[856]: accepted connection from pid 2010, user nix-grpc-daemon (trusted)6297worker1 # [ 179.792691] nix-daemon[856]: reaped child process 2013, status = succeeded6298lb # [ 179.462729] niks3-server[1094]: 2026/09/20 12:49:05 INFO Received uploads request method=POST path=/api/pending_closures6299worker1 # [ 179.816554] nix-grpc-daemon[1766]: time=2026-09-20T12:49:05.628Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6300worker1 # [ 179.818345] nix-grpc-daemon[1766]: time=2026-09-20T12:49:05.628Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6301lb # [ 179.505191] niks3-server[1094]: 2026/09/20 12:49:05 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6302lb # [ 179.517959] niks3-server[1094]: 2026/09/20 12:49:05 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6303lb # [ 179.530159] niks3-server[1094]: 2026/09/20 12:49:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6304worker1 # [ 179.879128] nix-grpc-daemon[1766]: time=2026-09-20T12:49:05.692Z level=INFO msg="Uploading 1 narinfos"6305lb # [ 179.540215] niks3-server[1094]: 2026/09/20 12:49:05 INFO Signed narinfos id=59 count=16306lb # [ 179.543177] niks3-server[1094]: 2026/09/20 12:49:05 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6307lb # [ 179.556997] niks3-server[1094]: 2026/09/20 12:49:05 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6308lb # [ 179.563261] niks3-server[1094]: 2026/09/20 12:49:05 INFO Completed upload id=596309worker1 # [ 179.906110] nix-grpc-daemon[1766]: time=2026-09-20T12:49:05.719Z level=INFO msg="Upload complete. (327ms)"6310lb # [ 179.566665] niks3-server[1094]: 2026/09/20 12:49:05 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6311worker2 # [ 179.894660] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:05Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16312worker1 # [ 179.917197] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16313worker1 # [ 179.938816] nix-daemon[856]: reaped child process 1993, status = succeeded6314worker1 # [ 179.970879] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006315worker1 # [ 179.986992] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006316worker1 # [ 180.001871] nix-grpc-daemon[1766]: time=2026-09-20T12:49:05.815Z level=INFO msg="All 2 paths already cached"6317worker1 # [ 180.011044] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006318worker1 # [ 180.026084] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6319worker1 # [ 180.233616] nix-daemon[856]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)6320worker1 # [ 180.252624] nix-daemon[856]: reaped child process 2020, status = succeeded6321worker1 # [ 180.430607] nix-daemon[856]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)6322worker1 # [ 180.448954] nix-daemon[856]: reaped child process 2025, status = succeeded6323lb # [ 180.120405] niks3-server[1094]: 2026/09/20 12:49:06 INFO Received uploads request method=POST path=/api/pending_closures6324worker1 # [ 180.472214] nix-grpc-daemon[1766]: time=2026-09-20T12:49:06.285Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6325worker1 # [ 180.473731] nix-grpc-daemon[1766]: time=2026-09-20T12:49:06.285Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6326lb # [ 180.153331] niks3-server[1094]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6327worker1 # [ 180.507545] nix-grpc-daemon[1766]: time=2026-09-20T12:49:06.320Z level=INFO msg="Uploading 1 narinfos"6328lb # [ 180.170576] niks3-server[1094]: 2026/09/20 12:49:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6329lb # [ 180.181621] niks3-server[1094]: 2026/09/20 12:49:06 INFO Signed narinfos id=60 count=16330lb # [ 180.187589] niks3-server[1094]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6331worker1 # [ 180.539025] nix-grpc-daemon[1766]: time=2026-09-20T12:49:06.352Z level=INFO msg="Upload complete. (496ms)"6332lb # [ 180.196752] niks3-server[1094]: 2026/09/20 12:49:06 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6333lb # [ 180.204292] niks3-server[1094]: 2026/09/20 12:49:06 INFO Completed upload id=606334lb # [ 180.208844] niks3-server[1094]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6335worker1 # [ 180.548254] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=7526336worker1 # [ 180.559515] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006337worker1 # [ 180.566134] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6338worker1 # [ 180.571314] nix-daemon[856]: reaped child process 2015, status = succeeded6339worker1 # [ 180.585945] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6340worker1 # [ 181.406566] nix-daemon[856]: reaped child process 2029, status = succeeded6341worker1 # [ 181.570442] nix-daemon[856]: accepted connection from pid 2044, user nix-grpc-daemon (trusted)6342worker1 # [ 181.591207] nix-daemon[856]: reaped child process 2047, status = succeeded6343lb # [ 181.262280] niks3-server[1094]: 2026/09/20 12:49:07 INFO Received uploads request method=POST path=/api/pending_closures6344worker1 # [ 181.615308] nix-grpc-daemon[1766]: time=2026-09-20T12:49:07.427Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6345worker1 # [ 181.617035] nix-grpc-daemon[1766]: time=2026-09-20T12:49:07.427Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6346lb # [ 181.298278] niks3-server[1094]: 2026/09/20 12:49:07 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6347lb # [ 181.310848] niks3-server[1094]: 2026/09/20 12:49:07 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6348worker1 # [ 181.666804] nix-grpc-daemon[1766]: time=2026-09-20T12:49:07.479Z level=INFO msg="Uploading 1 narinfos"6349lb # [ 181.323958] niks3-server[1094]: 2026/09/20 12:49:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6350lb # [ 181.329792] niks3-server[1094]: 2026/09/20 12:49:07 INFO Signed narinfos id=61 count=16351lb # [ 181.334912] niks3-server[1094]: 2026/09/20 12:49:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6352lb # [ 181.346322] niks3-server[1094]: 2026/09/20 12:49:07 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6353lb # [ 181.352484] niks3-server[1094]: 2026/09/20 12:49:07 INFO Completed upload id=616354worker1 # [ 181.695162] nix-grpc-daemon[1766]: time=2026-09-20T12:49:07.508Z level=INFO msg="Upload complete. (299ms)"6355lb # [ 181.355465] niks3-server[1094]: 2026/09/20 12:49:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6356worker2 # [ 181.682794] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:07Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16357worker1 # [ 181.705856] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16358worker1 # [ 181.722109] nix-daemon[856]: reaped child process 2027, status = succeeded6359worker1 # [ 181.750718] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006360worker1 # [ 181.766897] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006361worker1 # [ 181.781772] nix-grpc-daemon[1766]: time=2026-09-20T12:49:07.595Z level=INFO msg="All 2 paths already cached"6362worker1 # [ 181.791144] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561006363worker1 # [ 181.806324] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6364worker1 # [ 182.004626] nix-daemon[856]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6365worker1 # [ 182.025244] nix-daemon[856]: reaped child process 2054, status = succeeded6366worker1 # [ 182.197154] nix-daemon[856]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6367worker1 # [ 182.215496] nix-daemon[856]: reaped child process 2059, status = succeeded6368lb # [ 181.884244] niks3-server[1094]: 2026/09/20 12:49:07 INFO Received uploads request method=POST path=/api/pending_closures6369worker1 # [ 182.235344] nix-grpc-daemon[1766]: time=2026-09-20T12:49:08.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6370worker1 # [ 182.237013] nix-grpc-daemon[1766]: time=2026-09-20T12:49:08.048Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6371lb # [ 181.912229] niks3-server[1094]: 2026/09/20 12:49:07 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6372lb # [ 181.924630] niks3-server[1094]: 2026/09/20 12:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6373worker1 # [ 182.274655] nix-grpc-daemon[1766]: time=2026-09-20T12:49:08.087Z level=INFO msg="Uploading 1 narinfos"6374lb # [ 181.933537] niks3-server[1094]: 2026/09/20 12:49:08 INFO Signed narinfos id=62 count=16375lb # [ 181.939777] niks3-server[1094]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6376lb # [ 181.949942] niks3-server[1094]: 2026/09/20 12:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6377lb # [ 181.957376] niks3-server[1094]: 2026/09/20 12:49:08 INFO Completed upload id=626378worker1 # [ 182.300046] nix-grpc-daemon[1766]: time=2026-09-20T12:49:08.113Z level=INFO msg="Upload complete. (476ms)"6379lb # [ 181.960296] niks3-server[1094]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6380worker1 # [ 182.308141] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=0 paths=1 nar_bytes_in=7526381worker1 # [ 182.320597] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561006382worker1 # [ 182.327653] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6383worker1 # [ 182.333591] nix-daemon[856]: reaped child process 2049, status = succeeded6384worker1 # [ 182.347783] nix-daemon[856]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6385worker1 # [ 184.190522] nix-daemon[856]: reaped child process 2063, status = succeeded6386worker1 # [ 184.381581] nix-daemon[856]: accepted connection from pid 2078, user nix-grpc-daemon (trusted)6387worker1 # [ 184.402102] nix-daemon[856]: reaped child process 2081, status = succeeded6388lb # [ 184.075118] niks3-server[1094]: 2026/09/20 12:49:10 INFO Received uploads request method=POST path=/api/pending_closures6389worker1 # [ 184.430282] nix-grpc-daemon[1766]: time=2026-09-20T12:49:10.241Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6390worker1 # [ 184.432194] nix-grpc-daemon[1766]: time=2026-09-20T12:49:10.241Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6391lb # [ 184.119478] niks3-server[1094]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6392worker1 # [ 184.472474] nix-grpc-daemon[1766]: time=2026-09-20T12:49:10.285Z level=INFO msg="Uploading 1 narinfos"6393lb # [ 184.138514] niks3-server[1094]: 2026/09/20 12:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6394lb # [ 184.158254] niks3-server[1094]: 2026/09/20 12:49:10 INFO Signed narinfos id=63 count=16395worker1 # [ 184.503886] nix-grpc-daemon[1766]: time=2026-09-20T12:49:10.317Z level=INFO msg="Upload complete. (344ms)"6396lb # [ 184.164554] niks3-server[1094]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6397lb # [ 184.171405] niks3-server[1094]: 2026/09/20 12:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6398lb # [ 184.175518] niks3-server[1094]: 2026/09/20 12:49:10 INFO Completed upload id=636399lb # [ 184.178018] niks3-server[1094]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6400worker1 # [ 184.517038] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56100 duration_s=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16401worker1 # [ 184.532393] nix-grpc-daemon[1766]: time=2026-09-20T12:49:10.326Z level=WARN msg="Failed to register uploaded object" key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls error="server returned 404: object is not pending upload\n"6402worker1 # [ 184.540370] nix-daemon[856]: reaped child process 2061, status = succeeded6403client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.20 seconds)6404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6405worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6406client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16407worker1 # [ 184.913670] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561006408worker1 # [ 184.922609] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561006409worker2 # [ 184.927978] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:10Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06410lb # [ 187.250367] envoy[909]: conn peer=[2001:db8:1::1]:54570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27230 tx=10757 ms=208856411client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.32 seconds)6412(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.60 seconds)6413subtest: low disk drains a worker and builds go to the other6414worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6415worker1 # [ 216.089353] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:41Z level=info event=unhealthy reason=min_free6416worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)6417worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6418worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6419client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26420client # don't know how to build these paths:6421client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6422worker2 # [ 216.409477] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400526423worker2 # [ 216.417912] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400526424worker2 # [ 216.443624] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06425worker2 # [ 216.450389] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16426worker2 # [ 216.489025] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526427worker2 # [ 216.504975] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400526428client # copying 1 paths to 'lb:50051'...6429worker2 # [ 216.521582] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400526430client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6431worker2 # [ 216.540433] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6432worker2 # [ 216.746422] nix-daemon[841]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6433worker2 # [ 216.764166] nix-daemon[841]: reaped child process 1604, status = succeeded6434worker2 # [ 216.940266] nix-daemon[841]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6435worker2 # [ 216.959318] nix-daemon[841]: reaped child process 1609, status = succeeded6436lb # [ 216.651029] niks3-server[1094]: 2026/09/20 12:49:42 INFO Received uploads request method=POST path=/api/pending_closures6437worker2 # [ 216.987703] nix-grpc-daemon[1403]: time=2026-09-20T12:49:42.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6438worker2 # [ 216.989350] nix-grpc-daemon[1403]: time=2026-09-20T12:49:42.924Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6439lb # [ 216.691577] niks3-server[1094]: 2026/09/20 12:49:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6440worker2 # [ 217.019785] nix-grpc-daemon[1403]: time=2026-09-20T12:49:42.956Z level=INFO msg="Uploading 1 narinfos"6441lb # [ 216.706493] niks3-server[1094]: 2026/09/20 12:49:42 INFO Signed narinfos id=64 count=16442lb # [ 216.716004] niks3-server[1094]: 2026/09/20 12:49:42 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6443lb # [ 216.724671] niks3-server[1094]: 2026/09/20 12:49:42 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6444lb # [ 216.734000] niks3-server[1094]: 2026/09/20 12:49:42 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6445lb # [ 216.739589] niks3-server[1094]: 2026/09/20 12:49:42 INFO Completed upload id=646446worker2 # [ 217.060236] nix-grpc-daemon[1403]: time=2026-09-20T12:49:42.997Z level=INFO msg="Upload complete. (502ms)"6447lb # [ 216.742323] niks3-server[1094]: 2026/09/20 12:49:42 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6448worker2 # [ 217.068322] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=5126449worker2 # [ 217.083912] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526450worker2 # [ 217.094232] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6451worker2 # [ 217.098698] nix-daemon[841]: reaped child process 1599, status = succeeded6452client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6453worker2 # [ 217.109590] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6454client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6455lb # [ 217.255959] envoy[909]: conn peer=[2001:db8:1::1]:42900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300566456client # farm-a> LOG-farm-a-drain6457worker2 # [ 218.335977] nix-daemon[841]: reaped child process 1613, status = succeeded6458worker2 # [ 218.495641] nix-daemon[841]: accepted connection from pid 1628, user nix-grpc-daemon (trusted)6459worker2 # [ 218.515232] nix-daemon[841]: reaped child process 1631, status = succeeded6460lb # [ 218.207099] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received uploads request method=POST path=/api/pending_closures6461worker2 # [ 218.536931] nix-grpc-daemon[1403]: time=2026-09-20T12:49:44.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6462worker2 # [ 218.541413] nix-grpc-daemon[1403]: time=2026-09-20T12:49:44.475Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6463lb # [ 218.249129] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6464worker2 # [ 218.582126] nix-grpc-daemon[1403]: time=2026-09-20T12:49:44.519Z level=INFO msg="Uploading 1 narinfos"6465lb # [ 218.261885] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6466lb # [ 218.271864] niks3-server[1094]: 2026/09/20 12:49:44 INFO Signed narinfos id=65 count=16467lb # [ 218.277604] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6468lb # [ 218.281799] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6469lb # [ 218.288536] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6470lb # [ 218.292604] niks3-server[1094]: 2026/09/20 12:49:44 INFO Completed upload id=656471worker2 # [ 218.612783] nix-grpc-daemon[1403]: time=2026-09-20T12:49:44.549Z level=INFO msg="Upload complete. (289ms)"6472lb # [ 218.295315] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6473worker2 # [ 218.625355] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16474worker2 # [ 218.643452] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06475worker2 # [ 218.647497] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16476worker2 # [ 218.651468] nix-daemon[841]: reaped child process 1611, status = succeeded6477worker2 # [ 218.681633] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526478worker2 # [ 218.696692] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400526479worker2 # [ 218.712156] nix-grpc-daemon[1403]: time=2026-09-20T12:49:44.649Z level=INFO msg="All 1 paths already cached"6480client # copying 1 paths to 'lb:50051'...6481client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6482worker2 # [ 218.722826] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400526483worker2 # [ 218.736751] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6484worker2 # [ 218.940169] nix-daemon[841]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6485worker2 # [ 218.959326] nix-daemon[841]: reaped child process 1640, status = succeeded6486worker2 # [ 219.132423] nix-daemon[841]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6487worker2 # [ 219.151313] nix-daemon[841]: reaped child process 1645, status = succeeded6488lb # [ 218.842101] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received uploads request method=POST path=/api/pending_closures6489worker2 # [ 219.172843] nix-grpc-daemon[1403]: time=2026-09-20T12:49:45.109Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6490worker2 # [ 219.174474] nix-grpc-daemon[1403]: time=2026-09-20T12:49:45.109Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6491lb # [ 218.871534] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6492lb # [ 218.886175] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6493worker2 # [ 219.213067] nix-grpc-daemon[1403]: time=2026-09-20T12:49:45.150Z level=INFO msg="Uploading 1 narinfos"6494lb # [ 218.898905] niks3-server[1094]: 2026/09/20 12:49:44 INFO Signed narinfos id=66 count=16495lb # [ 218.904422] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6496worker2 # [ 219.233464] nix-grpc-daemon[1403]: time=2026-09-20T12:49:45.170Z level=INFO msg="Upload complete. (481ms)"6497lb # [ 218.913454] niks3-server[1094]: 2026/09/20 12:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6498lb # [ 218.918790] niks3-server[1094]: 2026/09/20 12:49:44 INFO Completed upload id=666499lb # [ 218.922322] niks3-server[1094]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6500worker2 # [ 219.240100] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=6326501worker2 # [ 219.248853] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526502worker2 # [ 219.259645] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6503worker2 # [ 219.264525] nix-daemon[841]: reaped child process 1635, status = succeeded6504client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6505worker2 # [ 219.277185] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6506client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6507client # farm-b> LOG-farm-b-drain6508worker2 # [ 220.502227] nix-daemon[841]: reaped child process 1649, status = succeeded6509worker2 # [ 220.664191] nix-daemon[841]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6510worker2 # [ 220.681628] nix-daemon[841]: reaped child process 1667, status = succeeded6511lb # [ 220.375038] niks3-server[1094]: 2026/09/20 12:49:46 INFO Received uploads request method=POST path=/api/pending_closures6512worker2 # [ 220.708403] nix-grpc-daemon[1403]: time=2026-09-20T12:49:46.642Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6513worker2 # [ 220.709891] nix-grpc-daemon[1403]: time=2026-09-20T12:49:46.642Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6514lb # [ 220.420382] niks3-server[1094]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6515lb # [ 220.432410] niks3-server[1094]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6516worker2 # [ 220.760783] nix-grpc-daemon[1403]: time=2026-09-20T12:49:46.697Z level=INFO msg="Uploading 1 narinfos"6517lb # [ 220.442985] niks3-server[1094]: 2026/09/20 12:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6518lb # [ 220.450113] niks3-server[1094]: 2026/09/20 12:49:46 INFO Signed narinfos id=67 count=16519lb # [ 220.453952] niks3-server[1094]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6520lb # [ 220.459098] niks3-server[1094]: 2026/09/20 12:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6521lb # [ 220.463908] niks3-server[1094]: 2026/09/20 12:49:46 INFO Completed upload id=676522worker2 # [ 220.783952] nix-grpc-daemon[1403]: time=2026-09-20T12:49:46.721Z level=INFO msg="Upload complete. (297ms)"6523lb # [ 220.466115] niks3-server[1094]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6524worker2 # [ 220.797738] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16525worker2 # [ 220.814171] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06526worker2 # [ 220.818607] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16527worker2 # [ 220.823758] nix-daemon[841]: reaped child process 1647, status = succeeded6528worker2 # [ 220.845253] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526529worker2 # [ 220.866011] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400526530worker2 # [ 220.881050] nix-grpc-daemon[1403]: time=2026-09-20T12:49:46.818Z level=INFO msg="All 2 paths already cached"6531client # copying 1 paths to 'lb:50051'...6532client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6533worker2 # [ 220.892052] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400526534worker2 # [ 220.905749] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6535worker2 # [ 221.113615] nix-daemon[841]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6536worker2 # [ 221.134644] nix-daemon[841]: reaped child process 1676, status = succeeded6537worker2 # [ 221.312824] nix-daemon[841]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)6538worker2 # [ 221.331208] nix-daemon[841]: reaped child process 1681, status = succeeded6539lb # [ 221.022205] niks3-server[1094]: 2026/09/20 12:49:47 INFO Received uploads request method=POST path=/api/pending_closures6540worker2 # [ 221.352068] nix-grpc-daemon[1403]: time=2026-09-20T12:49:47.289Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6541worker2 # [ 221.353597] nix-grpc-daemon[1403]: time=2026-09-20T12:49:47.289Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6542lb # [ 221.054245] niks3-server[1094]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6543worker2 # [ 221.388240] nix-grpc-daemon[1403]: time=2026-09-20T12:49:47.325Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 221.070954] niks3-server[1094]: 2026/09/20 12:49:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6545lb # [ 221.081590] niks3-server[1094]: 2026/09/20 12:49:47 INFO Signed narinfos id=68 count=16546lb # [ 221.090314] niks3-server[1094]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6547worker2 # [ 221.415397] nix-grpc-daemon[1403]: time=2026-09-20T12:49:47.352Z level=INFO msg="Upload complete. (493ms)"6548lb # [ 221.095408] niks3-server[1094]: 2026/09/20 12:49:47 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6549lb # [ 221.100390] niks3-server[1094]: 2026/09/20 12:49:47 INFO Completed upload id=686550lb # [ 221.103410] niks3-server[1094]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6551worker2 # [ 221.424640] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=0 paths=1 nar_bytes_in=7606552worker2 # [ 221.438054] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400526553worker2 # [ 221.443484] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6554worker2 # [ 221.447993] nix-daemon[841]: reaped child process 1671, status = succeeded6555client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6556client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6557worker2 # [ 221.461646] nix-daemon[841]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)6558client # farm-top> LOG-farm-top-drain6559worker2 # [ 222.509717] nix-daemon[841]: reaped child process 1685, status = succeeded6560worker2 # [ 222.686825] nix-daemon[841]: accepted connection from pid 1700, user nix-grpc-daemon (trusted)6561worker2 # [ 222.704760] nix-daemon[841]: reaped child process 1703, status = succeeded6562lb # [ 222.396790] niks3-server[1094]: 2026/09/20 12:49:48 INFO Received uploads request method=POST path=/api/pending_closures6563worker2 # [ 222.728484] nix-grpc-daemon[1403]: time=2026-09-20T12:49:48.665Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6564worker2 # [ 222.730306] nix-grpc-daemon[1403]: time=2026-09-20T12:49:48.665Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6565lb # [ 222.439869] niks3-server[1094]: 2026/09/20 12:49:48 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6566lb # [ 222.451335] niks3-server[1094]: 2026/09/20 12:49:48 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6567worker2 # [ 222.783113] nix-grpc-daemon[1403]: time=2026-09-20T12:49:48.720Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 222.464681] niks3-server[1094]: 2026/09/20 12:49:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6569lb # [ 222.471880] niks3-server[1094]: 2026/09/20 12:49:48 INFO Signed narinfos id=69 count=16570lb # [ 222.475043] niks3-server[1094]: 2026/09/20 12:49:48 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6571lb # [ 222.481807] niks3-server[1094]: 2026/09/20 12:49:48 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6572lb # [ 222.485668] niks3-server[1094]: 2026/09/20 12:49:48 INFO Completed upload id=696573worker2 # [ 222.805811] nix-grpc-daemon[1403]: time=2026-09-20T12:49:48.742Z level=INFO msg="Upload complete. (333ms)"6574lb # [ 222.487937] niks3-server[1094]: 2026/09/20 12:49:48 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6575worker2 # [ 222.817220] nix-grpc-daemon[1311]: ts=2026-09-20T12:49:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40052 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16576worker2 # [ 222.835404] nix-daemon[841]: reaped child process 1683, status = succeeded6577client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.75 seconds)6578client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6579client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6580worker1: must succeed: rm /nix/.rw-store/fill6581worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6582worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6583worker1 # [ 224.093635] nix-grpc-daemon[1710]: ts=2026-09-20T12:49:49Z level=info event=healthy reason=min_free6584worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6585(finished: subtest: low disk drains a worker and builds go to the other, in 9.55 seconds)6586(finished: run the VM test script, in 225.99 seconds)6587test script finished in 226.28s6588cleanup6589kill QemuMachine (pid 49)6590client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6591kill QemuMachine (pid 50)6592lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6593kill QemuMachine (pid 51)6594worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6595kill QemuMachine (pid 52)6596worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6597(finished: cleanup, in 1.33 seconds)6598additionally exposed symbols:6599 client, lb, worker1, worker2,6600 vlan1,6601 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh