vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #307
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.bYtru9oafP', fmt=raw size=107374182414client: QEMU running (pid 49)15client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: 176a664d-aa8a-4a92-9a1f-ee75b00a7b5919client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26client # 27client # Virtualisation disk image created.28lb # Disk image does not exist, creating the virtualisation disk image...29lb # Formatting '/build/vm-state-lb/tmp.FKAbiFNetL', fmt=raw size=107374182430lb # mke2fs 1.47.4 (6-Mar-2025)31lb # Discarding device blocks: 0/262144 done32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Filesystem UUID: 369acefe-5390-4564-8a4a-d54daae702fe34lb # Superblock backups stored on blocks:35lb # 32768, 98304, 163840, 22937636lb # 37lb # Allocating group tables: 0/8 done38lb # Writing inode tables: 0/8 done39lb # Creating journal (8192 blocks): done40lb # Writing superblocks and filesystem accounting information: 0/8 done41lb # 42lb # Virtualisation disk image created.43lb: QEMU running (pid 51)44worker2: QEMU running (pid 50)45worker2 # Disk image does not exist, creating the virtualisation disk image...46worker2 # Formatting '/build/vm-state-worker2/tmp.gwBFgSialJ', fmt=raw size=107374182447worker2 # mke2fs 1.47.4 (6-Mar-2025)48worker2 # Discarding device blocks: 0/262144 done49worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes50worker2 # Filesystem UUID: 443afb83-a879-4de8-ae82-60a3e9d83ae951worker2 # Superblock backups stored on blocks:52worker2 # 32768, 98304, 163840, 22937653worker2 # 54worker2 # Allocating group tables: 0/8 done55worker2 # Writing inode tables: 0/8 done56worker2 # Creating journal (8192 blocks): done57worker2 # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # 59worker2 # Virtualisation disk image created.60client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]61client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202662client # [ 0.000000] KASLR enabled63client # [ 0.000000] random: crng init done64client # [ 0.000000] Machine model: linux,dummy-virt65client # [ 0.000000] efi: UEFI not found.66client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT67client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]68client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]69client # [ 0.000000] Zone ranges:70client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]71client # [ 0.000000] DMA32 empty72client # [ 0.000000] Normal empty73client # [ 0.000000] Device empty74client # [ 0.000000] Movable zone start for each node75client # [ 0.000000] Early memory node ranges76client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]77client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]78client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000079client # [ 0.000000] psci: probing for conduit method from DT.80client # [ 0.000000] psci: PSCIv1.3 detected in firmware.81client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs82client # [ 0.000000] psci: Trusted OS migration not required83client # [ 0.000000] psci: SMC Calling Convention v1.184client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)85client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u31129686client # [ 0.000000] Detected PIPT I-cache on CPU087client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)88client # [ 0.000000] CPU features: detected: GICv3 CPU interface89client # [ 0.000000] CPU features: detected: Spectre-v490client # [ 0.000000] CPU features: detected: Spectre-BHB91client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3892client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2393client # [ 0.000000] alternatives: applying boot alternatives94client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test/init regInfo=/nix/store/wd1ka8jlsww7njwm410558vz4bsfdhn5-closure-info/registration console=ttyAMA0,115200n8 console=tty095client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/wd1ka8jlsww7njwm410558vz4bsfdhn5-closure-info/registration", will be passed to user space.96client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes97client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)98client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)99client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB100client # [ 0.000000] software IO TLB: area num 1.101client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB102client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)103client # [ 0.000000] Fallback order for Node 0: 0104client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216105client # [ 0.000000] Policy zone: DMA106client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off107client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1108client # [ 0.000000] allocated 3145728 bytes of page_ext109client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages110client # [ 0.000000] ftrace: allocated 294 pages with 4 groups111client # [ 0.000000] rcu: Hierarchical RCU implementation.112client # [ 0.000000] rcu: RCU event tracing is enabled.113client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.114client # [ 0.000000] Trampoline variant of Tasks RCU enabled.115client # [ 0.000000] Rude variant of Tasks RCU enabled.116client # [ 0.000000] Tracing variant of Tasks RCU enabled.117client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.118client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1119client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.120client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.121client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.122client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0123client # [ 0.000000] GICv3: 256 SPIs implemented124client # [ 0.000000] GICv3: 0 Extended SPIs implemented125client # [ 0.000000] Root IRQ handler: gic_handle_irq126client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI127client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0128client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000129client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]130client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)131client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)132client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000133client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000134client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.135client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns136client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).137client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns138client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns139client # [ 0.000040] arm-pv: using stolen time PV140client # [ 0.000699] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)141client # [ 0.000902] Console: colour dummy device 80x25142client # [ 0.000911] printk: legacy console [tty0] enabled143client # [ 0.001120] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)144client # [ 0.001127] pid_max: default: 32768 minimum: 301145client # [ 0.001213] LSM: initializing lsm=capability,landlock,yama,bpf,ima146client # [ 0.001380] landlock: Up and running.147client # [ 0.001382] Yama: becoming mindful.148client # [ 0.002063] LSM support for eBPF active149client # [ 0.002231] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)150client # [ 0.002274] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)151client # [ 0.003616] cacheinfo: Unable to detect cache hierarchy for CPU 0152client # [ 0.004445] rcu: Hierarchical SRCU implementation.153client # [ 0.004451] rcu: Max phase no-delay instances is 1000.154client # [ 0.005741] fsl-mc MSI: its@8080000 domain created155client # [ 0.005838] EFI services will not be available.156client # [ 0.005970] smp: Bringing up secondary CPUs ...157client # [ 0.005980] smp: Brought up 1 node, 1 CPU158client # [ 0.005983] SMP: Total of 1 processors activated.159client # [ 0.005990] CPU: All CPU(s) started at EL1160client # [ 0.006004] CPU features: detected: Branch Target Identification161client # [ 0.006009] CPU features: detected: ARMv8.4 Translation Table Level162client # [ 0.006012] CPU features: detected: Instruction cache invalidation not required for I/D coherence163client # [ 0.006016] CPU features: detected: Data cache clean to the PoU not required for I/D coherence164client # [ 0.006020] CPU features: detected: Common not Private translations165client # [ 0.006023] CPU features: detected: CRC32 instructions166client # [ 0.006026] CPU features: detected: Data cache clean to Point of Deep Persistence167client # [ 0.006030] CPU features: detected: Data cache clean to Point of Persistence168client # [ 0.006033] CPU features: detected: Data independent timing control (DIT)169client # [ 0.006036] CPU features: detected: E0PD170client # [ 0.006039] CPU features: detected: Enhanced Counter Virtualization171client # [ 0.006042] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)172client # [ 0.006046] CPU features: detected: Enhanced Virtualization Traps173client # [ 0.006048] CPU features: detected: Fine Grained Traps174client # [ 0.006052] CPU features: detected: Generic authentication (architected QARMA5 algorithm)175client # [ 0.006058] CPU features: detected: RCpc load-acquire (LDAPR)176client # [ 0.006061] CPU features: detected: LSE atomic instructions177client # [ 0.006064] CPU features: detected: Privileged Access Never178client # [ 0.006067] CPU features: detected: PMUv3179client # [ 0.006069] CPU features: detected: RAS Extension Support180client # [ 0.006072] CPU features: detected: RASv1p1 Extension Support181client # [ 0.006075] CPU features: detected: Random Number Generator182client # [ 0.006077] CPU features: detected: Speculation barrier (SB)183client # [ 0.006081] CPU features: detected: Stage-2 Force Write-Back184client # [ 0.006083] CPU features: detected: TLB range maintenance instructions185client # [ 0.006090] CPU features: detected: Speculative Store Bypassing Safe (SSBS)186client # [ 0.006140] alternatives: applying system-wide alternatives187client # [ 0.009675] CPU features: detected: BBM Level 2 without TLB conflict abort188client # [ 0.009923] Memory: 1406520K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)189client # [ 0.010413] devtmpfs: initialized190client # [ 0.012484] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)191client # [ 0.012514] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).192client # [ 0.012722] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL193client # [ 0.012728] 0 pages in range for non-PLT usage194client # [ 0.012729] 508288 pages in range for PLT usage195client # [ 0.012873] pinctrl core: initialized pinctrl subsystem196client # [ 0.013788] DMI not present or invalid.197client # [ 0.017543] NET: Registered PF_NETLINK/PF_ROUTE protocol family198client # [ 0.021034] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations199client # [ 0.021240] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations200client # [ 0.021468] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations201client # [ 0.021506] audit: initializing netlink subsys (disabled)202client # [ 0.021731] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1203client # [ 0.022284] thermal_sys: Registered thermal governor 'fair_share'204client # [ 0.022287] thermal_sys: Registered thermal governor 'bang_bang'205client # [ 0.022295] thermal_sys: Registered thermal governor 'step_wise'206client # [ 0.022298] thermal_sys: Registered thermal governor 'user_space'207client # [ 0.022300] thermal_sys: Registered thermal governor 'power_allocator'208client # [ 0.022337] cpuidle: using governor ladder209client # [ 0.022345] cpuidle: using governor menu210client # [ 0.022565] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.211client # [ 0.022584] ASID allocator initialised with 65536 entries212client # [ 0.023907] Serial: AMBA PL011 UART driver213client # [ 0.029495] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1214client # [ 0.029658] printk: console [ttyAMA0] enabled215client # [ 0.154596] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages216client # [ 0.154624] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page217client # [ 0.154630] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages218client # [ 0.154635] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page219client # [ 0.154639] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages220client # [ 0.154644] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page221client # [ 0.154648] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages222client # [ 0.154652] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page223client # [ 0.162573] fbcon: Taking over console224client # [ 0.162592] ACPI: Interpreter disabled.225client # [ 0.170086] iommu: Default domain type: Translated226client # [ 0.170100] iommu: DMA domain TLB invalidation policy: strict mode227client # [ 0.171810] SCSI subsystem initialized228client # [ 0.172520] usbcore: registered new interface driver usbfs229client # [ 0.172556] usbcore: registered new interface driver hub230client # [ 0.172574] usbcore: registered new device driver usb231client # [ 0.172901] pps_core: LinuxPPS API ver. 1 registered232client # [ 0.172908] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>233client # [ 0.172918] PTP clock support registered234client # [ 0.172982] EDAC MC: Ver: 3.0.0235client # [ 0.177469] scmi_core: SCMI protocol bus registered236client # [ 0.178504] FPGA manager framework237client # [ 0.179491] vgaarb: loaded238client # [ 0.182493] clocksource: Switched to clocksource arch_sys_counter239client # [ 0.183211] VFS: Disk quotas dquot_6.6.0240client # [ 0.183245] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)241client # [ 0.185555] netfs: FS-Cache loaded242client # [ 0.185696] pnp: PnP ACPI: disabled243client # [ 0.192034] NET: Registered PF_INET protocol family244client # [ 0.193011] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)245lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]246lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026247lb # [ 0.000000] KASLR enabled248lb # [ 0.000000] random: crng init done249lb # [ 0.000000] Machine model: linux,dummy-virt250lb # [ 0.000000] efi: UEFI not found.251lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT252lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]253lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]254lb # [ 0.000000] Zone ranges:255lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]256lb # [ 0.000000] DMA32 empty257lb # [ 0.000000] Normal empty258lb # [ 0.000000] Device empty259lb # [ 0.000000] Movable zone start for each node260lb # [ 0.000000] Early memory node ranges261lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]262lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]263lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000264lb # [ 0.000000] psci: probing for conduit method from DT.265lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.266lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs267lb # [ 0.000000] psci: Trusted OS migration not required268lb # [ 0.000000] psci: SMC Calling Convention v1.1269lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)270lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296271lb # [ 0.000000] Detected PIPT I-cache on CPU0272lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)273lb # [ 0.000000] CPU features: detected: GICv3 CPU interface274lb # [ 0.000000] CPU features: detected: Spectre-v4275lb # [ 0.000000] CPU features: detected: Spectre-BHB276lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38277lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23278lb # [ 0.000000] alternatives: applying boot alternatives279lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0280lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.281lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes282lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)283lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)284client # [ 0.226838] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)285client # [ 0.226910] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)286client # [ 0.226949] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)287lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB288lb # [ 0.000000] software IO TLB: area num 1.289lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB290lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)291lb # [ 0.000000] Fallback order for Node 0: 0292lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216293lb # [ 0.000000] Policy zone: DMA294lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off295lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1296lb # [ 0.000000] allocated 3145728 bytes of page_ext297client # [ 0.227054] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)298lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages299lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups300client # [ 0.227241] TCP: Hash tables configured (established 16384 bind 16384)301lb # [ 0.000000] rcu: Hierarchical RCU implementation.302lb # [ 0.000000] rcu: RCU event tracing is enabled.303client # [ 0.227362] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)304lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.305client # [ 0.227407] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)306lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.307client # [ 0.227448] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)308lb # [ 0.000000] Rude variant of Tasks RCU enabled.309lb # [ 0.000000] Tracing variant of Tasks RCU enabled.310lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.311lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1312lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.313lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.314lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.315lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0316lb # [ 0.000000] GICv3: 256 SPIs implemented317lb # [ 0.000000] GICv3: 0 Extended SPIs implemented318lb # [ 0.000000] Root IRQ handler: gic_handle_irq319lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI320worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]321worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026322worker2 # [ 0.000000] KASLR enabled323worker2 # [ 0.000000] random: crng init done324worker2 # [ 0.000000] Machine model: linux,dummy-virt325worker2 # [ 0.000000] efi: UEFI not found.326worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT327worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]328worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]329worker2 # [ 0.000000] Zone ranges:330lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0331lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000332lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]333lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)334lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)335lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000336lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000337lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.338lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns339lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).340worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]341lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns342worker2 # [ 0.000000] DMA32 empty343worker2 # [ 0.000000] Normal empty344lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns345worker2 # [ 0.000000] Device empty346lb # [ 0.000034] arm-pv: using stolen time PV347worker2 # [ 0.000000] Movable zone start for each node348lb # [ 0.000418] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)349lb # [ 0.000621] Console: colour dummy device 80x25350lb # [ 0.000630] printk: legacy console [tty0] enabled351lb # [ 0.000853] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)352lb # [ 0.000860] pid_max: default: 32768 minimum: 301353lb # [ 0.000942] LSM: initializing lsm=capability,landlock,yama,bpf,ima354lb # [ 0.001109] landlock: Up and running.355lb # [ 0.001112] Yama: becoming mindful.356lb # [ 0.001610] LSM support for eBPF active357lb # [ 0.001775] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)358lb # [ 0.001818] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)359lb # [ 0.003015] cacheinfo: Unable to detect cache hierarchy for CPU 0360lb # [ 0.004100] rcu: Hierarchical SRCU implementation.361worker2 # [ 0.000000] Early memory node ranges362client # [ 0.263562] NET: Registered PF_UNIX/PF_LOCAL protocol family363client # [ 0.263653] NET: Registered PF_XDP protocol family364client # [ 0.263678] PCI: CLS 0 bytes, default 64365client # [ 0.263969] Trying to unpack rootfs image as initramfs...366client # [ 0.277865] kvm [1]: HYP mode not available367lb # [ 0.004106] rcu: Max phase no-delay instances is 1000.368lb # [ 0.005359] fsl-mc MSI: its@8080000 domain created369lb # [ 0.005465] EFI services will not be available.370lb # [ 0.005592] smp: Bringing up secondary CPUs ...371lb # [ 0.005601] smp: Brought up 1 node, 1 CPU372lb # [ 0.005604] SMP: Total of 1 processors activated.373lb # [ 0.005610] CPU: All CPU(s) started at EL1374lb # [ 0.005622] CPU features: detected: Branch Target Identification375lb # [ 0.005628] CPU features: detected: ARMv8.4 Translation Table Level376lb # [ 0.005631] CPU features: detected: Instruction cache invalidation not required for I/D coherence377lb # [ 0.005634] CPU features: detected: Data cache clean to the PoU not required for I/D coherence378lb # [ 0.005638] CPU features: detected: Common not Private translations379lb # [ 0.005641] CPU features: detected: CRC32 instructions380lb # [ 0.005644] CPU features: detected: Data cache clean to Point of Deep Persistence381lb # [ 0.005648] CPU features: detected: Data cache clean to Point of Persistence382lb # [ 0.005652] CPU features: detected: Data independent timing control (DIT)383lb # [ 0.005655] CPU features: detected: E0PD384lb # [ 0.005657] CPU features: detected: Enhanced Counter Virtualization385lb # [ 0.005660] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)386lb # [ 0.005664] CPU features: detected: Enhanced Virtualization Traps387lb # [ 0.005666] CPU features: detected: Fine Grained Traps388lb # [ 0.005670] CPU features: detected: Generic authentication (architected QARMA5 algorithm)389lb # [ 0.005675] CPU features: detected: RCpc load-acquire (LDAPR)390lb # [ 0.005678] CPU features: detected: LSE atomic instructions391worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]392worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]393worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000394worker2 # [ 0.000000] psci: probing for conduit method from DT.395worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.396worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs397worker2 # [ 0.000000] psci: Trusted OS migration not required398worker2 # [ 0.000000] psci: SMC Calling Convention v1.1399worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)400worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296401worker2 # [ 0.000000] Detected PIPT I-cache on CPU0402worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)403worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface404worker2 # [ 0.000000] CPU features: detected: Spectre-v4405worker2 # [ 0.000000] CPU features: detected: Spectre-BHB406worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38407worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23408worker2 # [ 0.000000] alternatives: applying boot alternatives409worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0r31wjlygcn47dyyzl916b35v9f3ng6k-nixos-system-worker2-test/init regInfo=/nix/store/ik3fad33q44if18fhw4ncajdi342n3lv-closure-info/registration console=ttyAMA0,115200n8 console=tty0410worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/ik3fad33q44if18fhw4ncajdi342n3lv-closure-info/registration", will be passed to user space.411worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes412worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)413worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)414worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB415worker2 # [ 0.000000] software IO TLB: area num 1.416worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB417worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)418worker2 # [ 0.000000] Fallback order for Node 0: 0419worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216420worker2 # [ 0.000000] Policy zone: DMA421worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off422lb # [ 0.005682] CPU features: detected: Privileged Access Never423worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1424worker2 # [ 0.000000] allocated 3145728 bytes of page_ext425lb # [ 0.005684] CPU features: detected: PMUv3426lb # [ 0.005687] CPU features: detected: RAS Extension Support427worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages428lb # [ 0.005690] CPU features: detected: RASv1p1 Extension Support429lb # [ 0.005692] CPU features: detected: Random Number Generator430lb # [ 0.005695] CPU features: detected: Speculation barrier (SB)431lb # [ 0.005698] CPU features: detected: Stage-2 Force Write-Back432lb # [ 0.005701] CPU features: detected: TLB range maintenance instructions433lb # [ 0.005706] CPU features: detected: Speculative Store Bypassing Safe (SSBS)434lb # [ 0.005749] alternatives: applying system-wide alternatives435lb # [ 0.008971] CPU features: detected: BBM Level 2 without TLB conflict abort436lb # [ 0.009155] Memory: 1406548K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)437lb # [ 0.009591] devtmpfs: initialized438lb # [ 0.011581] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)439lb # [ 0.011608] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).440lb # [ 0.011812] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL441lb # [ 0.011817] 0 pages in range for non-PLT usage442lb # [ 0.011818] 508288 pages in range for PLT usage443lb # [ 0.011966] pinctrl core: initialized pinctrl subsystem444worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups445worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.446worker2 # [ 0.000000] rcu: RCU event tracing is enabled.447worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.448worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.449lb # [ 0.012810] DMI not present or invalid.450worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.451worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.452worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.453worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1454worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.455worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.456worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.457worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0458worker2 # [ 0.000000] GICv3: 256 SPIs implemented459worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented460worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq461worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI462worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0463lb # [ 0.016219] NET: Registered PF_NETLINK/PF_ROUTE protocol family464worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000465worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]466lb # [ 0.019700] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations467lb # [ 0.019916] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations468worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)469lb # [ 0.020141] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations470worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)471lb # [ 0.020177] audit: initializing netlink subsys (disabled)472worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000473lb # [ 0.020839] thermal_sys: Registered thermal governor 'fair_share'474lb # [ 0.020842] thermal_sys: Registered thermal governor 'bang_bang'475lb # [ 0.020846] thermal_sys: Registered thermal governor 'step_wise'476lb # [ 0.020851] thermal_sys: Registered thermal governor 'user_space'477lb # [ 0.020854] thermal_sys: Registered thermal governor 'power_allocator'478lb # [ 0.020882] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1479lb # [ 0.020890] cpuidle: using governor ladder480lb # [ 0.020896] cpuidle: using governor menu481lb # [ 0.021129] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.482lb # [ 0.021151] ASID allocator initialised with 65536 entries483worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000484worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.485worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns486worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).487worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns488worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns489worker2 # [ 0.000032] arm-pv: using stolen time PV490lb # [ 0.022397] Serial: AMBA PL011 UART driver491worker2 # [ 0.000450] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)492worker2 # [ 0.000635] Console: colour dummy device 80x25493lb # [ 0.027885] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1494worker2 # [ 0.000645] printk: legacy console [tty0] enabled495lb # [ 0.028090] printk: console [ttyAMA0] enabled496worker2 # [ 0.000842] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)497worker2 # [ 0.000849] pid_max: default: 32768 minimum: 301498worker2 # [ 0.000933] LSM: initializing lsm=capability,landlock,yama,bpf,ima499worker2 # [ 0.046762] landlock: Up and running.500worker2 # [ 0.046777] Yama: becoming mindful.501worker2 # [ 0.047416] LSM support for eBPF active502worker2 # [ 0.047690] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)503worker2 # [ 0.047726] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)504worker2 # [ 0.049183] cacheinfo: Unable to detect cache hierarchy for CPU 0505worker2 # [ 0.050076] rcu: Hierarchical SRCU implementation.506worker2 # [ 0.050084] rcu: Max phase no-delay instances is 1000.507worker2 # [ 0.051445] fsl-mc MSI: its@8080000 domain created508lb # [ 0.144010] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages509worker2 # [ 0.051549] EFI services will not be available.510lb # [ 0.144039] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page511lb # [ 0.144044] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages512lb # [ 0.144049] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page513lb # [ 0.144053] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages514lb # [ 0.144058] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page515worker2 # [ 0.051689] smp: Bringing up secondary CPUs ...516worker2 # [ 0.051700] smp: Brought up 1 node, 1 CPU517worker2 # [ 0.051703] SMP: Total of 1 processors activated.518worker2 # [ 0.051712] CPU: All CPU(s) started at EL1519worker2 # [ 0.051725] CPU features: detected: Branch Target Identification520worker2 # [ 0.051731] CPU features: detected: ARMv8.4 Translation Table Level521lb # [ 0.144062] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages522worker2 # [ 0.051734] CPU features: detected: Instruction cache invalidation not required for I/D coherence523worker2 # [ 0.051738] CPU features: detected: Data cache clean to the PoU not required for I/D coherence524lb # [ 0.144066] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page525lb # [ 0.151681] fbcon: Taking over console526lb # [ 0.151704] ACPI: Interpreter disabled.527worker2 # [ 0.051741] CPU features: detected: Common not Private translations528worker2 # [ 0.051744] CPU features: detected: CRC32 instructions529worker2 # [ 0.051747] CPU features: detected: Data cache clean to Point of Deep Persistence530worker2 # [ 0.051750] CPU features: detected: Data cache clean to Point of Persistence531worker2 # [ 0.051754] CPU features: detected: Data independent timing control (DIT)532worker2 # [ 0.051757] CPU features: detected: E0PD533worker2 # [ 0.051759] CPU features: detected: Enhanced Counter Virtualization534worker2 # [ 0.051762] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)535worker2 # [ 0.051766] CPU features: detected: Enhanced Virtualization Traps536worker2 # [ 0.051769] CPU features: detected: Fine Grained Traps537worker2 # [ 0.051772] CPU features: detected: Generic authentication (architected QARMA5 algorithm)538worker2 # [ 0.051777] CPU features: detected: RCpc load-acquire (LDAPR)539worker2 # [ 0.051779] CPU features: detected: LSE atomic instructions540worker2 # [ 0.051783] CPU features: detected: Privileged Access Never541worker2 # [ 0.051786] CPU features: detected: PMUv3542worker2 # [ 0.051788] CPU features: detected: RAS Extension Support543worker2 # [ 0.051791] CPU features: detected: RASv1p1 Extension Support544worker2 # [ 0.051793] CPU features: detected: Random Number Generator545lb # [ 0.160825] iommu: Default domain type: Translated546lb # [ 0.160840] iommu: DMA domain TLB invalidation policy: strict mode547lb # [ 0.161239] SCSI subsystem initialized548lb # [ 0.163259] usbcore: registered new interface driver usbfs549lb # [ 0.163293] usbcore: registered new interface driver hub550lb # [ 0.163328] usbcore: registered new device driver usb551lb # [ 0.163645] pps_core: LinuxPPS API ver. 1 registered552lb # [ 0.163651] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>553worker2 # [ 0.051796] CPU features: detected: Speculation barrier (SB)554worker2 # [ 0.051799] CPU features: detected: Stage-2 Force Write-Back555worker2 # [ 0.051801] CPU features: detected: TLB range maintenance instructions556worker2 # [ 0.051806] CPU features: detected: Speculative Store Bypassing Safe (SSBS)557worker2 # [ 0.051847] alternatives: applying system-wide alternatives558worker2 # [ 0.055465] CPU features: detected: BBM Level 2 without TLB conflict abort559lb # [ 0.163662] PTP clock support registered560lb # [ 0.163730] EDAC MC: Ver: 3.0.0561lb # [ 0.168234] scmi_core: SCMI protocol bus registered562worker2 # [ 0.055686] Memory: 1406572K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)563worker2 # [ 0.056166] devtmpfs: initialized564lb # [ 0.169322] FPGA manager framework565worker2 # [ 0.058250] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)566worker2 # [ 0.058276] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).567lb # [ 0.170429] vgaarb: loaded568worker2 # [ 0.058490] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL569worker2 # [ 0.058496] 0 pages in range for non-PLT usage570worker2 # [ 0.058497] 508288 pages in range for PLT usage571worker2 # [ 0.058623] pinctrl core: initialized pinctrl subsystem572worker2 # [ 0.059600] DMI not present or invalid.573lb # [ 0.174018] clocksource: Switched to clocksource arch_sys_counter574worker2 # [ 0.063770] NET: Registered PF_NETLINK/PF_ROUTE protocol family575lb # [ 0.174728] VFS: Disk quotas dquot_6.6.0576worker2 # [ 0.066604] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations577lb # [ 0.174770] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)578worker2 # [ 0.066831] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations579worker2 # [ 0.067053] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations580worker2 # [ 0.067091] audit: initializing netlink subsys (disabled)581worker2 # [ 0.067540] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1582worker2 # [ 0.067842] thermal_sys: Registered thermal governor 'fair_share'583worker2 # [ 0.067845] thermal_sys: Registered thermal governor 'bang_bang'584worker2 # [ 0.067852] thermal_sys: Registered thermal governor 'step_wise'585worker2 # [ 0.067855] thermal_sys: Registered thermal governor 'user_space'586worker2 # [ 0.067858] thermal_sys: Registered thermal governor 'power_allocator'587lb # [ 0.177049] netfs: FS-Cache loaded588lb # [ 0.177202] pnp: PnP ACPI: disabled589worker2 # [ 0.067884] cpuidle: using governor ladder590worker2 # [ 0.067893] cpuidle: using governor menu591worker2 # [ 0.068111] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.592worker2 # [ 0.068132] ASID allocator initialised with 65536 entries593worker2 # [ 0.069597] Serial: AMBA PL011 UART driver594worker2 # [ 0.075194] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1595worker2 # [ 0.075376] printk: console [ttyAMA0] enabled596lb # [ 0.183574] NET: Registered PF_INET protocol family597lb # [ 0.183868] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)598worker2 # [ 0.192769] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages599worker2 # [ 0.192798] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page600worker2 # [ 0.192804] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages601worker2 # [ 0.192808] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page602worker2 # [ 0.192812] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages603worker2 # [ 0.192817] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page604worker2 # [ 0.192821] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages605worker2 # [ 0.192825] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page606worker2 # [ 0.200510] fbcon: Taking over console607worker2 # [ 0.200534] ACPI: Interpreter disabled.608worker2 # [ 0.209665] iommu: Default domain type: Translated609worker2 # [ 0.209681] iommu: DMA domain TLB invalidation policy: strict mode610worker2 # [ 0.211440] SCSI subsystem initialized611worker2 # [ 0.212157] usbcore: registered new interface driver usbfs612worker2 # [ 0.212192] usbcore: registered new interface driver hub613worker2 # [ 0.212220] usbcore: registered new device driver usb614worker2 # [ 0.212535] pps_core: LinuxPPS API ver. 1 registered615worker2 # [ 0.212542] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>616worker2 # [ 0.212553] PTP clock support registered617worker2 # [ 0.212607] EDAC MC: Ver: 3.0.0618worker2 # [ 0.217251] scmi_core: SCMI protocol bus registered619worker2 # [ 0.218307] FPGA manager framework620worker2 # [ 0.219452] vgaarb: loaded621worker2 # [ 0.220139] clocksource: Switched to clocksource arch_sys_counter622lb # [ 0.218172] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)623worker2 # [ 0.226521] VFS: Disk quotas dquot_6.6.0624lb # [ 0.218240] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)625worker2 # [ 0.226592] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)626lb # [ 0.218277] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)627lb # [ 0.218378] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)628worker2 # [ 0.228265] netfs: FS-Cache loaded629lb # [ 0.218521] TCP: Hash tables configured (established 16384 bind 16384)630worker2 # [ 0.228390] pnp: PnP ACPI: disabled631lb # [ 0.218667] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)632lb # [ 0.218712] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)633lb # [ 0.218752] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)634lb # [ 0.218871] NET: Registered PF_UNIX/PF_LOCAL protocol family635lb # [ 0.218897] NET: Registered PF_XDP protocol family636lb # [ 0.218918] PCI: CLS 0 bytes, default 64637lb # [ 0.219219] Trying to unpack rootfs image as initramfs...638worker2 # [ 0.232903] NET: Registered PF_INET protocol family639worker2 # [ 0.233209] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)640lb # [ 0.236838] kvm [1]: HYP mode not available641worker2 # [ 0.266937] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)642worker2 # [ 0.267007] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)643worker2 # [ 0.267048] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)644worker2 # [ 0.267145] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)645worker2 # [ 0.267292] TCP: Hash tables configured (established 16384 bind 16384)646worker2 # [ 0.267404] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)647worker2 # [ 0.267451] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)648worker2 # [ 0.267492] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)649worker2 # [ 0.267600] NET: Registered PF_UNIX/PF_LOCAL protocol family650worker2 # [ 0.267634] NET: Registered PF_XDP protocol family651worker2 # [ 0.267654] PCI: CLS 0 bytes, default 64652worker2 # [ 0.267926] Trying to unpack rootfs image as initramfs...653worker2 # [ 0.285116] kvm [1]: HYP mode not available654client # [ 0.551709] Initialise system trusted keyrings655client # [ 0.552599] workingset: timestamp_bits=42 max_order=19 bucket_order=0656client # [ 0.553886] squashfs: version 4.0 (2009/01/31) Phillip Lougher657client # [ 0.553947] 9p: Installing v9fs 9p2000 file system support658client # [ 0.570386] Key type asymmetric registered659client # [ 0.570417] Asymmetric key parser 'x509' registered660client # [ 0.570562] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)661client # [ 0.574624] io scheduler mq-deadline registered662client # [ 0.574641] io scheduler kyber registered663client # [ 0.584421] pl061_gpio 9030000.pl061: PL061 GPIO chip registered664client # [ 0.585897] ledtrig-cpu: registered to indicate activity on CPUs665client # [ 0.586345] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:666client # [ 0.586365] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000667client # [ 0.586378] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000668client # [ 0.586386] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000669client # [ 0.586426] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits670client # [ 0.586453] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]671client # [ 0.586541] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00672client # [ 0.586551] pci_bus 0000:00: root bus resource [bus 00-ff]673client # [ 0.586558] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]674client # [ 0.586563] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]675client # [ 0.586568] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]676client # [ 0.586634] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint677client # [ 0.587126] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint678client # [ 0.587316] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]679client # [ 0.587332] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]680client # [ 0.587362] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]681client # [ 0.587379] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]682client # [ 0.587855] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint683client # [ 0.588044] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]684client # [ 0.588060] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]685client # [ 0.588093] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]686lb # [ 0.430851] Initialise system trusted keyrings687lb # [ 0.431749] workingset: timestamp_bits=42 max_order=19 bucket_order=0688lb # [ 0.433042] squashfs: version 4.0 (2009/01/31) Phillip Lougher689lb # [ 0.433784] 9p: Installing v9fs 9p2000 file system support690client # [ 0.616771] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint691client # [ 0.617004] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]692client # [ 0.617023] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]693client # [ 0.617055] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]694client # [ 0.617545] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint695client # [ 0.617732] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]696client # [ 0.617749] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]697client # [ 0.617782] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]698client # [ 0.618254] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint699client # [ 0.618440] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]700client # [ 0.618456] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]701client # [ 0.618486] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]702client # [ 0.618952] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint703client # [ 0.632519] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]704client # [ 0.632561] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]705client # [ 0.632592] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]706client # [ 0.633275] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint707client # [ 0.633477] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]708client # [ 0.633494] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]709client # [ 0.633524] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]710client # [ 0.633541] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]711client # [ 0.634016] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint712client # [ 0.634207] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]713client # [ 0.634239] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]714client # [ 0.634726] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint715client # [ 0.634921] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]716client # [ 0.634952] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]717client # [ 0.635382] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint718client # [ 0.635568] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]719client # [ 0.635819] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint720client # [ 0.636131] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]721client # [ 0.656203] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]722client # [ 0.656289] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]723client # [ 0.656860] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint724client # [ 0.657059] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]725client # [ 0.657076] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]726client # [ 0.657106] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]727worker2 # [ 0.479527] Initialise system trusted keyrings728worker2 # [ 0.480393] workingset: timestamp_bits=42 max_order=19 bucket_order=0729worker2 # [ 0.481747] squashfs: version 4.0 (2009/01/31) Phillip Lougher730worker2 # [ 0.482659] 9p: Installing v9fs 9p2000 file system support731client # [ 0.659904] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned732client # [ 0.659918] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned733client # [ 0.659924] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned734client # [ 0.659975] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned735client # [ 0.660026] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned736client # [ 0.660075] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned737client # [ 0.660123] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned738client # [ 0.676298] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned739client # [ 0.676371] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned740client # [ 0.676429] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned741client # [ 0.676479] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned742client # [ 0.676532] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned743client # [ 0.676731] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned744client # [ 0.676784] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned745client # [ 0.676808] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned746client # [ 0.676832] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned747client # [ 0.676855] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned748client # [ 0.676884] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned749client # [ 0.676907] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned750client # [ 0.676929] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned751client # [ 0.676951] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned752worker2 # [ 0.506961] Key type asymmetric registered753client # [ 0.676973] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned754worker2 # [ 0.506996] Asymmetric key parser 'x509' registered755client # [ 0.676996] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned756worker2 # [ 0.507108] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)757client # [ 0.677020] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned758client # [ 0.677047] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned759client # [ 0.677070] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned760client # [ 0.677093] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned761client # [ 0.677115] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned762client # [ 0.677137] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned763client # [ 0.677159] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned764client # [ 0.677182] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned765client # [ 0.677206] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned766client # [ 0.677229] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned767client # [ 0.677250] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned768client # [ 0.677285] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]769client # [ 0.677295] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]770client # [ 0.677301] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]771worker2 # [ 0.516222] io scheduler mq-deadline registered772client # [ 0.678509] pci 0000:00:0a.0: enabling device (0000 -> 0002)773worker2 # [ 0.516255] io scheduler kyber registered774worker2 # [ 0.521838] pl061_gpio 9030000.pl061: PL061 GPIO chip registered775worker2 # [ 0.523198] ledtrig-cpu: registered to indicate activity on CPUs776worker2 # [ 0.523629] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:777worker2 # [ 0.523650] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000778worker2 # [ 0.523664] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000779worker2 # [ 0.523673] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000780worker2 # [ 0.523702] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits781worker2 # [ 0.523728] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]782worker2 # [ 0.523813] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00783worker2 # [ 0.523823] pci_bus 0000:00: root bus resource [bus 00-ff]784worker2 # [ 0.523829] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]785worker2 # [ 0.523835] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]786worker2 # [ 0.523840] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]787worker2 # [ 0.523909] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint788worker2 # [ 0.524390] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint789worker2 # [ 0.524576] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]790worker2 # [ 0.524593] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]791worker2 # [ 0.524623] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]792worker2 # [ 0.524639] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]793worker2 # [ 0.525090] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint794worker2 # [ 0.525271] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]795worker2 # [ 0.525288] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]796worker2 # [ 0.525317] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]797worker2 # [ 0.525770] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint798worker2 # [ 0.525956] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]799worker2 # [ 0.525972] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]800worker2 # [ 0.526001] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]801worker2 # [ 0.526484] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint802worker2 # [ 0.526664] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]803worker2 # [ 0.526680] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]804worker2 # [ 0.526724] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]805worker2 # [ 0.527179] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint806worker2 # [ 0.527361] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]807client # [ 0.731480] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)808worker2 # [ 0.527377] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]809worker2 # [ 0.527407] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]810worker2 # [ 0.527864] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint811worker2 # [ 0.528049] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]812worker2 # [ 0.528066] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]813worker2 # [ 0.528095] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]814client # [ 0.741952] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)815client # [ 0.746396] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)816lb # [ 0.457877] Key type asymmetric registered817lb # [ 0.457918] Asymmetric key parser 'x509' registered818client # [ 0.751025] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)819worker2 # [ 0.564755] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint820worker2 # [ 0.564978] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]821worker2 # [ 0.564996] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]822worker2 # [ 0.565025] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]823worker2 # [ 0.565042] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]824worker2 # [ 0.565521] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint825worker2 # [ 0.565713] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]826client # [ 0.758983] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)827worker2 # [ 0.565745] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]828worker2 # [ 0.566231] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint829worker2 # [ 0.566417] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]830worker2 # [ 0.566447] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]831worker2 # [ 0.566836] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint832worker2 # [ 0.567020] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]833worker2 # [ 0.567263] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint834worker2 # [ 0.567592] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]835worker2 # [ 0.567611] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]836worker2 # [ 0.567640] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]837worker2 # [ 0.568118] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint838client # [ 0.765354] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)839client # [ 0.767793] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)840worker2 # [ 0.568334] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]841worker2 # [ 0.568350] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]842worker2 # [ 0.568379] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]843worker2 # [ 0.569001] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned844worker2 # [ 0.569013] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned845worker2 # [ 0.569018] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned846worker2 # [ 0.569065] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned847worker2 # [ 0.569113] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned848worker2 # [ 0.569160] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned849worker2 # [ 0.569218] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned850worker2 # [ 0.569265] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned851worker2 # [ 0.569313] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned852worker2 # [ 0.569360] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned853client # [ 0.781886] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)854worker2 # [ 0.569412] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned855worker2 # [ 0.569460] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned856worker2 # [ 0.569556] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned857worker2 # [ 0.569603] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned858worker2 # [ 0.569626] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned859worker2 # [ 0.569648] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned860worker2 # [ 0.569669] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned861worker2 # [ 0.569696] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned862worker2 # [ 0.569718] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned863worker2 # [ 0.569739] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned864worker2 # [ 0.569761] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned865worker2 # [ 0.569783] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned866worker2 # [ 0.569805] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned867worker2 # [ 0.569829] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned868worker2 # [ 0.569853] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned869client # [ 0.788414] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)870worker2 # [ 0.569875] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned871client # [ 0.790610] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)872worker2 # [ 0.569896] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned873worker2 # [ 0.569917] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned874worker2 # [ 0.569939] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned875worker2 # [ 0.569960] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned876worker2 # [ 0.569984] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned877worker2 # [ 0.570006] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned878worker2 # [ 0.570044] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned879worker2 # [ 0.570067] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned880worker2 # [ 0.570094] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]881worker2 # [ 0.570104] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]882worker2 # [ 0.570108] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]883worker2 # [ 0.571263] pci 0000:00:0a.0: enabling device (0000 -> 0002)884client # [ 0.805949] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)885client # [ 0.820879] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled886client # [ 0.823804] msm_serial: driver initialized887client # [ 0.823962] SuperH (H)SCI(F) driver initialized888client # [ 0.824017] STM32 USART driver initialized889worker2 # [ 0.648358] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)890worker2 # [ 0.650942] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)891worker2 # [ 0.669777] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)892worker2 # [ 0.677457] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)893lb # [ 0.679261] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)894lb # [ 0.680439] io scheduler mq-deadline registered895lb # [ 0.680455] io scheduler kyber registered896lb # [ 0.691328] pl061_gpio 9030000.pl061: PL061 GPIO chip registered897client # [ 0.873959] loop: module loaded898client # [ 0.874177] virtio_blk virtio5: 1/0/0 default/read/poll queues899client # [ 0.875026] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)900lb # [ 0.692859] ledtrig-cpu: registered to indicate activity on CPUs901worker2 # [ 0.689706] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)902lb # [ 0.693321] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:903lb # [ 0.693342] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000904lb # [ 0.693356] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000905lb # [ 0.693364] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000906lb # [ 0.693390] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits907lb # [ 0.693419] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]908lb # [ 0.693518] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00909lb # [ 0.693528] pci_bus 0000:00: root bus resource [bus 00-ff]910lb # [ 0.693535] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]911worker2 # [ 0.695486] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)912lb # [ 0.693539] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]913lb # [ 0.693544] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]914lb # [ 0.693609] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint915lb # [ 0.694058] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint916lb # [ 0.694247] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]917lb # [ 0.694269] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]918lb # [ 0.694299] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]919lb # [ 0.694316] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]920lb # [ 0.694778] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint921worker2 # [ 0.704190] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)922lb # [ 0.694967] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]923lb # [ 0.694983] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]924lb # [ 0.695014] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]925client # [ 0.897132] megasas: 07.734.00.00-rc1926client # [ 0.898041] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]927worker2 # [ 0.716543] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)928worker2 # [ 0.719291] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)929lb # [ 0.719724] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint930lb # [ 0.719947] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]931lb # [ 0.719967] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]932client # [ 0.909619] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000933lb # [ 0.719996] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]934client # [ 0.909675] Intel/Sharp Extended Query Table at 0x0031935client # [ 0.912106] Using buffer write method936lb # [ 0.720471] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint937lb # [ 0.720658] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]938lb # [ 0.720675] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]939lb # [ 0.720707] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]940lb # [ 0.721185] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint941lb # [ 0.721369] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]942lb # [ 0.721385] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]943worker2 # [ 0.730062] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)944lb # [ 0.721415] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]945lb # [ 0.721870] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint946lb # [ 0.722059] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]947worker2 # [ 0.734635] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)948lb # [ 0.722075] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]949lb # [ 0.722106] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]950lb # [ 0.722574] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint951lb # [ 0.722763] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]952lb # [ 0.722780] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]953client # [ 0.920299] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]954lb # [ 0.722809] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955lb # [ 0.722826] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]956client # [ 0.923041] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000957lb # [ 0.723352] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint958client # [ 0.923169] Intel/Sharp Extended Query Table at 0x0031959lb # [ 0.723539] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]960lb # [ 0.723569] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]961lb # [ 0.724022] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint962lb # [ 0.724219] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]963lb # [ 0.724249] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]964lb # [ 0.724638] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint965lb # [ 0.724823] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]966lb # [ 0.725085] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint967lb # [ 0.725426] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]968client # [ 0.925126] Using buffer write method969client # [ 0.925173] Concatenating MTD devices:970client # [ 0.925177] (0): "0.flash"971client # [ 0.925181] (1): "0.flash"972client # [ 0.925185] into device "0.flash"973lb # [ 0.725445] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]974lb # [ 0.725476] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975lb # [ 0.725950] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint976worker2 # [ 0.750563] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled977lb # [ 0.726134] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]978lb # [ 0.726150] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]979lb # [ 0.726179] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980lb # [ 0.726792] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned981lb # [ 0.726805] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned982lb # [ 0.726811] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned983lb # [ 0.726855] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned984lb # [ 0.726902] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned985lb # [ 0.726948] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned986worker2 # [ 0.757674] msm_serial: driver initialized987worker2 # [ 0.757834] SuperH (H)SCI(F) driver initialized988worker2 # [ 0.757891] STM32 USART driver initialized989lb # [ 0.726993] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned990lb # [ 0.727039] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned991lb # [ 0.727086] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned992lb # [ 0.775287] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned993lb # [ 0.775407] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned994lb # [ 0.775458] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned995lb # [ 0.775630] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned996lb # [ 0.775686] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned997lb # [ 0.775709] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned998lb # [ 0.775733] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned999lb # [ 0.775757] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1000lb # [ 0.775789] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1001lb # [ 0.775813] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1002lb # [ 0.775838] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1003lb # [ 0.775861] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1004lb # [ 0.775884] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1005lb # [ 0.775907] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1006lb # [ 0.775933] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1007lb # [ 0.775962] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1008lb # [ 0.775987] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1009lb # [ 0.776009] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1010lb # [ 0.776032] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1011lb # [ 0.776054] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1012lb # [ 0.776076] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1013lb # [ 0.776099] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1014lb # [ 0.776121] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1015lb # [ 0.776145] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1016lb # [ 0.776168] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1017lb # [ 0.776206] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1018lb # [ 0.776216] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1019lb # [ 0.776221] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1020lb # [ 0.777404] pci 0000:00:0a.0: enabling device (0000 -> 0002)1021worker2 # [ 0.797249] loop: module loaded1022worker2 # [ 0.797505] virtio_blk virtio5: 1/0/0 default/read/poll queues1023worker2 # [ 0.803967] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1024worker2 # [ 0.816385] megasas: 07.734.00.00-rc11025worker2 # [ 0.817432] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1026lb # [ 0.822123] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1027lb # [ 0.832878] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1028lb # [ 0.837112] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1029worker2 # [ 0.830180] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001030worker2 # [ 0.830230] Intel/Sharp Extended Query Table at 0x00311031lb # [ 0.844796] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1032worker2 # [ 0.849364] Using buffer write method1033worker2 # [ 0.849506] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1034worker2 # [ 0.851583] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001035worker2 # [ 0.851643] Intel/Sharp Extended Query Table at 0x00311036lb # [ 0.856823] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1037worker2 # [ 0.865445] Using buffer write method1038worker2 # [ 0.865514] Concatenating MTD devices:1039worker2 # [ 0.865518] (0): "0.flash"1040worker2 # [ 0.865522] (1): "0.flash"1041worker2 # [ 0.865525] into device "0.flash"1042lb # [ 0.867407] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1043lb # [ 0.870035] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1044lb # [ 0.872713] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1045lb # [ 0.892257] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1046lb # [ 0.894556] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1047lb # [ 0.900095] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1048lb # [ 0.920562] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1049lb # [ 0.924313] msm_serial: driver initialized1050lb # [ 0.924471] SuperH (H)SCI(F) driver initialized1051lb # [ 0.924528] STM32 USART driver initialized1052lb # [ 0.974244] loop: module loaded1053lb # [ 0.974497] virtio_blk virtio5: 1/0/0 default/read/poll queues1054lb # [ 0.980739] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1055lb # [ 0.984570] megasas: 07.734.00.00-rc11056lb # [ 0.985427] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1057lb # [ 1.013028] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001058lb # [ 1.013094] Intel/Sharp Extended Query Table at 0x00311059lb # [ 1.025736] Using buffer write method1060lb # [ 1.025889] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1061lb # [ 1.032910] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001062lb # [ 1.032967] Intel/Sharp Extended Query Table at 0x00311063lb # [ 1.044386] Using buffer write method1064lb # [ 1.044444] Concatenating MTD devices:1065lb # [ 1.044448] (0): "0.flash"1066lb # [ 1.044454] (1): "0.flash"1067lb # [ 1.044458] into device "0.flash"1068client # [ 1.306807] Freeing initrd memory: 26160K1069client # [ 1.313758] tun: Universal TUN/TAP device driver, 1.61070client # [ 1.318471] thunder_xcv, ver 1.01071client # [ 1.318538] thunder_bgx, ver 1.01072client # [ 1.318562] nicpf, ver 1.01073client # [ 1.319246] e1000: Intel(R) PRO/1000 Network Driver1074client # [ 1.319257] e1000: Copyright (c) 1999-2006 Intel Corporation.1075client # [ 1.319282] e1000e: Intel(R) PRO/1000 Network Driver1076client # [ 1.326452] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1077client # [ 1.326576] igb: Intel(R) Gigabit Ethernet Network Driver1078client # [ 1.326582] igb: Copyright (c) 2007-2014 Intel Corporation.1079client # [ 1.326611] igbvf: Intel(R) Gigabit Virtual Function Network Driver1080client # [ 1.326618] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1081client # [ 1.326919] sky2: driver version 1.301082client # [ 1.333251] usbcore: registered new interface driver usb-storage1083client # [ 1.333420] usbcore: registered new interface driver usbserial_generic1084client # [ 1.333437] usbserial: USB Serial support registered for generic1085client # [ 1.334099] hv_vmbus: registering driver hyperv_keyboard1086client # [ 1.335764] ehci-pci 0000:00:0a.0: EHCI Host Controller1087client # [ 1.335799] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11088client # [ 1.336119] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001089client # [ 1.342080] rtc-pl031 9010000.pl031: registered as rtc01090client # [ 1.342136] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:20:46 UTC (1789906846)1091client # [ 1.342693] i2c_dev: i2c /dev entries driver1092client # [ 1.348461] sdhci: Secure Digital Host Controller Interface driver1093client # [ 1.348479] sdhci: Copyright(c) Pierre Ossman1094client # [ 1.348760] Synopsys Designware Multimedia Card Interface Driver1095client # [ 1.349162] sdhci-pltfm: SDHCI platform and OF driver helper1096client # [ 1.351192] hid: raw HID events driver (C) Jiri Kosina1097client # [ 1.351491] usbcore: registered new interface driver usbhid1098client # [ 1.351512] usbhid: USB HID core driver1099client # [ 1.357061] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001100client # [ 1.358272] hub 1-0:1.0: USB hub found1101client # [ 1.358849] hub 1-0:1.0: 6 ports detected1102client # [ 1.360691] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1103client # [ 1.362367] drop_monitor: Initializing network drop monitor service1104client # [ 1.362611] NET: Registered PF_INET6 protocol family1105client # [ 1.366132] Segment Routing with IPv61106client # [ 1.366177] In-situ OAM (IOAM) with IPv61107client # [ 1.366216] NET: Registered PF_PACKET protocol family1108client # [ 1.367904] 9pnet: Installing 9P2000 support1109client # [ 1.371241] Key type dns_resolver registered1110client # [ 1.378231] registered taskstats version 11111client # [ 1.378432] Loading compiled-in X.509 certificates1112client # [ 1.387905] Demotion targets for Node 0: null1113client # [ 1.388068] Key type .fscrypt registered1114client # [ 1.388076] Key type fscrypt-provisioning registered1115client # [ 1.389881] ima: No TPM chip found, activating TPM-bypass!1116client # [ 1.389910] ima: Allocated hash algorithm: sha11117client # [ 1.389936] ima: No architecture policies found1118client # [ 1.392630] input: gpio-keys as /devices/platform/gpio-keys/input/input01119client # [ 1.418392] clk: Disabling unused clocks1120client # [ 1.418442] PM: genpd: Disabling unused power domains1121client # [ 1.422881] Freeing unused kernel memory: 4736K1122client # [ 1.423142] Run /init as init process1123client # [ 1.443577] systemd[1]: Successfully made /usr/ read-only.1124worker2 # [ 1.267372] Freeing initrd memory: 26148K1125worker2 # [ 1.275301] tun: Universal TUN/TAP device driver, 1.61126worker2 # [ 1.279992] thunder_xcv, ver 1.01127worker2 # [ 1.280059] thunder_bgx, ver 1.01128worker2 # [ 1.280083] nicpf, ver 1.01129worker2 # [ 1.281912] e1000: Intel(R) PRO/1000 Network Driver1130worker2 # [ 1.281930] e1000: Copyright (c) 1999-2006 Intel Corporation.1131worker2 # [ 1.281959] e1000e: Intel(R) PRO/1000 Network Driver1132worker2 # [ 1.281970] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1133worker2 # [ 1.282001] igb: Intel(R) Gigabit Ethernet Network Driver1134worker2 # [ 1.282030] igb: Copyright (c) 2007-2014 Intel Corporation.1135worker2 # [ 1.282057] igbvf: Intel(R) Gigabit Virtual Function Network Driver1136worker2 # [ 1.282063] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1137worker2 # [ 1.282206] sky2: driver version 1.301138worker2 # [ 1.283892] usbcore: registered new interface driver usb-storage1139worker2 # [ 1.284056] usbcore: registered new interface driver usbserial_generic1140worker2 # [ 1.284073] usbserial: USB Serial support registered for generic1141worker2 # [ 1.284951] ehci-pci 0000:00:0a.0: EHCI Host Controller1142worker2 # [ 1.285008] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11143worker2 # [ 1.285186] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001144worker2 # [ 1.296515] hv_vmbus: registering driver hyperv_keyboard1145worker2 # [ 1.297997] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001146worker2 # [ 1.299166] hub 1-0:1.0: USB hub found1147worker2 # [ 1.299657] hub 1-0:1.0: 6 ports detected1148worker2 # [ 1.300846] rtc-pl031 9010000.pl031: registered as rtc01149worker2 # [ 1.300879] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:20:46 UTC (1789906846)1150worker2 # [ 1.301269] i2c_dev: i2c /dev entries driver1151worker2 # [ 1.306489] sdhci: Secure Digital Host Controller Interface driver1152worker2 # [ 1.306506] sdhci: Copyright(c) Pierre Ossman1153worker2 # [ 1.306779] Synopsys Designware Multimedia Card Interface Driver1154worker2 # [ 1.307152] sdhci-pltfm: SDHCI platform and OF driver helper1155worker2 # [ 1.311685] hid: raw HID events driver (C) Jiri Kosina1156worker2 # [ 1.311958] usbcore: registered new interface driver usbhid1157worker2 # [ 1.311966] usbhid: USB HID core driver1158worker2 # [ 1.314934] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1159worker2 # [ 1.317575] drop_monitor: Initializing network drop monitor service1160worker2 # [ 1.317805] NET: Registered PF_INET6 protocol family1161worker2 # [ 1.319792] Segment Routing with IPv61162worker2 # [ 1.319815] In-situ OAM (IOAM) with IPv61163worker2 # [ 1.319849] NET: Registered PF_PACKET protocol family1164worker2 # [ 1.321514] 9pnet: Installing 9P2000 support1165worker2 # [ 1.324010] Key type dns_resolver registered1166worker2 # [ 1.331773] registered taskstats version 11167worker2 # [ 1.331990] Loading compiled-in X.509 certificates1168lb # [ 1.332438] Freeing initrd memory: 26140K1169lb # [ 1.339847] tun: Universal TUN/TAP device driver, 1.61170worker2 # [ 1.341063] Demotion targets for Node 0: null1171worker2 # [ 1.341216] Key type .fscrypt registered1172worker2 # [ 1.341223] Key type fscrypt-provisioning registered1173worker2 # [ 1.341337] ima: No TPM chip found, activating TPM-bypass!1174worker2 # [ 1.341359] ima: Allocated hash algorithm: sha11175worker2 # [ 1.341387] ima: No architecture policies found1176worker2 # [ 1.345659] input: gpio-keys as /devices/platform/gpio-keys/input/input01177lb # [ 1.344633] thunder_xcv, ver 1.01178lb # [ 1.344691] thunder_bgx, ver 1.01179lb # [ 1.344717] nicpf, ver 1.01180lb # [ 1.345301] e1000: Intel(R) PRO/1000 Network Driver1181lb # [ 1.345310] e1000: Copyright (c) 1999-2006 Intel Corporation.1182lb # [ 1.345334] e1000e: Intel(R) PRO/1000 Network Driver1183lb # [ 1.345343] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1184lb # [ 1.345371] igb: Intel(R) Gigabit Ethernet Network Driver1185lb # [ 1.345376] igb: Copyright (c) 2007-2014 Intel Corporation.1186lb # [ 1.345400] igbvf: Intel(R) Gigabit Virtual Function Network Driver1187lb # [ 1.345406] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1188lb # [ 1.345550] sky2: driver version 1.301189lb # [ 1.348236] ehci-pci 0000:00:0a.0: EHCI Host Controller1190lb # [ 1.348316] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11191lb # [ 1.348554] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001192lb # [ 1.358656] usbcore: registered new interface driver usb-storage1193lb # [ 1.358731] usbcore: registered new interface driver usbserial_generic1194lb # [ 1.358746] usbserial: USB Serial support registered for generic1195lb # [ 1.359156] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001196lb # [ 1.359751] hub 1-0:1.0: USB hub found1197lb # [ 1.359776] hub 1-0:1.0: 6 ports detected1198lb # [ 1.364561] hv_vmbus: registering driver hyperv_keyboard1199lb # [ 1.366258] rtc-pl031 9010000.pl031: registered as rtc01200lb # [ 1.366294] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:20:46 UTC (1789906846)1201lb # [ 1.366634] i2c_dev: i2c /dev entries driver1202worker2 # [ 1.369034] clk: Disabling unused clocks1203worker2 # [ 1.369081] PM: genpd: Disabling unused power domains1204worker2 # [ 1.373607] Freeing unused kernel memory: 4736K1205lb # [ 1.372081] sdhci: Secure Digital Host Controller Interface driver1206worker2 # [ 1.373820] Run /init as init process1207lb # [ 1.372103] sdhci: Copyright(c) Pierre Ossman1208lb # [ 1.372381] Synopsys Designware Multimedia Card Interface Driver1209lb # [ 1.372764] sdhci-pltfm: SDHCI platform and OF driver helper1210lb # [ 1.374946] hid: raw HID events driver (C) Jiri Kosina1211lb # [ 1.378783] usbcore: registered new interface driver usbhid1212lb # [ 1.378798] usbhid: USB HID core driver1213lb # [ 1.381374] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1214lb # [ 1.382957] drop_monitor: Initializing network drop monitor service1215lb # [ 1.385518] NET: Registered PF_INET6 protocol family1216lb # [ 1.386787] Segment Routing with IPv61217lb # [ 1.386807] In-situ OAM (IOAM) with IPv61218lb # [ 1.386839] NET: Registered PF_PACKET protocol family1219lb # [ 1.388706] 9pnet: Installing 9P2000 support1220lb # [ 1.391087] Key type dns_resolver registered1221worker2 # [ 1.391236] systemd[1]: Successfully made /usr/ read-only.1222lb # [ 1.398174] registered taskstats version 11223lb # [ 1.398368] Loading compiled-in X.509 certificates1224lb # [ 1.409375] Demotion targets for Node 0: null1225lb # [ 1.409535] Key type .fscrypt registered1226lb # [ 1.409542] Key type fscrypt-provisioning registered1227lb # [ 1.409658] ima: No TPM chip found, activating TPM-bypass!1228lb # [ 1.409678] ima: Allocated hash algorithm: sha11229lb # [ 1.409703] ima: No architecture policies found1230lb # [ 1.415146] input: gpio-keys as /devices/platform/gpio-keys/input/input01231client # [ 1.604301] usb 1-1: new high-speed USB device number 2 using ehci-pci1232lb # [ 1.439705] clk: Disabling unused clocks1233lb # [ 1.439742] PM: genpd: Disabling unused power domains1234lb # [ 1.445641] Freeing unused kernel memory: 4736K1235lb # [ 1.445872] Run /init as init process1236lb # [ 1.471066] systemd[1]: Successfully made /usr/ read-only.1237worker2 # [ 1.548238] usb 1-1: new high-speed USB device number 2 using ehci-pci1238client # [ 1.780389] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1239client # [ 1.784351] systemd[1]: Detected virtualization qemu.1240client # [ 1.784976] systemd[1]: Detected architecture arm64.1241client # [ 1.785552] systemd[1]: Running in initrd.1242client # [ 1.786475] systemd[1]: Initializing machine ID from random generator.1243client # [ 1.787557] systemd[1]: Hostname set to <client>.1244lb # [ 1.611255] usb 1-1: new high-speed USB device number 2 using ehci-pci1245client # [ 1.808695] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11246worker2 # [ 1.697958] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11247client # [ 1.888635] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01248worker2 # [ 1.726499] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1249worker2 # [ 1.730616] systemd[1]: Detected virtualization qemu.1250worker2 # [ 1.731267] systemd[1]: Detected architecture arm64.1251worker2 # [ 1.731859] systemd[1]: Running in initrd.1252worker2 # [ 1.732843] systemd[1]: Initializing machine ID from random generator.1253worker2 # [ 1.737352] systemd[1]: Hostname set to <worker2>.1254lb # [ 1.762358] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11255worker2 # [ 1.793951] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01256lb # [ 1.819592] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1257lb # [ 1.828539] systemd[1]: Detected virtualization qemu.1258lb # [ 1.828624] systemd[1]: Detected architecture arm64.1259lb # [ 1.828633] systemd[1]: Running in initrd.1260lb # [ 1.829107] systemd[1]: Initializing machine ID from random generator.1261lb # [ 1.829224] systemd[1]: Hostname set to <lb>.1262lb # [ 1.839644] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01263client # [ 2.032326] usb 1-2: new high-speed USB device number 3 using ehci-pci1264client # [ 2.070930] systemd[1]: bpf-restrict-fs: LSM BPF program attached1265worker2 # [ 1.912296] usb 1-2: new high-speed USB device number 3 using ehci-pci1266lb # [ 1.971198] usb 1-2: new high-speed USB device number 3 using ehci-pci1267client # [ 2.183867] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21268client # [ 2.188323] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01269worker2 # [ 2.003371] systemd[1]: bpf-restrict-fs: LSM BPF program attached1270lb # [ 2.093336] systemd[1]: bpf-restrict-fs: LSM BPF program attached1271client # [ 2.275941] systemd[1]: Queued start job for default target Initrd Default Target.1272client # [ 2.301170] systemd[1]: Created slice Slice /system/modprobe.1273client # [ 2.302253] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1274client # [ 2.303287] systemd[1]: Expecting device /dev/disk/by-label/nixos...1275client # [ 2.307996] systemd[1]: Reached target Path Units.1276client # [ 2.308731] systemd[1]: Reached target Slice Units.1277client # [ 2.308758] systemd[1]: Reached target Swaps.1278client # [ 2.308781] systemd[1]: Reached target Timer Units.1279client # [ 2.309147] systemd[1]: Listening on D-Bus System Message Bus Socket.1280client # [ 2.309300] systemd[1]: Listening on Journal Socket (/dev/log).1281client # [ 2.309421] systemd[1]: Listening on Journal Sockets.1282client # [ 2.309542] systemd[1]: Listening on udev Control Socket.1283client # [ 2.309617] systemd[1]: Listening on udev Kernel Socket.1284client # [ 2.309636] systemd[1]: Reached target Socket Units.1285client # [ 2.323976] systemd[1]: Starting Create List of Static Device Nodes...1286lb # [ 2.137187] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21287lb # [ 2.140351] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01288client # [ 2.336439] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1289client # [ 2.337407] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1290worker2 # [ 2.159926] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21291worker2 # [ 2.161838] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01292client # [ 2.352464] systemd[1]: Mounting Kernel Configuration File System...1293worker2 # [ 2.175963] systemd[1]: Queued start job for default target Initrd Default Target.1294client # [ 2.370399] systemd[1]: Starting Journal Service...1295worker2 # [ 2.182603] systemd[1]: Created slice Slice /system/modprobe.1296worker2 # [ 2.183640] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1297worker2 # [ 2.184678] systemd[1]: Expecting device /dev/disk/by-label/nixos...1298worker2 # [ 2.185444] systemd[1]: Reached target Path Units.1299worker2 # [ 2.186090] systemd[1]: Reached target Slice Units.1300worker2 # [ 2.186687] systemd[1]: Reached target Swaps.1301worker2 # [ 2.187222] systemd[1]: Reached target Timer Units.1302worker2 # [ 2.188027] systemd[1]: Listening on D-Bus System Message Bus Socket.1303worker2 # [ 2.189132] systemd[1]: Listening on Journal Socket (/dev/log).1304worker2 # [ 2.189998] systemd[1]: Listening on Journal Sockets.1305worker2 # [ 2.190836] systemd[1]: Listening on udev Control Socket.1306worker2 # [ 2.191726] systemd[1]: Listening on udev Kernel Socket.1307worker2 # [ 2.192513] systemd[1]: Reached target Socket Units.1308worker2 # [ 2.195099] systemd[1]: Starting Create List of Static Device Nodes...1309worker2 # [ 2.202269] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1310worker2 # [ 2.203259] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1311lb # [ 2.213641] systemd[1]: Queued start job for default target Initrd Default Target.1312worker2 # [ 2.213208] systemd[1]: Mounting Kernel Configuration File System...1313lb # [ 2.221310] systemd[1]: Created slice Slice /system/modprobe.1314lb # [ 2.228908] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1315lb # [ 2.230175] systemd[1]: Expecting device /dev/disk/by-label/nixos...1316lb # [ 2.231189] systemd[1]: Reached target Path Units.1317lb # [ 2.231945] systemd[1]: Reached target Slice Units.1318lb # [ 2.232657] systemd[1]: Reached target Swaps.1319lb # [ 2.233688] systemd[1]: Reached target Timer Units.1320lb # [ 2.234793] systemd[1]: Listening on D-Bus System Message Bus Socket.1321lb # [ 2.235910] systemd[1]: Listening on Journal Socket (/dev/log).1322lb # [ 2.236810] systemd[1]: Listening on Journal Sockets.1323lb # [ 2.237607] systemd[1]: Listening on udev Control Socket.1324lb # [ 2.238412] systemd[1]: Listening on udev Kernel Socket.1325lb # [ 2.239131] systemd[1]: Reached target Socket Units.1326lb # [ 2.241577] systemd[1]: Starting Create List of Static Device Nodes...1327client # [ 2.427554] systemd[1]: Starting Load Kernel Modules...1328client # [ 2.427769] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1329worker2 # [ 2.248490] systemd[1]: Starting Journal Service...1330lb # [ 2.255443] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1331client # [ 2.441007] systemd[1]: Starting Coldplug All udev Devices...1332worker2 # [ 2.256482] systemd[1]: Starting Load Kernel Modules...1333worker2 # [ 2.257440] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1334lb # [ 2.256426] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1335lb # [ 2.271883] systemd[1]: Mounting Kernel Configuration File System...1336client # [ 2.467253] systemd-journald[73]: Collecting audit messages is disabled.1337worker2 # [ 2.272909] systemd[1]: Starting Coldplug All udev Devices...1338client # [ 2.476706] systemd[1]: Finished Create List of Static Device Nodes.1339client # [ 2.478885] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1340lb # [ 2.299492] systemd[1]: Starting Journal Service...1341worker2 # [ 2.296449] systemd[1]: Finished Create List of Static Device Nodes.1342worker2 # [ 2.297789] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1343worker2 # [ 2.304352] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1344worker2 # [ 2.305610] systemd[1]: Mounted Kernel Configuration File System.1345lb # [ 2.307449] systemd[1]: Starting Load Kernel Modules...1346lb # [ 2.308280] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1347client # [ 2.492760] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1348worker2 # [ 2.320870] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1349client # [ 2.514284] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1350client # [ 2.514910] systemd[1]: Mounted Kernel Configuration File System.1351worker2 # [ 2.331920] systemd-journald[73]: Collecting audit messages is disabled.1352lb # [ 2.331756] systemd[1]: Starting Coldplug All udev Devices...1353lb # [ 2.334602] systemd[1]: Finished Create List of Static Device Nodes.1354lb # [ 2.341658] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1355worker2 # [ 2.343083] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1356lb # [ 2.355842] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1357worker2 # [ 2.364238] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1358lb # [ 2.369229] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1359lb # [ 2.379756] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1360lb # [ 2.381131] systemd[1]: Mounted Kernel Configuration File System.1361lb # [ 2.387391] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1362lb # [ 2.398494] systemd-journald[73]: Collecting audit messages is disabled.1363lb # [ 2.404213] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01364lb # [ 2.404477] [drm] features: -virgl +edid -resource_blob -host_visible1365lb # [ 2.404491] [drm] features: -context_init1366lb # [ 2.405242] [drm] number of scanouts: 11367lb # [ 2.405264] [drm] number of cap sets: 01368client # [ 2.589034] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1369client # [ 2.604072] systemd[1]: Starting Create Static Device Nodes in /dev...1370lb # [ 2.423680] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1371lb # [ 2.423720] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01372lb # [ 2.447341] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1373client # [ 2.637814] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1374lb # [ 2.455819] systemd[1]: Starting Create Static Device Nodes in /dev...1375client # [ 2.647474] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1376lb # [ 2.469923] Console: switching to colour frame buffer device 160x501377lb # [ 2.472095] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1378lb # [ 2.488071] systemd[1]: Finished Load Kernel Modules.1379lb # [ 2.512638] systemd[1]: Starting Apply Kernel Variables...1380client # [ 2.713117] systemd[1]: Finished Create Static Device Nodes in /dev.1381client # [ 2.713387] systemd[1]: Reached target Preparation for Local File Systems.1382client # [ 2.713415] systemd[1]: Reached target Local File Systems.1383client # [ 2.722192] systemd[1]: Starting Rule-based Manager for Device Events and Files...1384client # [ 2.724463] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01385client # [ 2.724729] [drm] features: -virgl +edid -resource_blob -host_visible1386client # [ 2.724741] [drm] features: -context_init1387client # [ 2.725537] [drm] number of scanouts: 11388worker2 # [ 2.528448] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1389worker2 # [ 2.536724] systemd[1]: Starting Create Static Device Nodes in /dev...1390client # [ 2.725558] [drm] number of cap sets: 01391worker2 # [ 2.501815] systemd-modules-load[74]: Inserted module 'dm_mod'1392client # [ 2.736896] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1393lb # [ 2.560001] systemd[1]: Finished Create Static Device Nodes in /dev.1394lb # [ 2.560363] systemd[1]: Reached target Preparation for Local File Systems.1395lb # [ 2.560413] systemd[1]: Reached target Local File Systems.1396client # [ 2.736933] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01397client # [ 2.765470] systemd[1]: Started Journal Service.1398lb # [ 2.577798] systemd[1]: Starting Rule-based Manager for Device Events and Files...1399worker2 # [ 2.527509] systemd-modules-load[74]: Module 'virtio_balloon' is built in1400worker2 # [ 2.532448] systemd-modules-load[74]: Module 'virtio_console' is built in1401client # [ 2.755923] systemd-modules-load[74]: Inserted module 'dm_mod'1402client # [ 2.765636] systemd-modules-load[74]: Module 'virtio_balloon' is built in1403lb # [ 2.644461] systemd[1]: Finished Apply Kernel Variables.1404lb # [ 2.655514] systemd[1]: Started Journal Service.1405lb # [ 2.652588] systemd-modules-load[74]: Inserted module 'dm_mod'1406client # [ 2.841586] systemd-modules-load[74]: Module 'virtio_console' is built in1407lb # [ 2.660486] systemd-modules-load[74]: Module 'virtio_balloon' is built in1408lb # [ 2.662438] systemd-modules-load[74]: Module 'virtio_console' is built in1409lb # [ 2.673040] systemd-modules-load[74]: Inserted module 'virtio_gpu'1410lb # [ 2.678464] systemd-modules-load[74]: Module 'virtio_rng' is built in1411lb # [ 2.679505] systemd[1]: Starting Create System Files and Directories...1412worker2 # [ 2.676339] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01413worker2 # [ 2.676608] [drm] features: -virgl +edid -resource_blob -host_visible1414worker2 # [ 2.676631] [drm] features: -context_init1415worker2 # [ 2.677424] [drm] number of scanouts: 11416worker2 # [ 2.677446] [drm] number of cap sets: 01417worker2 # [ 2.680574] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1418worker2 # [ 2.680598] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01419worker2 # [ 2.694397] systemd[1]: Started Journal Service.1420client # [ 2.859282] systemd[1]: Starting Create System Files and Directories...1421worker2 # [ 2.698235] systemd-modules-load[74]: Inserted module 'virtio_gpu'1422worker2 # [ 2.701172] systemd-modules-load[74]: Module 'virtio_rng' is built in1423worker2 # [ 2.740598] Console: switching to colour frame buffer device 160x501424worker2 # [ 2.741266] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1425worker2 # [ 2.712870] systemd[1]: Finished Load Kernel Modules.1426worker2 # [ 2.730276] systemd[1]: Starting Apply Kernel Variables...1427client # [ 3.005298] systemd[1]: Finished Create System Files and Directories.1428worker2 # [ 2.786001] systemd[1]: Finished Apply Kernel Variables.1429client # [ 3.033708] systemd-udevd[79]: Using default interface naming scheme 'v261'.1430lb # [ 2.853291] systemd-udevd[81]: Using default interface naming scheme 'v261'.1431lb # [ 2.857711] systemd[1]: Finished Create System Files and Directories.1432client # [ 3.067587] systemd[1]: Started Rule-based Manager for Device Events and Files.1433worker2 # [ 2.841815] systemd[1]: Finished Create Static Device Nodes in /dev.1434worker2 # [ 2.843425] systemd[1]: Reached target Preparation for Local File Systems.1435lb # [ 2.894525] systemd[1]: Started Rule-based Manager for Device Events and Files.1436worker2 # [ 2.850629] systemd[1]: Reached target Local File Systems.1437worker2 # [ 2.851581] systemd[1]: Starting Create System Files and Directories...1438client # [ 3.094489] Console: switching to colour frame buffer device 160x501439client # [ 3.101032] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1440client # [ 3.088951] systemd-modules-load[74]: Inserted module 'virtio_gpu'1441client # [ 3.090131] systemd-modules-load[74]: Module 'virtio_rng' is built in1442client # [ 3.095041] systemd[1]: Finished Load Kernel Modules.1443worker2 # [ 2.874197] systemd[1]: Starting Rule-based Manager for Device Events and Files...1444client # [ 3.107362] systemd[1]: Starting Apply Kernel Variables...1445client # [ 3.148256] systemd[1]: Finished Apply Kernel Variables.1446worker2 # [ 2.939365] systemd[1]: Finished Create System Files and Directories.1447client # [ 3.183090] systemd[1]: Starting Virtual Console Setup...1448worker2 # [ 2.998370] systemd-udevd[88]: Using default interface naming scheme 'v261'.1449lb # [ 3.066548] systemd[1]: Starting Virtual Console Setup...1450worker2 # [ 3.033395] systemd[1]: Started Rule-based Manager for Device Events and Files.1451client # [ 3.297733] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones.1452client # [ 3.303301] systemd[1]: Finished Virtual Console Setup.1453lb # [ 3.133359] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1454lb # [ 3.157631] systemd[1]: Finished Virtual Console Setup.1455worker2 # [ 3.158100] systemd[1]: Starting Virtual Console Setup...1456worker2 # [ 3.224682] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1457worker2 # [ 3.229259] systemd[1]: Finished Virtual Console Setup.1458client # [ 4.039946] systemd[1]: Finished Coldplug All udev Devices.1459client # [ 4.040947] systemd[1]: Reached target System Initialization.1460client # [ 4.041728] systemd[1]: Reached target Basic System.1461lb # [ 3.999883] systemd[1]: Finished Coldplug All udev Devices.1462lb # [ 4.001712] systemd[1]: Reached target System Initialization.1463lb # [ 4.002575] systemd[1]: Reached target Basic System.1464lb # [ 4.155658] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1465worker2 # [ 4.137268] systemd[1]: Finished Coldplug All udev Devices.1466worker2 # [ 4.138340] systemd[1]: Reached target System Initialization.1467worker2 # [ 4.139104] systemd[1]: Reached target Basic System.1468lb # [ 4.190319] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1469lb # [ 4.195602] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1470lb # [ 4.302084] systemd[1]: Found device /dev/disk/by-label/nixos.1471lb # [ 4.306705] systemd[1]: Reached target Initrd Root Device.1472lb # [ 4.314349] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1473worker2 # [ 4.303981] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1474client # [ 4.573537] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1475worker2 # [ 4.326292] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1476client # [ 4.575758] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1477worker2 # [ 4.329727] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1478lb # [ 4.381499] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1479lb # [ 4.404219] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1480lb # [ 4.408448] systemd[1]: Mounting /sysroot...1481worker2 # [ 4.372491] systemd[1]: Found device /dev/disk/by-label/nixos.1482worker2 # [ 4.374614] systemd[1]: Reached target Initrd Root Device.1483worker2 # [ 4.380201] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1484client # [ 4.634775] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1485lb # [ 4.496748] EXT4-fs (vda): mounted filesystem 369acefe-5390-4564-8a4a-d54daae702fe r/w with ordered data mode. Quota mode: none.1486worker2 # [ 4.439152] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1487lb # [ 4.494907] systemd[1]: Mounted /sysroot.1488lb # [ 4.501795] systemd[1]: Reached target Initrd Root File System.1489lb # [ 4.511715] systemd[1]: Starting Mountpoints Configured in the Real Root...1490worker2 # [ 4.468517] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1491worker2 # [ 4.478640] systemd[1]: Mounting /sysroot...1492client # [ 4.727738] systemd[1]: Found device /dev/disk/by-label/nixos.1493client # [ 4.729973] systemd[1]: Reached target Initrd Root Device.1494lb # [ 4.545705] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1495lb # [ 4.550866] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1496lb # [ 4.558441] systemd[1]: Reloading...1497client # [ 4.740298] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1498worker2 # [ 4.590348] EXT4-fs (vda): mounted filesystem 443afb83-a879-4de8-ae82-60a3e9d83ae9 r/w with ordered data mode. Quota mode: none.1499worker2 # [ 4.545953] systemd[1]: Mounted /sysroot.1500worker2 # [ 4.553375] systemd[1]: Reached target Initrd Root File System.1501worker2 # [ 4.566051] systemd[1]: Starting Mountpoints Configured in the Real Root...1502client # [ 4.814252] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1503client # [ 4.835820] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1504client # [ 4.840274] systemd[1]: Mounting /sysroot...1505worker2 # [ 4.609716] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.1506worker2 # [ 4.618679] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)...1507worker2 # [ 4.623525] systemd[1]: Reloading...1508client # [ 4.899904] EXT4-fs (vda): mounted filesystem 176a664d-aa8a-4a92-9a1f-ee75b00a7b59 r/w with ordered data mode. Quota mode: none.1509client # [ 4.893447] systemd[1]: Mounted /sysroot.1510client # [ 4.895137] systemd[1]: Reached target Initrd Root File System.1511client # [ 4.907862] systemd[1]: Mounting /sysroot/nix/.ro-store...1512client # [ 4.918120] systemd[1]: Mounting /sysroot/nix/.rw-store...1513client # [ 4.942239] systemd[1]: Mounting /sysroot/run...1514client # [ 4.965676] systemd[1]: Mounting /sysroot/tmp/shared...1515lb # [ 4.815020] systemd[1]: Reloading finished in 259 ms.1516client # [ 4.998499] systemd[1]: Mounting /sysroot/tmp/xchg...1517client # [ 5.020798] systemd[1]: Starting Mountpoints Configured in the Real Root...1518lb # [ 4.847798] systemd-sysroot-fstab-check[120]: Requesting initrd-fs.target/start/replace...1519lb # [ 4.853112] systemd-sysroot-fstab-check[120]: Requesting swap.target/start/replace...1520client # [ 5.039055] systemd[1]: Mounted /sysroot/nix/.ro-store.1521lb # [ 4.860446] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1522lb # [ 4.866871] systemd[1]: initrd-parse-etc.service: Deactivated successfully.1523lb # [ 4.873137] systemd[1]: Finished Mountpoints Configured in the Real Root.1524client # [ 5.057012] systemd[1]: Mounted /sysroot/nix/.rw-store.1525lb # [ 4.875627] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.1526lb # [ 4.904344] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1527lb # [ 4.907175] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1528client # [ 5.103974] systemd-sysroot-fstab-check[125]: /sysroot should be mounted in the initrd, will request daemon-reload.1529client # [ 5.116181] systemd[1]: Starting rw-sysroot-nix-store.service...1530client # [ 5.122530] systemd[1]: Mounted /sysroot/run.1531client # [ 5.129729] systemd[1]: Mounted /sysroot/tmp/shared.1532client # [ 5.134696] systemd[1]: Mounted /sysroot/tmp/xchg.1533client # [ 5.160239] systemd[1]: Reload requested from client PID 125 ('systemd-sysroot') (unit initrd-parse-etc.service)...1534client # [ 5.161863] systemd[1]: Reloading...1535worker2 # [ 4.927979] systemd[1]: Reloading finished in 309 ms.1536worker2 # [ 4.960158] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace...1537worker2 # [ 4.973922] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace...1538worker2 # [ 4.989050] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1539worker2 # [ 5.014282] systemd[1]: initrd-parse-etc.service: Deactivated successfully.1540worker2 # [ 5.023636] systemd[1]: Finished Mountpoints Configured in the Real Root.1541worker2 # [ 5.036266] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.1542worker2 # [ 5.040214] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1543worker2 # [ 5.044354] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1544client # [ 5.435830] systemd[1]: Reloading finished in 276 ms.1545client # [ 5.470221] systemd-sysroot-fstab-check[125]: Requesting initrd-fs.target/start/replace...1546client # [ 5.474873] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1547client # [ 5.480395] systemd[1]: Finished rw-sysroot-nix-store.service.1548client # [ 5.483305] systemd-sysroot-fstab-check[125]: Requesting swap.target/start/replace...1549client # [ 5.495657] systemd[1]: Starting rw-sysroot-nix-store.service...1550client # [ 5.519880] systemd[1]: initrd-parse-etc.service: Deactivated successfully.1551client # [ 5.523690] systemd[1]: Finished Mountpoints Configured in the Real Root.1552client # [ 5.526554] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.1553lb # [ 5.361973] systemd[1]: Mounting /sysroot/nix/.ro-store...1554client # [ 5.551348] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1555client # [ 5.556621] systemd[1]: Finished rw-sysroot-nix-store.service.1556lb # [ 5.381932] systemd[1]: Mounting /sysroot/nix/.rw-store...1557lb # [ 5.422225] systemd[1]: Mounting /sysroot/run...1558lb # [ 5.441863] systemd[1]: Mounting /sysroot/tmp/shared...1559worker2 # [ 5.410955] systemd[1]: Mounting /sysroot/nix/.ro-store...1560lb # [ 5.462897] systemd[1]: Mounting /sysroot/tmp/xchg...1561worker2 # [ 5.423175] systemd[1]: Mounting /sysroot/nix/.rw-store...1562worker2 # [ 5.442998] systemd[1]: Mounting /sysroot/run...1563worker2 # [ 5.460934] systemd[1]: Mounting /sysroot/tmp/shared...1564lb # [ 5.530127] systemd[1]: Mounted /sysroot/nix/.ro-store.1565lb # [ 5.551771] systemd[1]: Mounted /sysroot/nix/.rw-store.1566worker2 # [ 5.508872] systemd[1]: Mounting /sysroot/tmp/xchg...1567client # [ 5.775882] systemd[1]: Mounting /sysroot/nix/store...1568lb # [ 5.600627] systemd[1]: Mounted /sysroot/run.1569lb # [ 5.619029] systemd[1]: Starting rw-sysroot-nix-store.service...1570lb # [ 5.623328] systemd[1]: Mounted /sysroot/tmp/shared.1571worker2 # [ 5.578830] systemd[1]: Mounted /sysroot/nix/.ro-store.1572lb # [ 5.634970] systemd[1]: Mounted /sysroot/tmp/xchg.1573client # [ 5.826005] systemd[1]: Mounted /sysroot/nix/store.1574client # [ 5.829695] systemd[1]: Reached target Initrd File Systems.1575worker2 # [ 5.603579] systemd[1]: Mounted /sysroot/run.1576worker2 # [ 5.620997] systemd[1]: Mounted /sysroot/nix/.rw-store.1577lb # [ 5.667692] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1578lb # [ 5.670518] systemd[1]: Finished rw-sysroot-nix-store.service.1579worker2 # [ 5.630559] systemd[1]: Mounted /sysroot/tmp/shared.1580client # [ 5.863162] systemd[1]: Starting Find NixOS closure...1581client # [ 5.874014] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...1582worker2 # [ 5.650963] systemd[1]: Mounted /sysroot/tmp/xchg.1583worker2 # [ 5.664230] systemd[1]: Starting rw-sysroot-nix-store.service...1584client # [ 5.932252] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.1585client # [ 5.936967] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.1586client # [ 5.946208] systemd[1]: Finished Find NixOS closure.1587worker2 # [ 5.715575] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1588client # [ 5.950235] systemd[1]: Reached target Initrd Default Target.1589client # [ 5.953955] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...1590worker2 # [ 5.723664] systemd[1]: Finished rw-sysroot-nix-store.service.1591client # [ 6.010020] systemd[1]: Stopped target Initrd Default Target.1592client # [ 6.016500] systemd[1]: Stopped target Basic System.1593client # [ 6.027073] systemd[1]: Stopped target Initrd Root Device.1594client # [ 6.029930] systemd[1]: Stopped target Path Units.1595client # [ 6.033016] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.1596client # [ 6.036678] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.1597client # [ 6.040215] systemd[1]: Stopped target Slice Units.1598client # [ 6.045204] systemd[1]: Stopped target Socket Units.1599client # [ 6.047483] systemd[1]: Stopped target System Initialization.1600client # [ 6.056312] systemd[1]: Stopped target Swaps.1601client # [ 6.057882] systemd[1]: Stopped target Timer Units.1602client # [ 6.059065] systemd[1]: dbus.socket: Deactivated successfully.1603client # [ 6.068287] systemd[1]: Closed D-Bus System Message Bus Socket.1604client # [ 6.069767] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.1605client # [ 6.071625] systemd[1]: Stopped Find NixOS closure.1606client # [ 6.077447] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1607client # [ 6.080565] systemd[1]: Starting rw-sysroot-nix-store.service...1608client # [ 6.088211] systemd[1]: systemd-sysctl.service: Deactivated successfully.1609client # [ 6.092246] systemd[1]: Stopped Apply Kernel Variables.1610client # [ 6.094006] systemd[1]: systemd-modules-load.service: Deactivated successfully.1611client # [ 6.105313] systemd[1]: Stopped Load Kernel Modules.1612client # [ 6.108505] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.1613client # [ 6.117697] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.1614client # [ 6.126339] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.1615client # [ 6.133391] systemd[1]: Stopped Create System Files and Directories.1616client # [ 6.136235] systemd[1]: Stopped target Local File Systems.1617client # [ 6.139381] systemd[1]: Stopped target Preparation for Local File Systems.1618client # [ 6.143986] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.1619client # [ 6.147801] systemd[1]: Stopped Coldplug All udev Devices.1620client # [ 6.151247] systemd[1]: Stopping Rule-based Manager for Device Events and Files...1621client # [ 6.156332] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1622client # [ 6.157704] systemd[1]: Stopped Virtual Console Setup.1623client # [ 6.160253] systemd[1]: initrd-cleanup.service: Deactivated successfully.1624client # [ 6.163428] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.1625client # [ 6.168216] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1626client # [ 6.173619] systemd[1]: Finished rw-sysroot-nix-store.service.1627client # [ 6.174567] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1628client # [ 6.180310] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1629client # [ 6.194754] systemd[1]: systemd-udevd.service: Deactivated successfully.1630client # [ 6.196799] systemd[1]: Stopped Rule-based Manager for Device Events and Files.1631client # [ 6.200234] systemd[1]: systemd-udevd.service: Consumed 1.230s CPU time over 3.490s wall clock time, 17.3M memory peak.1632client # [ 6.202190] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.1633client # [ 6.205872] systemd[1]: Closed udev Control Socket.1634client # [ 6.207989] systemd[1]: Starting Cleanup udev Database...1635client # [ 6.217062] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.1636client # [ 6.228622] systemd[1]: Stopped Create Static Device Nodes in /dev.1637client # [ 6.244935] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.1638client # [ 6.246024] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.1639client # [ 6.247010] systemd[1]: kmod-static-nodes.service: Deactivated successfully.1640client # [ 6.247932] systemd[1]: Stopped Create List of Static Device Nodes.1641client # [ 6.282296] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.1642client # [ 6.284677] systemd[1]: Finished Cleanup udev Database.1643client # [ 6.287974] systemd[1]: Reached target Switch Root.1644client # [ 6.292684] systemd[1]: Starting NixOS Activation...1645lb # [ 6.165255] (udev-worker)[155]: mtd0ro: Failed to find and pin callout binary "/nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/mtd_probe": No such file or directory1646lb # [ 6.178352] (udev-worker)[155]: mtd0ro: /nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory1647lb # [ 6.211586] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1648lb # [ 6.216213] systemd[1]: Stopped Virtual Console Setup.1649lb # [ 6.222393] systemd[1]: Stopping Virtual Console Setup...1650lb # [ 6.228890] systemd[1]: Starting Virtual Console Setup...1651lb # [ 6.261267] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1652lb # [ 6.263733] systemd[1]: Stopped Virtual Console Setup.1653lb # [ 6.271061] systemd[1]: Starting Virtual Console Setup...1654lb # [ 6.338891] systemd-vconsole-setup[166]: Configuration of first virtual console was skipped, ignoring remaining ones.1655lb # [ 6.352992] systemd[1]: Finished Virtual Console Setup.1656lb # [ 6.355075] systemd[1]: Mounting /sysroot/nix/store...1657lb # [ 6.418142] systemd[1]: Mounted /sysroot/nix/store.1658lb # [ 6.427837] systemd[1]: Reached target Initrd File Systems.1659lb # [ 6.436465] systemd[1]: Starting Find NixOS closure...1660lb # [ 6.444824] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...1661worker2 # [ 6.410555] systemd[1]: Mounting /sysroot/nix/store...1662worker2 # [ 6.478259] systemd[1]: Mounted /sysroot/nix/store.1663lb # [ 6.522967] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.1664lb # [ 6.525540] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.1665worker2 # [ 6.481800] systemd[1]: Reached target Initrd File Systems.1666worker2 # [ 6.492396] systemd[1]: Starting Find NixOS closure...1667lb # [ 6.548494] systemd[1]: Finished Find NixOS closure.1668lb # [ 6.555102] systemd[1]: Reached target Initrd Default Target.1669lb # [ 6.558876] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...1670worker2 # [ 6.511247] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...1671lb # [ 6.590783] systemd[1]: Stopped target Initrd Default Target.1672lb # [ 6.593026] systemd[1]: Stopped target Basic System.1673lb # [ 6.596206] systemd[1]: Stopped target Initrd Root Device.1674lb # [ 6.598015] systemd[1]: Stopped target Path Units.1675lb # [ 6.599897] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.1676lb # [ 6.604675] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.1677lb # [ 6.623454] systemd[1]: Stopped target Slice Units.1678worker2 # [ 6.581385] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.1679lb # [ 6.626051] systemd[1]: Stopped target Socket Units.1680worker2 # [ 6.589536] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.1681lb # [ 6.640773] systemd[1]: Stopped target System Initialization.1682lb # [ 6.644235] systemd[1]: Stopped target Swaps.1683lb # [ 6.645091] systemd[1]: Stopped target Timer Units.1684lb # [ 6.648352] systemd[1]: dbus.socket: Deactivated successfully.1685lb # [ 6.649436] systemd[1]: Closed D-Bus System Message Bus Socket.1686lb # [ 6.651066] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.1687lb # [ 6.656421] systemd[1]: Stopped Find NixOS closure.1688lb # [ 6.657312] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1689lb # [ 6.658338] systemd[1]: Starting rw-sysroot-nix-store.service...1690lb # [ 6.659259] systemd[1]: systemd-sysctl.service: Deactivated successfully.1691lb # [ 6.664678] systemd[1]: Stopped Apply Kernel Variables.1692lb # [ 6.669273] systemd[1]: systemd-modules-load.service: Deactivated successfully.1693worker2 # [ 6.627979] systemd[1]: Finished Find NixOS closure.1694lb # [ 6.674015] systemd[1]: Stopped Load Kernel Modules.1695worker2 # [ 6.630870] systemd[1]: Reached target Initrd Default Target.1696worker2 # [ 6.636575] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...1697lb # [ 6.681421] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.1698lb # [ 6.689371] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.1699lb # [ 6.695424] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.1700lb # [ 6.708625] systemd[1]: Stopped Create System Files and Directories.1701lb # [ 6.709663] systemd[1]: Stopped target Local File Systems.1702lb # [ 6.710614] systemd[1]: Stopped target Preparation for Local File Systems.1703lb # [ 6.711528] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.1704lb # [ 6.725268] systemd[1]: Stopped Coldplug All udev Devices.1705lb # [ 6.726955] systemd[1]: Stopping Rule-based Manager for Device Events and Files...1706lb # [ 6.732552] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1707lb # [ 6.735876] systemd[1]: Stopped Virtual Console Setup.1708lb # [ 6.738579] systemd[1]: initrd-cleanup.service: Deactivated successfully.1709lb # [ 6.741698] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.1710lb # [ 6.744939] systemd[1]: systemd-udevd.service: Deactivated successfully.1711lb # [ 6.748512] systemd[1]: Stopped Rule-based Manager for Device Events and Files.1712lb # [ 6.752474] systemd[1]: systemd-udevd.service: Consumed 1.813s CPU time over 4.172s wall clock time, 28.4M memory peak.1713lb # [ 6.757255] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1714lb # [ 6.760351] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1715worker2 # [ 6.716566] systemd[1]: Stopped target Initrd Default Target.1716lb # [ 6.764298] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1717lb # [ 6.767200] systemd[1]: Finished rw-sysroot-nix-store.service.1718lb # [ 6.768436] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.1719lb # [ 6.769406] systemd[1]: Closed udev Control Socket.1720lb # [ 6.770104] systemd[1]: Starting Cleanup udev Database...1721worker2 # [ 6.723092] systemd[1]: Stopped target Basic System.1722lb # [ 6.770840] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.1723worker2 # [ 6.728626] systemd[1]: Stopped target Initrd Root Device.1724worker2 # [ 6.730907] systemd[1]: Stopped target Path Units.1725lb # [ 6.771844] systemd[1]: Stopped Create Static Device Nodes in /dev.1726worker2 # [ 6.732636] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.1727lb # [ 6.772834] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.1728lb # [ 6.773901] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.1729worker2 # [ 6.738760] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.1730worker2 # [ 6.742938] systemd[1]: Stopped target Slice Units.1731worker2 # [ 6.744930] systemd[1]: Stopped target Socket Units.1732lb # [ 6.774842] systemd[1]: kmod-static-nodes.service: Deactivated successfully.1733lb # [ 6.775762] systemd[1]: Stopped Create List of Static Device Nodes.1734worker2 # [ 6.749124] systemd[1]: Stopped target System Initialization.1735worker2 # [ 6.755203] systemd[1]: Stopped target Swaps.1736worker2 # [ 6.757770] systemd[1]: Stopped target Timer Units.1737worker2 # [ 6.760989] systemd[1]: dbus.socket: Deactivated successfully.1738worker2 # [ 6.767849] systemd[1]: Closed D-Bus System Message Bus Socket.1739lb # [ 6.819677] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.1740worker2 # [ 6.777511] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.1741worker2 # [ 6.778624] systemd[1]: Stopped Find NixOS closure.1742worker2 # [ 6.779283] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1743lb # [ 6.833407] systemd[1]: Finished Cleanup udev Database.1744lb # [ 6.834275] systemd[1]: Reached target Switch Root.1745lb # [ 6.834958] systemd[1]: Starting NixOS Activation...1746worker2 # [ 6.785805] systemd[1]: Starting rw-sysroot-nix-store.service...1747worker2 # [ 6.794507] systemd[1]: systemd-sysctl.service: Deactivated successfully.1748worker2 # [ 6.798926] systemd[1]: Stopped Apply Kernel Variables.1749worker2 # [ 6.801596] systemd[1]: systemd-modules-load.service: Deactivated successfully.1750worker2 # [ 6.812248] systemd[1]: Stopped Load Kernel Modules.1751client # [ 7.045374] initrd-nixos-activation-start[173]: booting system configuration /nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test1752worker2 # [ 6.823021] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.1753worker2 # [ 6.829164] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.1754worker2 # [ 6.830929] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.1755worker2 # [ 6.839707] systemd[1]: Stopped Create System Files and Directories.1756worker2 # [ 6.845763] systemd[1]: Stopped target Local File Systems.1757worker2 # [ 6.854078] systemd[1]: Stopped target Preparation for Local File Systems.1758worker2 # [ 6.858031] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.1759worker2 # [ 6.868513] systemd[1]: Stopped Coldplug All udev Devices.1760worker2 # [ 6.870456] systemd[1]: Stopping Rule-based Manager for Device Events and Files...1761worker2 # [ 6.876861] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1762worker2 # [ 6.883102] systemd[1]: Stopped Virtual Console Setup.1763worker2 # [ 6.885928] systemd[1]: initrd-cleanup.service: Deactivated successfully.1764worker2 # [ 6.891039] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.1765worker2 # [ 6.893248] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1766worker2 # [ 6.896338] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1767worker2 # [ 6.900699] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.1768worker2 # [ 6.904337] systemd[1]: Finished rw-sysroot-nix-store.service.1769worker2 # [ 6.907943] systemd[1]: systemd-udevd.service: Deactivated successfully.1770worker2 # [ 6.909744] systemd[1]: Stopped Rule-based Manager for Device Events and Files.1771worker2 # [ 6.911616] systemd[1]: systemd-udevd.service: Consumed 1.946s CPU time over 4.019s wall clock time, 24.8M memory peak.1772worker2 # [ 6.914386] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.1773worker2 # [ 6.916396] systemd[1]: Closed udev Control Socket.1774worker2 # [ 6.917127] systemd[1]: Starting Cleanup udev Database...1775worker2 # [ 6.917928] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.1776worker2 # [ 6.918962] systemd[1]: Stopped Create Static Device Nodes in /dev.1777worker2 # [ 6.919825] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.1778worker2 # [ 6.926686] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.1779worker2 # [ 6.929015] systemd[1]: kmod-static-nodes.service: Deactivated successfully.1780worker2 # [ 6.932600] systemd[1]: Stopped Create List of Static Device Nodes.1781worker2 # [ 6.970444] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.1782worker2 # [ 6.975477] systemd[1]: Finished Cleanup udev Database.1783worker2 # [ 6.977170] systemd[1]: Reached target Switch Root.1784worker2 # [ 6.980257] systemd[1]: Starting NixOS Activation...1785client # [ 7.226674] initrd-nixos-activation-start[173]: running activation script...1786lb # [ 7.334212] initrd-nixos-activation-start[191]: booting system configuration /nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test1787worker2 # [ 7.533932] initrd-nixos-activation-start[185]: booting system configuration /nix/store/0r31wjlygcn47dyyzl916b35v9f3ng6k-nixos-system-worker2-test1788lb # [ 7.599845] initrd-nixos-activation-start[191]: running activation script...1789worker1 # Disk image does not exist, creating the virtualisation disk image...1790worker1: QEMU running (pid 52)1791worker1 # Formatting '/build/vm-state-worker1/tmp.oy7GDA21G2', fmt=raw size=10737418241792worker1 # mke2fs 1.47.4 (6-Mar-2025)1793worker1 # Discarding device blocks: 0/262144 done1794worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes1795worker1 # Filesystem UUID: e6ea0c84-1d13-4a22-9125-3ee6332bc7cf1796worker1 # Superblock backups stored on blocks:1797worker1 # 32768, 98304, 163840, 2293761798worker1 # 1799worker1 # Allocating group tables: 0/8 done1800worker1 # Writing inode tables: 0/8 done1801worker1 # Creating journal (8192 blocks): done1802worker1 # Writing superblocks and filesystem accounting information: 0/8 done1803worker1 # 1804worker1 # Virtualisation disk image created.1805(finished: start all VMs, in 9.44 seconds)1806client: waiting for unit network-addresses-eth1.service1807client: waiting for the VM to finish booting1808worker2 # [ 7.812193] initrd-nixos-activation-start[185]: running activation script...1809worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]1810worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 20261811worker1 # [ 0.000000] KASLR enabled1812worker1 # [ 0.000000] random: crng init done1813worker1 # [ 0.000000] Machine model: linux,dummy-virt1814worker1 # [ 0.000000] efi: UEFI not found.1815worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT1816worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]1817worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]1818worker1 # [ 0.000000] Zone ranges:1819worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]1820worker1 # [ 0.000000] DMA32 empty1821worker1 # [ 0.000000] Normal empty1822worker1 # [ 0.000000] Device empty1823worker1 # [ 0.000000] Movable zone start for each node1824worker1 # [ 0.000000] Early memory node ranges1825worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]1826worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]1827worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c4000001828worker1 # [ 0.000000] psci: probing for conduit method from DT.1829worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.1830worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs1831worker1 # [ 0.000000] psci: Trusted OS migration not required1832worker1 # [ 0.000000] psci: SMC Calling Convention v1.11833worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)1834worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u3112961835worker1 # [ 0.000000] Detected PIPT I-cache on CPU01836worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)1837worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface1838worker1 # [ 0.000000] CPU features: detected: Spectre-v41839worker1 # [ 0.000000] CPU features: detected: Spectre-BHB1840worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_381841worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_231842worker1 # [ 0.000000] alternatives: applying boot alternatives1843worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zwgirfamq9cjz73mhvx6r4x90a72kbbz-nixos-system-worker1-test/init regInfo=/nix/store/8vxdl5gm74j84gimba2hrvfgq4533bsm-closure-info/registration console=ttyAMA0,115200n8 console=tty01844worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8vxdl5gm74j84gimba2hrvfgq4533bsm-closure-info/registration", will be passed to user space.1845worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes1846worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)1847worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)1848worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB1849worker1 # [ 0.000000] software IO TLB: area num 1.1850worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB1851worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)1852worker1 # [ 0.000000] Fallback order for Node 0: 01853worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 3932161854worker1 # [ 0.000000] Policy zone: DMA1855worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off1856worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=11857worker1 # [ 0.000000] allocated 3145728 bytes of page_ext1858worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages1859worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups1860worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.1861worker1 # [ 0.000000] rcu: RCU event tracing is enabled.1862worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.1863worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.1864worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.1865worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.1866worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.1867worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=11868worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1869worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1870worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1871worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 01872worker1 # [ 0.000000] GICv3: 256 SPIs implemented1873worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented1874worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq1875worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI1876worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=01877worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a00001878worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]1879worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)1880worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)1881worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a100001882worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a200001883worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.1884worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns1885worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).1886worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns1887worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns1888worker1 # [ 0.000038] arm-pv: using stolen time PV1889worker1 # [ 0.000534] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)1890worker1 # [ 0.004972] Console: colour dummy device 80x251891worker1 # [ 0.004989] printk: legacy console [tty0] enabled1892worker1 # [ 0.005205] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)1893worker1 # [ 0.005213] pid_max: default: 32768 minimum: 3011894worker1 # [ 0.005300] LSM: initializing lsm=capability,landlock,yama,bpf,ima1895worker1 # [ 0.005499] landlock: Up and running.1896worker1 # [ 0.005503] Yama: becoming mindful.1897worker1 # [ 0.006070] LSM support for eBPF active1898worker1 # [ 0.006299] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1899worker1 # [ 0.006332] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1900worker1 # [ 0.008293] cacheinfo: Unable to detect cache hierarchy for CPU 01901worker1 # [ 0.009217] rcu: Hierarchical SRCU implementation.1902worker1 # [ 0.009223] rcu: Max phase no-delay instances is 1000.1903worker1 # [ 0.010571] fsl-mc MSI: its@8080000 domain created1904worker1 # [ 0.010673] EFI services will not be available.1905worker1 # [ 0.010752] smp: Bringing up secondary CPUs ...1906worker1 # [ 0.010763] smp: Brought up 1 node, 1 CPU1907worker1 # [ 0.010766] SMP: Total of 1 processors activated.1908worker1 # [ 0.010774] CPU: All CPU(s) started at EL11909worker1 # [ 0.010788] CPU features: detected: Branch Target Identification1910worker1 # [ 0.010794] CPU features: detected: ARMv8.4 Translation Table Level1911worker1 # [ 0.010797] CPU features: detected: Instruction cache invalidation not required for I/D coherence1912worker1 # [ 0.010801] CPU features: detected: Data cache clean to the PoU not required for I/D coherence1913worker1 # [ 0.010805] CPU features: detected: Common not Private translations1914worker1 # [ 0.010809] CPU features: detected: CRC32 instructions1915worker1 # [ 0.010812] CPU features: detected: Data cache clean to Point of Deep Persistence1916worker1 # [ 0.010815] CPU features: detected: Data cache clean to Point of Persistence1917worker1 # [ 0.010819] CPU features: detected: Data independent timing control (DIT)1918worker1 # [ 0.010822] CPU features: detected: E0PD1919worker1 # [ 0.010825] CPU features: detected: Enhanced Counter Virtualization1920worker1 # [ 0.010828] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)1921worker1 # [ 0.010831] CPU features: detected: Enhanced Virtualization Traps1922worker1 # [ 0.010834] CPU features: detected: Fine Grained Traps1923worker1 # [ 0.010839] CPU features: detected: Generic authentication (architected QARMA5 algorithm)1924worker1 # [ 0.010844] CPU features: detected: RCpc load-acquire (LDAPR)1925worker1 # [ 0.010848] CPU features: detected: LSE atomic instructions1926worker1 # [ 0.010851] CPU features: detected: Privileged Access Never1927worker1 # [ 0.010854] CPU features: detected: PMUv31928worker1 # [ 0.010857] CPU features: detected: RAS Extension Support1929worker1 # [ 0.010860] CPU features: detected: RASv1p1 Extension Support1930worker1 # [ 0.010862] CPU features: detected: Random Number Generator1931worker1 # [ 0.010865] CPU features: detected: Speculation barrier (SB)1932worker1 # [ 0.010868] CPU features: detected: Stage-2 Force Write-Back1933worker1 # [ 0.010872] CPU features: detected: TLB range maintenance instructions1934worker1 # [ 0.010876] CPU features: detected: Speculative Store Bypassing Safe (SSBS)1935worker1 # [ 0.010921] alternatives: applying system-wide alternatives1936worker1 # [ 0.018792] CPU features: detected: BBM Level 2 without TLB conflict abort1937worker1 # [ 0.019037] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)1938worker1 # [ 0.019610] devtmpfs: initialized1939worker1 # [ 0.021761] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)1940worker1 # [ 0.021793] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).1941worker1 # [ 0.022022] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL1942worker1 # [ 0.022028] 0 pages in range for non-PLT usage1943worker1 # [ 0.022029] 508288 pages in range for PLT usage1944worker1 # [ 0.022145] pinctrl core: initialized pinctrl subsystem1945worker1 # [ 0.023084] DMI not present or invalid.1946worker1 # [ 0.045167] NET: Registered PF_NETLINK/PF_ROUTE protocol family1947worker1 # [ 0.047793] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1948worker1 # [ 0.047994] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1949worker1 # [ 0.048220] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1950worker1 # [ 0.048285] audit: initializing netlink subsys (disabled)1951worker1 # [ 0.048742] audit: type=2000 audit(0.028:1): state=initialized audit_enabled=0 res=11952worker1 # [ 0.049087] thermal_sys: Registered thermal governor 'fair_share'1953worker1 # [ 0.049089] thermal_sys: Registered thermal governor 'bang_bang'1954worker1 # [ 0.049096] thermal_sys: Registered thermal governor 'step_wise'1955worker1 # [ 0.049099] thermal_sys: Registered thermal governor 'user_space'1956worker1 # [ 0.049102] thermal_sys: Registered thermal governor 'power_allocator'1957worker1 # [ 0.049140] cpuidle: using governor ladder1958worker1 # [ 0.049149] cpuidle: using governor menu1959worker1 # [ 0.049369] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.1960worker1 # [ 0.049390] ASID allocator initialised with 65536 entries1961worker1 # [ 0.050668] Serial: AMBA PL011 UART driver1962worker1 # [ 0.060498] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev11963worker1 # [ 0.060738] printk: console [ttyAMA0] enabled1964worker1 # [ 0.255882] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1965worker1 # [ 0.255911] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page1966worker1 # [ 0.255919] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages1967worker1 # [ 0.255923] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page1968worker1 # [ 0.255928] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1969worker1 # [ 0.255932] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page1970worker1 # [ 0.255936] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages1971worker1 # [ 0.255941] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page1972worker1 # [ 0.279811] fbcon: Taking over console1973worker1 # [ 0.279846] ACPI: Interpreter disabled.1974worker1 # [ 0.285840] iommu: Default domain type: Translated1975worker1 # [ 0.285859] iommu: DMA domain TLB invalidation policy: strict mode1976worker1 # [ 0.288194] SCSI subsystem initialized1977worker1 # [ 0.289147] usbcore: registered new interface driver usbfs1978worker1 # [ 0.290156] usbcore: registered new interface driver hub1979worker1 # [ 0.290187] usbcore: registered new device driver usb1980worker1 # [ 0.290523] pps_core: LinuxPPS API ver. 1 registered1981worker1 # [ 0.290529] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>1982worker1 # [ 0.290541] PTP clock support registered1983worker1 # [ 0.290595] EDAC MC: Ver: 3.0.01984worker1 # [ 0.296031] scmi_core: SCMI protocol bus registered1985worker1 # [ 0.297346] FPGA manager framework1986worker1 # [ 0.298602] vgaarb: loaded1987worker1 # [ 0.299546] clocksource: Switched to clocksource arch_sys_counter1988worker1 # [ 0.300293] VFS: Disk quotas dquot_6.6.01989worker1 # [ 0.300331] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1990worker1 # [ 0.306815] netfs: FS-Cache loaded1991worker1 # [ 0.306987] pnp: PnP ACPI: disabled1992worker1 # [ 0.311536] NET: Registered PF_INET protocol family1993worker1 # [ 0.312798] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1994worker1 # [ 0.357676] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1995worker1 # [ 0.357748] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1996worker1 # [ 0.357786] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1997worker1 # [ 0.357883] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1998worker1 # [ 0.358030] TCP: Hash tables configured (established 16384 bind 16384)1999worker1 # [ 0.358146] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)2000worker1 # [ 0.358189] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)
loading 1,698 hidden lines…
3699worker1 # [ 31.325596] nix-grpc-daemon[803]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 31.356351] systemd[1]: Started DHCP Client.3701worker1 # [ 31.359986] systemd[1]: Reached target Multi-User System.3702worker1 # [ 31.361533] systemd[1]: Startup finished in 1.487s (kernel) + 8.425s (initrd) + 21.446s (userspace) = 31.359s.3703worker2 # [ 39.651028] nix-grpc-daemon[871]: ts=2026-09-20T12:21:24Z level=info event=scheduler_connected addr=lb:500513704worker2 # [ 39.659106] nix-grpc-daemon[871]: ts=2026-09-20T12:21:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker1 # [ 31.526451] nix-grpc-daemon[803]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 31.529642] nix-grpc-daemon[803]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 39.860395] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 39.863631] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker1 # [ 31.730263] nix-grpc-daemon[803]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 31.737834] nix-grpc-daemon[803]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 40.087832] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 40.140674] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker1 # [ 31.939333] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 31.943076] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 40.342689] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 40.346477] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker1 # [ 32.144243] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 32.147306] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 40.548477] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_connected addr=lb:500513720worker2 # [ 40.556965] nix-grpc-daemon[871]: ts=2026-09-20T12:21:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker1 # [ 32.348517] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 32.351936] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 40.759174] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 40.765741] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker1 # [ 32.553490] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 32.559070] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 40.968210] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 40.977019] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 32.760593] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 32.769734] nix-grpc-daemon[803]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 41.179017] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 41.182710] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 32.975695] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 32.985859] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 41.389350] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 41.400393] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker1 # [ 33.186038] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 33.201087] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker2 # [ 41.601383] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 41.603994] nix-grpc-daemon[871]: ts=2026-09-20T12:21:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker1 # [ 33.402057] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513742worker1 # [ 33.405369] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker2 # [ 41.805832] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 41.812664] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker1 # [ 33.605947] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 33.608275] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 42.014206] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 42.020145] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker1 # [ 33.809965] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 33.817723] nix-grpc-daemon[803]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker2 # [ 42.221930] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513752worker2 # [ 42.228520] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker1 # [ 34.019290] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 34.021751] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker2 # [ 42.432292] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513756worker2 # [ 42.436638] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3757worker1 # [ 34.222541] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513758worker1 # [ 34.225545] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker2 # [ 42.633737] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500513760worker2 # [ 42.639826] nix-grpc-daemon[871]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker1 # [ 34.426217] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513762worker1 # [ 34.429079] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763worker2 # [ 42.841068] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513764worker2 # [ 42.844896] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3765worker1 # [ 34.629814] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513766worker1 # [ 34.632949] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3767worker2 # [ 43.045936] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513768worker2 # [ 43.048823] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3769worker1 # [ 34.833641] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513770worker1 # [ 34.838080] nix-grpc-daemon[803]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3771worker2 # [ 43.249756] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513772worker2 # [ 43.252209] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3773worker1 # [ 35.040648] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513774worker1 # [ 35.045065] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3775worker2 # [ 43.456753] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513776worker2 # [ 43.461337] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3777worker1 # [ 35.241001] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513778worker1 # [ 35.247006] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3779worker2 # [ 43.658210] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500513780worker2 # [ 43.663568] nix-grpc-daemon[871]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3781worker1 # [ 35.447611] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513782worker1 # [ 35.451768] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3783worker2 # [ 43.864815] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513784worker2 # [ 43.867877] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3785worker1 # [ 35.652766] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513786worker1 # [ 35.657120] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3787worker2 # [ 44.068944] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513788worker2 # [ 44.071330] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3789worker1 # [ 35.858350] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 35.861277] nix-grpc-daemon[803]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791worker2 # [ 44.273488] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513792worker2 # [ 44.281008] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3793worker1 # [ 36.065024] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513794worker1 # [ 36.070029] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795worker2 # [ 44.482349] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513796worker2 # [ 44.484687] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3797worker1 # [ 36.266213] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513798worker1 # [ 36.268717] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3799worker2 # [ 44.686207] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500513800worker2 # [ 44.690461] nix-grpc-daemon[871]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3801worker1 # [ 36.469542] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513802worker1 # [ 36.476504] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803worker2 # [ 44.892616] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513804worker2 # [ 44.900829] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3805worker1 # [ 36.677174] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513806worker1 # [ 36.680300] nix-grpc-daemon[803]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807worker2 # [ 45.115092] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513808worker2 # [ 45.117678] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3809worker1 # [ 36.891214] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513810worker1 # [ 36.898158] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3811worker2 # [ 45.317324] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513812worker2 # [ 45.323725] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3813worker1 # [ 37.100095] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513814worker1 # [ 37.105257] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3815worker2 # [ 45.526313] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513816worker2 # [ 45.532765] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3817worker1 # [ 37.307241] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513818worker1 # [ 37.316787] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3819worker2 # [ 45.735461] nix-grpc-daemon[871]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500513820worker2 # [ 45.743634] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3821worker1 # [ 37.518558] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513822worker1 # [ 37.526099] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3823worker2 # [ 45.945705] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513824worker2 # [ 45.951934] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3825worker1 # [ 37.726708] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513826worker1 # [ 37.731357] nix-grpc-daemon[803]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3827worker2 # [ 46.152754] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513828worker2 # [ 46.155653] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker1 # [ 37.932137] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513830worker1 # [ 37.936184] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3831worker2 # [ 46.357566] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513832worker2 # [ 46.363672] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833worker1 # [ 38.144740] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513834worker1 # [ 38.154806] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3835worker2 # [ 46.566569] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500513836worker2 # [ 46.572751] nix-grpc-daemon[871]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3837worker1 # [ 38.341266] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513838worker1 # [ 38.347225] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3839worker2 # [ 46.774581] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513840worker1 # [ 38.548074] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513841worker2 # [ 46.780721] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842worker1 # [ 38.551981] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3843worker2 # [ 46.982355] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513844worker1 # [ 38.753386] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513845worker2 # [ 46.990493] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3846worker1 # [ 38.761912] nix-grpc-daemon[803]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3847worker2 # [ 47.192258] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513848worker1 # [ 38.962999] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513849worker2 # [ 47.199967] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850worker1 # [ 38.972137] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3851worker1 # [ 39.165690] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513852worker2 # [ 47.400944] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513853worker1 # [ 39.169921] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3854worker2 # [ 47.403927] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3855worker1 # [ 39.370935] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513856worker2 # [ 47.605851] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500513857worker1 # [ 39.374906] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3858worker2 # [ 47.609027] nix-grpc-daemon[871]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3859worker1 # [ 39.576436] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513860worker1 # [ 39.581590] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3861worker2 # [ 47.812486] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513862worker2 # [ 47.820825] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3863worker1 # [ 39.782263] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513864worker1 # [ 39.785586] nix-grpc-daemon[803]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3865worker2 # [ 48.022710] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513866worker2 # [ 48.026339] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3867worker1 # [ 39.986355] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513868worker1 # [ 39.993802] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3869worker2 # [ 48.241012] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513870worker2 # [ 48.243790] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3871worker1 # [ 40.195322] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513872worker1 # [ 40.203004] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3873worker2 # [ 48.446782] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513874worker2 # [ 48.449908] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3875worker1 # [ 40.403563] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513876worker1 # [ 40.410079] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3877worker2 # [ 48.651908] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500513878worker2 # [ 48.656720] nix-grpc-daemon[871]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3879worker1 # [ 40.611392] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513880worker1 # [ 40.615710] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3881worker2 # [ 48.857777] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513882worker2 # [ 48.862305] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3883worker1 # [ 40.817410] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513884worker2 # [ 49.066407] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513885worker2 # [ 49.071465] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886worker1 # [ 40.846469] nix-grpc-daemon[803]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3887worker2 # [ 49.268517] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513888worker2 # [ 49.274786] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3889worker1 # [ 41.047383] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513890worker1 # [ 41.065185] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3891worker2 # [ 49.477233] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513892worker2 # [ 49.485083] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3893worker1 # [ 41.275406] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513894worker1 # [ 41.279293] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3895worker2 # [ 49.686789] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500513896worker2 # [ 49.692901] nix-grpc-daemon[871]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3897worker1 # [ 41.475668] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513898worker1 # [ 41.479696] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3899worker2 # [ 49.895162] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513900worker2 # [ 49.901745] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3901worker1 # [ 41.681032] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513902worker1 # [ 41.683698] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3903worker2 # [ 50.103154] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513904worker2 # [ 50.106871] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3905worker1 # [ 41.884588] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513906worker1 # [ 41.888313] nix-grpc-daemon[803]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3907worker2 # [ 50.307616] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513908worker2 # [ 50.313625] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3909worker1 # [ 42.089126] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513910worker1 # [ 42.091711] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3911worker2 # [ 50.516734] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513912worker2 # [ 50.525040] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3913worker1 # [ 42.295796] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513914worker1 # [ 42.301591] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3915worker2 # [ 50.727215] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500513916worker2 # [ 50.730526] nix-grpc-daemon[871]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3917worker1 # [ 42.496882] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513918worker1 # [ 42.503683] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3919worker2 # [ 50.929042] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513920worker2 # [ 50.931700] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3921worker1 # [ 42.704812] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513922worker1 # [ 42.710684] nix-grpc-daemon[803]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3923worker2 # [ 51.132964] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513924worker2 # [ 51.139500] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3925worker1 # [ 42.911644] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513926worker1 # [ 42.917282] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3927worker2 # [ 51.339565] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513928worker2 # [ 51.342856] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3929worker1 # [ 43.117812] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513930worker1 # [ 43.123647] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3931worker2 # [ 51.544308] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500513932worker2 # [ 51.550663] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3933worker1 # [ 43.327204] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513934worker1 # [ 43.332150] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3935worker2 # [ 51.735400] nix-grpc-daemon[871]: ts=2026-09-20T12:21:36Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3936worker2 # [ 51.751915] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513937worker2 # [ 51.758363] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3938worker1 # [ 43.528163] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513939worker1 # [ 43.533774] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3940worker2 # [ 51.958950] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513941worker2 # [ 51.962764] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3942worker1 # [ 43.734895] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513943worker1 # [ 43.742182] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3944worker1 # [ 43.804943] nix-grpc-daemon[803]: ts=2026-09-20T12:21:37Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3945worker2 # [ 52.164718] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513946worker2 # [ 52.171281] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3947worker1 # [ 43.943967] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513948worker1 # [ 43.950521] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3949worker2 # [ 52.372929] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513950worker2 # [ 52.377640] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3951worker1 # [ 44.151611] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513952worker1 # [ 44.158336] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3953worker2 # [ 52.579320] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500513954worker2 # [ 52.585343] nix-grpc-daemon[871]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3955worker1 # [ 44.361288] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513956worker1 # [ 44.365800] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3957worker2 # [ 52.789781] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513958worker1 # [ 44.561839] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513959worker2 # [ 52.795961] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3960worker1 # [ 44.567471] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3961worker2 # [ 52.990030] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513962worker2 # [ 52.996134] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3963worker1 # [ 44.769077] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513964worker1 # [ 44.776531] nix-grpc-daemon[803]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3965worker2 # [ 53.197232] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513966worker2 # [ 53.201252] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3967worker1 # [ 44.976735] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513968worker1 # [ 44.980739] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3969worker2 # [ 53.402445] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513970worker2 # [ 53.407904] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3971worker1 # [ 45.181744] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513972worker1 # [ 45.187406] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3973worker2 # [ 53.609147] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500513974worker2 # [ 53.615004] nix-grpc-daemon[871]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3975worker1 # [ 45.389896] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513976worker1 # [ 45.396210] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3977worker2 # [ 53.818627] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513978worker2 # [ 53.823367] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3979worker1 # [ 45.598074] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513980worker1 # [ 45.603729] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3981worker2 # [ 54.020100] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513982worker2 # [ 54.025937] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3983worker1 # [ 45.804907] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513984worker1 # [ 45.810970] nix-grpc-daemon[803]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3985worker2 # [ 54.227419] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513986worker2 # [ 54.233327] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3987worker1 # [ 46.011386] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500513988worker1 # [ 46.015313] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3989worker2 # [ 54.434634] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513990worker2 # [ 54.440553] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3991worker1 # [ 46.216077] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500513992worker1 # [ 46.221682] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3993worker2 # [ 54.641568] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500513994worker2 # [ 54.644875] nix-grpc-daemon[871]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3995worker1 # [ 46.424987] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500513996worker1 # [ 46.429745] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3997worker2 # [ 54.848781] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500513998worker2 # [ 54.853260] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3999worker1 # [ 46.625165] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514000worker1 # [ 46.629605] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4001worker2 # [ 55.049718] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514002worker2 # [ 55.055688] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4003worker1 # [ 46.830516] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514004worker1 # [ 46.836500] nix-grpc-daemon[803]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4005lb # [ 55.273353] rustfs-bucket-start[1084]: mb s3://farm4006lb # [ 55.281677] systemd[1]: Finished rustfs-bucket.service.4007lb # [ 55.292632] systemd[1]: Starting niks3 server...4008worker2 # [ 55.257248] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514009worker2 # [ 55.263254] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4010worker1 # [ 47.037686] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514011worker1 # [ 47.043849] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4012worker2 # [ 55.464741] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514013worker2 # [ 55.470915] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4014worker1 # [ 47.244573] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514015worker1 # [ 47.248577] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4016lb # [ 55.589613] postgres[1099]: [1099] ERROR: relation "goose_db_version" does not exist at character 364017lb # [ 55.591066] postgres[1099]: [1099] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4018lb # [ 55.636609] niks3-server[1093]: 2026/09/20 12:21:40 OK 20241026095416_initial_model.sql (29.09ms)4019lb # [ 55.642185] niks3-server[1093]: 2026/09/20 12:21:40 OK 20251210153512_drop_unused_gin_index.sql (3.63ms)4020lb # [ 55.650748] niks3-server[1093]: 2026/09/20 12:21:40 OK 20251218171726_add_pins.sql (8.55ms)4021lb # [ 55.657829] niks3-server[1093]: 2026/09/20 12:21:40 OK 20260628120000_add_object_size_and_stats.sql (7.01ms)4022lb # [ 55.664828] niks3-server[1093]: 2026/09/20 12:21:40 OK 20260905000000_add_claims.sql (6.93ms)4023lb # [ 55.671212] niks3-server[1093]: 2026/09/20 12:21:40 OK 20260920000000_drop_claims.sql (6.18ms)4024lb # [ 55.673942] niks3-server[1093]: 2026/09/20 12:21:40 goose: successfully migrated database to version: 202609200000004025lb # [ 55.679916] niks3-server[1093]: 2026/09/20 12:21:40 OK 1_commit_pending_closure.sql (8.63ms)4026lb # [ 55.684246] niks3-server[1093]: 2026/09/20 12:21:40 OK 2_object_stats_trigger.sql (4.09ms)4027lb # [ 55.685584] niks3-server[1093]: 2026/09/20 12:21:40 goose: up to current file version: 24028lb # [ 55.691007] niks3-server[1093]: 2026/09/20 12:21:40 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key4029worker2 # [ 55.672403] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514030worker2 # [ 55.678381] nix-grpc-daemon[871]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4031worker1 # [ 47.451650] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514032worker1 # [ 47.456746] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4033lb # [ 55.754648] niks3-server[1093]: 2026/09/20 12:21:40 INFO Read proxy enabled — serving cache objects from S34034lb # [ 55.757917] niks3-server[1093]: 2026/09/20 12:21:40 INFO Using socket-activated listener address=0.0.0.0:57514035lb # [ 55.762218] niks3-server[1093]: 2026/09/20 12:21:40 INFO systemd watchdog enabled interval=15s4036lb # [ 55.763557] systemd[1]: Started niks3 server.4037lb # [ 55.769952] systemd[1]: Reached target Multi-User System.4038lb # [ 55.774004] niks3-server[1093]: 2026/09/20 12:21:40 INFO Starting HTTP server address=0.0.0.0:57514039lb # [ 55.782742] systemd[1]: Startup finished in 1.435s (kernel) + 8.540s (initrd) + 45.791s (userspace) = 55.767s.4040lb # [ 55.791302] niks3-server[1093]: 2026/09/20 12:21:40 INFO lead: acquired remote=192.168.1.3:441084041worker1 # [ 47.523507] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_take_over4042worker2 # [ 55.882565] nix-grpc-daemon[871]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514043worker1 # [ 47.651936] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514044worker2 # [ 55.887972] nix-grpc-daemon[871]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4045worker1 # [ 47.658322] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4046worker2 # [ 56.082677] nix-grpc-daemon[871]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514047worker1 # [ 47.858801] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500514048worker1 # [ 47.863402] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:518064049worker1 # [ 47.866343] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04050worker1 # [ 47.884183] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=oidc_jwks provider=mock keys=14051worker1 # [ 47.889503] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:518064052worker1 # [ 47.893676] nix-grpc-daemon[803]: ts=2026-09-20T12:21:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04053lb: (finished: waiting for unit niks3.service, in 22.78 seconds)4054lb: waiting for TCP port 5751 on localhost4055lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4056lb: (finished: waiting for TCP port 5751 on localhost, in 0.10 seconds)4057lb: waiting for unit envoy.service4058lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4059lb: waiting for TCP port 50051 on localhost4060lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4061lb # [ 56.449235] envoy[851]: conn peer=[::1]:45348 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44062lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)4063worker2 # [ 56.536817] systemd[1]: Starting scheduler-token.service...4064worker2 # [ 56.639157] systemd[1]: scheduler-token.service: Deactivated successfully.4065worker2 # [ 56.643857] systemd[1]: Finished scheduler-token.service.4066worker2 # [ 56.645805] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 105ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4067lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4068??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4069 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284070lb: must succeed: curl -sf localhost:9901/clusters4071lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4072lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4073subtest: one node schedules and both builders hold a WorkerSession on it4074lb: must succeed: curl -sf localhost:9901/clusters4075lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4076worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4077worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4078lb: must succeed: curl -sf localhost:9901/clusters4079lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4080lb: must succeed: curl -sf localhost:9901/clusters4081lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4082worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4083worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)4084(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.38 seconds)4085subtest: balancer auth: client cert, bearer token, nothing, unknown CN4086worker2 # [ 57.592185] nix-grpc-daemon[871]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:600084087worker2 # [ 57.611217] systemd[1]: Started Nix Daemon.4088worker2 # [ 58.498038] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4089worker2 # [ 58.584768] nix-grpc-daemon[871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)4090worker2 # [ 58.625858] nix-grpc-daemon[871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 2/5)4091worker2 # [ 58.658769] nix-grpc-daemon[871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 3/5)4092worker2 # [ 58.762889] nix-grpc-daemon[871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 4/5)4093worker2 # [ 58.833938] nix-grpc-daemon[871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4094worker1 # [ 50.697868] nix-grpc-daemon[803]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:518204095worker1 # [ 50.726984] systemd[1]: Started Nix Daemon.4096worker1 # [ 51.422262] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4097worker1 # [ 51.538803] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)4098worker1 # [ 51.587619] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)4099worker1 # [ 51.639325] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 3/5)4100worker1 # [ 51.693782] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 695 ms (attempt 4/5)4101worker1 # [ 52.391419] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4102worker1 # [ 52.426041] nix-grpc-daemon[803]: ts=2026-09-20T12:21:46Z level=debug event=path_info_miss path=a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test error="error: path '/nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test' is required, but there is no substituter that can build it"4103worker2 # [ 60.686130] nix-grpc-daemon[871]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600084104worker2 # [ 60.716331] nix-grpc-daemon[871]: ts=2026-09-20T12:21:45Z level=debug event=path_info_miss path=a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test error="error: path '/nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test' is required, but there is no substituter that can build it"4105client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4106client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.15 seconds)4107client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4108client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.08 seconds)4109worker1 # [ 53.187324] nix-grpc-daemon[803]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:518204110worker1 # [ 53.229404] nix-grpc-daemon[803]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:518204111worker1 # [ 53.249317] nix-grpc-daemon[803]: ts=2026-09-20T12:21:47Z level=debug event=path_info_miss path=a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test error="error: path '/nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test' is required, but there is no substituter that can build it"4112worker2 # [ 61.512998] nix-grpc-daemon[871]: ts=2026-09-20T12:21:46Z level=info event=oidc_jwks provider=mock keys=14113worker2 # [ 61.520741] nix-grpc-daemon[871]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:600084114worker2 # [ 61.529493] nix-grpc-daemon[871]: ts=2026-09-20T12:21:46Z level=debug event=path_info_miss path=a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test error="error: path '/nix/store/a8gc22jhjd6rmf0wand1fsy50rgzy95i-nixos-system-client-test' is required, but there is no substituter that can build it"4115client: must succeed: rm -r /var/lib/nix-grpc-store4116client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4117worker2 # [ 61.841092] nix-grpc-daemon[871]: ts=2026-09-20T12:21:47Z level=info event=denied method=QueryMissing cn=- role=none4118worker1 # [ 53.877397] nix-grpc-daemon[803]: ts=2026-09-20T12:21:47Z level=info event=denied method=QueryMissing cn=stranger role=none4119(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.02 seconds)4120subtest: DAG build is scheduled across workers and lands in the cache4121client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24122worker2 # [ 62.388878] nix-grpc-daemon[871]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:600084123client # don't know how to build these paths:4124client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4125worker1 # [ 54.169372] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204126worker1 # [ 54.217204] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04127worker1 # [ 54.223491] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14128worker1 # [ 54.264788] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204129worker1 # [ 54.286762] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204130client # copying 1 paths to 'lb:50051'...4131worker1 # [ 54.306219] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204132client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4133worker1 # [ 54.341262] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4134worker1 # [ 54.372694] nix-grpc-daemon[803]: ts=2026-09-20T12:21:48Z level=info event=niks3_push_started pid=9464135worker1 # [ 54.688220] nix-grpc-daemon[946]: time=2026-09-20T12:21:48.796Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4136worker1 # [ 54.928176] nix-daemon[934]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4137worker1 # [ 54.962550] nix-daemon[934]: reaped child process 962, status = succeeded4138worker1 # [ 55.134562] nix-daemon[934]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4139worker1 # [ 55.157494] nix-daemon[934]: reaped child process 968, status = succeeded4140lb # [ 63.450559] niks3-server[1093]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures4141worker1 # [ 55.218364] nix-grpc-daemon[946]: time=2026-09-20T12:21:49.326Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4142worker1 # [ 55.222479] nix-grpc-daemon[946]: time=2026-09-20T12:21:49.327Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4143lb # [ 63.541704] niks3-server[1093]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4144lb # [ 63.547404] niks3-server[1093]: 2026/09/20 12:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4145lb # [ 63.557610] niks3-server[1093]: 2026/09/20 12:21:48 INFO Signed narinfos id=1 count=14146worker1 # [ 55.282170] nix-grpc-daemon[946]: time=2026-09-20T12:21:49.391Z level=INFO msg="Uploading 1 narinfos"4147lb # [ 63.573674] niks3-server[1093]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4148lb # [ 63.581837] niks3-server[1093]: 2026/09/20 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4149lb # [ 63.596167] niks3-server[1093]: 2026/09/20 12:21:48 INFO Completed upload id=14150worker1 # [ 55.318129] nix-grpc-daemon[946]: time=2026-09-20T12:21:49.428Z level=INFO msg="Upload complete. (621ms)"4151lb # [ 63.600512] niks3-server[1093]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4152worker1 # [ 55.331602] nix-grpc-daemon[803]: ts=2026-09-20T12:21:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 paths=1 nar_bytes_in=5044153worker1 # [ 55.352464] nix-grpc-daemon[803]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204154worker1 # [ 55.364807] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4155worker1 # [ 55.368157] nix-daemon[934]: reaped child process 944, status = succeeded4156client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4157worker1 # [ 55.385508] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4158client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4159lb # [ 63.777155] envoy[851]: conn peer=[2001:db8:1::1]:54272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=32204160lb # [ 63.779765] envoy[851]: conn peer=[2001:db8:1::1]:45460 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=1414161lb # [ 63.782460] envoy[851]: conn peer=[2001:db8:1::1]:45476 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134162lb # [ 63.784758] envoy[851]: conn peer=[2001:db8:1::1]:45492 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=224163client # farm-a> LOG-farm-a-t14164worker1 # [ 56.227737] nix-daemon[934]: reaped child process 972, status = succeeded4165worker1 # [ 56.406380] nix-daemon[934]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4166worker1 # [ 56.428927] nix-daemon[934]: reaped child process 990, status = succeeded4167lb # [ 64.721534] niks3-server[1093]: 2026/09/20 12:21:49 INFO Received uploads request method=POST path=/api/pending_closures4168worker1 # [ 56.456525] nix-grpc-daemon[946]: time=2026-09-20T12:21:50.565Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4169worker1 # [ 56.458830] nix-grpc-daemon[946]: time=2026-09-20T12:21:50.565Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4170lb # [ 64.769580] niks3-server[1093]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4171lb # [ 64.783411] niks3-server[1093]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4172lb # [ 64.803366] niks3-server[1093]: 2026/09/20 12:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4173worker1 # [ 56.532064] nix-grpc-daemon[946]: time=2026-09-20T12:21:50.641Z level=INFO msg="Uploading 1 narinfos"4174lb # [ 64.818554] niks3-server[1093]: 2026/09/20 12:21:49 INFO Signed narinfos id=2 count=14175lb # [ 64.825512] niks3-server[1093]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4176lb # [ 64.838893] niks3-server[1093]: 2026/09/20 12:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4177lb # [ 64.848626] niks3-server[1093]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4178lb # [ 64.855361] niks3-server[1093]: 2026/09/20 12:21:50 INFO Completed upload id=24179worker1 # [ 56.576970] nix-grpc-daemon[946]: time=2026-09-20T12:21:50.687Z level=INFO msg="Upload complete. (378ms)"4180worker1 # [ 56.604868] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14181worker1 # [ 56.620609] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04182worker1 # [ 56.626208] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14183worker1 # [ 56.630388] nix-daemon[934]: reaped child process 970, status = succeeded4184worker1 # [ 56.657284] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204185worker1 # [ 56.677121] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204186worker1 # [ 56.695690] nix-grpc-daemon[946]: time=2026-09-20T12:21:50.805Z level=INFO msg="All 1 paths already cached"4187client # copying 1 paths to 'lb:50051'...4188client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4189worker1 # [ 56.708171] nix-grpc-daemon[803]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204190worker1 # [ 56.727857] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4191worker1 # [ 56.965629] nix-daemon[934]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4192worker1 # [ 56.989785] nix-daemon[934]: reaped child process 999, status = succeeded4193worker1 # [ 57.182918] nix-daemon[934]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4194worker1 # [ 57.204608] nix-daemon[934]: reaped child process 1004, status = succeeded4195lb # [ 65.496662] niks3-server[1093]: 2026/09/20 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures4196worker1 # [ 57.229495] nix-grpc-daemon[946]: time=2026-09-20T12:21:51.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4197worker1 # [ 57.231149] nix-grpc-daemon[946]: time=2026-09-20T12:21:51.339Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4198lb # [ 65.532077] niks3-server[1093]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4199lb # [ 65.545766] niks3-server[1093]: 2026/09/20 12:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4200worker1 # [ 57.273681] nix-grpc-daemon[946]: time=2026-09-20T12:21:51.383Z level=INFO msg="Uploading 1 narinfos"4201lb # [ 65.558728] niks3-server[1093]: 2026/09/20 12:21:50 INFO Signed narinfos id=3 count=14202lb # [ 65.567053] niks3-server[1093]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4203lb # [ 65.581689] niks3-server[1093]: 2026/09/20 12:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4204lb # [ 65.589113] niks3-server[1093]: 2026/09/20 12:21:50 INFO Completed upload id=34205worker1 # [ 57.309622] nix-grpc-daemon[946]: time=2026-09-20T12:21:51.419Z level=INFO msg="Upload complete. (562ms)"4206lb # [ 65.592695] niks3-server[1093]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4207worker1 # [ 57.320466] nix-grpc-daemon[803]: ts=2026-09-20T12:21:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=6244208worker1 # [ 57.336480] nix-grpc-daemon[803]: ts=2026-09-20T12:21:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204209worker1 # [ 57.345255] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4210worker1 # [ 57.351674] nix-daemon[934]: reaped child process 994, status = succeeded4211client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4212worker1 # [ 57.365389] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4213client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4214client # farm-b> LOG-farm-b-t14215worker1 # [ 58.798859] nix-daemon[934]: reaped child process 1008, status = succeeded4216worker1 # [ 58.981627] nix-daemon[934]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4217worker1 # [ 59.007828] nix-daemon[934]: reaped child process 1027, status = succeeded4218lb # [ 67.305378] niks3-server[1093]: 2026/09/20 12:21:52 INFO Received uploads request method=POST path=/api/pending_closures4219worker1 # [ 59.033773] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.142Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4220worker1 # [ 59.035384] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.142Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4221lb # [ 67.342870] niks3-server[1093]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4222lb # [ 67.351172] niks3-server[1093]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4223worker1 # [ 59.079700] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.189Z level=INFO msg="Uploading 1 narinfos"4224lb # [ 67.361007] niks3-server[1093]: 2026/09/20 12:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4225lb # [ 67.363905] niks3-server[1093]: 2026/09/20 12:21:52 INFO Signed narinfos id=4 count=14226lb # [ 67.372753] niks3-server[1093]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4227lb # [ 67.382050] niks3-server[1093]: 2026/09/20 12:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4228worker1 # [ 59.109155] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.219Z level=INFO msg="Upload complete. (328ms)"4229lb # [ 67.387939] niks3-server[1093]: 2026/09/20 12:21:52 INFO Completed upload id=44230lb # [ 67.391764] niks3-server[1093]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4231worker1 # [ 59.116976] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14232worker1 # [ 59.125505] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04233worker1 # [ 59.128290] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14234worker1 # [ 59.134986] nix-daemon[934]: reaped child process 1006, status = succeeded4235worker1 # [ 59.173958] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204236worker1 # [ 59.189037] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204237worker1 # [ 59.202754] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.313Z level=INFO msg="All 2 paths already cached"4238client # copying 1 paths to 'lb:50051'...4239worker1 # [ 59.213674] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204240client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4241worker1 # [ 59.228217] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4242worker1 # [ 59.429039] nix-daemon[934]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4243worker1 # [ 59.454575] nix-daemon[934]: reaped child process 1036, status = succeeded4244worker1 # [ 59.645710] nix-daemon[934]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)4245worker1 # [ 59.667508] nix-daemon[934]: reaped child process 1041, status = succeeded4246lb # [ 67.958199] niks3-server[1093]: 2026/09/20 12:21:53 INFO Received uploads request method=POST path=/api/pending_closures4247worker1 # [ 59.690942] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.800Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4248worker1 # [ 59.692802] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.800Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4249lb # [ 67.993111] niks3-server[1093]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4250lb # [ 68.007580] niks3-server[1093]: 2026/09/20 12:21:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4251worker1 # [ 59.736142] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.846Z level=INFO msg="Uploading 1 narinfos"4252lb # [ 68.019433] niks3-server[1093]: 2026/09/20 12:21:53 INFO Signed narinfos id=5 count=14253lb # [ 68.025627] niks3-server[1093]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4254lb # [ 68.033620] niks3-server[1093]: 2026/09/20 12:21:53 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4255worker1 # [ 59.762654] nix-grpc-daemon[946]: time=2026-09-20T12:21:53.872Z level=INFO msg="Upload complete. (525ms)"4256lb # [ 68.041490] niks3-server[1093]: 2026/09/20 12:21:53 INFO Completed upload id=54257lb # [ 68.043932] niks3-server[1093]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4258worker1 # [ 59.768283] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=7524259worker1 # [ 59.777414] nix-grpc-daemon[803]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204260worker1 # [ 59.783694] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4261worker1 # [ 59.787783] nix-daemon[934]: reaped child process 1031, status = succeeded4262client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4263worker1 # [ 59.800260] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4264client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4265client # farm-top> LOG-farm-top-t14266worker1 # [ 61.035326] nix-daemon[934]: reaped child process 1045, status = succeeded4267worker1 # [ 61.209525] nix-daemon[934]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)4268worker1 # [ 61.233116] nix-daemon[934]: reaped child process 1063, status = succeeded4269lb # [ 69.522432] niks3-server[1093]: 2026/09/20 12:21:54 INFO Received uploads request method=POST path=/api/pending_closures4270worker1 # [ 61.256760] nix-grpc-daemon[946]: time=2026-09-20T12:21:55.365Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4271worker1 # [ 61.259922] nix-grpc-daemon[946]: time=2026-09-20T12:21:55.365Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4272lb # [ 69.573652] niks3-server[1093]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4273lb # [ 69.585964] niks3-server[1093]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4274worker1 # [ 61.316991] nix-grpc-daemon[946]: time=2026-09-20T12:21:55.426Z level=INFO msg="Uploading 1 narinfos"4275lb # [ 69.598470] niks3-server[1093]: 2026/09/20 12:21:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4276lb # [ 69.604888] niks3-server[1093]: 2026/09/20 12:21:54 INFO Signed narinfos id=6 count=14277lb # [ 69.606852] niks3-server[1093]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4278lb # [ 69.617877] niks3-server[1093]: 2026/09/20 12:21:54 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4279lb # [ 69.622432] niks3-server[1093]: 2026/09/20 12:21:54 INFO Completed upload id=64280worker1 # [ 61.343684] nix-grpc-daemon[946]: time=2026-09-20T12:21:55.453Z level=INFO msg="Upload complete. (332ms)"4281lb # [ 69.625306] niks3-server[1093]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4282worker1 # [ 61.357316] nix-grpc-daemon[803]: ts=2026-09-20T12:21:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14283worker1 # [ 61.375188] nix-daemon[934]: reaped child process 1043, status = succeeded4284client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.49 seconds)4285client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4286client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.25 seconds)4287client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4288client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4289client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4290client # copying 3 paths...4291client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4292client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4293client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4294client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4295(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.20 seconds)4296subtest: repeat is answered Cached by the scheduler without building4297worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4298worker1 # finding garbage collector roots...4299worker1 # deleting specified paths...4300worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4301worker1 # deleting unused links...4302worker1 # note: hard linking is currently saving 0.0 KiB4303worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4304worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4305worker2 # finding garbage collector roots...4306worker2 # deleting specified paths...4307worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4308worker2 # deleting unused links...4309worker2 # note: hard linking is currently saving 0.0 KiB4310worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4311client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24312worker1 # [ 63.015314] nix-grpc-daemon[803]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518204313client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4314client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4315worker2 # [ 71.267569] nix-grpc-daemon[871]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084316worker1 # [ 63.058389] nix-grpc-daemon[803]: ts=2026-09-20T12:21:57Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14317client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.37 seconds)4318client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4319client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.25 seconds)4320lb: must succeed: curl -sf localhost:9901/clusters4321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4322lb: must succeed: curl -sf localhost:9901/clusters4323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4324worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4325worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4326(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.54 seconds)4327subtest: inputs already in the cache are not sent to the scheduler4328lb: must succeed: curl -sf localhost:9901/clusters4329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4330lb: must succeed: curl -sf localhost:9901/clusters4331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4332worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4333worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4334client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24335client # don't know how to build these paths:4336worker1 # [ 64.098965] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518204337client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4338worker1 # [ 64.105932] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204339worker1 # [ 64.115680] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.225Z level=INFO msg="All 2 paths already cached"4340worker1 # [ 64.136008] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04341worker1 # [ 64.143610] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14342worker1 # [ 64.181777] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204343worker1 # [ 64.198161] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204344worker1 # [ 64.214672] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.324Z level=INFO msg="All 2 paths already cached"4345client # copying 1 paths to 'lb:50051'...4346client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4347worker1 # [ 64.226719] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204348worker1 # [ 64.244570] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4349worker1 # [ 64.458876] nix-daemon[934]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4350worker1 # [ 64.485548] nix-daemon[934]: reaped child process 1095, status = succeeded4351worker1 # [ 64.667615] nix-daemon[934]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4352worker1 # [ 64.690487] nix-daemon[934]: reaped child process 1100, status = succeeded4353lb # [ 72.982223] niks3-server[1093]: 2026/09/20 12:21:58 INFO Received uploads request method=POST path=/api/pending_closures4354worker1 # [ 64.715366] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.825Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4355worker1 # [ 64.717452] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.825Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4356lb # [ 73.017536] niks3-server[1093]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4357lb # [ 73.036480] niks3-server[1093]: 2026/09/20 12:21:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4358worker1 # [ 64.765222] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.875Z level=INFO msg="Uploading 1 narinfos"4359lb # [ 73.049930] niks3-server[1093]: 2026/09/20 12:21:58 INFO Signed narinfos id=7 count=14360lb # [ 73.056720] niks3-server[1093]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4361lb # [ 73.068275] niks3-server[1093]: 2026/09/20 12:21:58 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4362lb # [ 73.077454] niks3-server[1093]: 2026/09/20 12:21:58 INFO Completed upload id=74363worker1 # [ 64.798771] nix-grpc-daemon[946]: time=2026-09-20T12:21:58.909Z level=INFO msg="Upload complete. (541ms)"4364lb # [ 73.081874] niks3-server[1093]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4365worker1 # [ 64.804453] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=7524366worker1 # [ 64.814348] nix-grpc-daemon[803]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204367worker1 # [ 64.826296] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4368worker1 # [ 64.831133] nix-daemon[934]: reaped child process 1090, status = succeeded4369client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4370client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4371worker1 # [ 64.846426] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4372lb # [ 73.779182] envoy[851]: conn peer=[2001:db8:1::1]:45498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9553 tx=3878 ms=72294373lb # [ 73.779505] envoy[851]: conn peer=[2001:db8:1::1]:37902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=684374client # farm-top> LOG-farm-top-t1b4375worker1 # [ 65.689875] nix-daemon[934]: reaped child process 1104, status = succeeded4376worker1 # [ 65.861207] nix-daemon[934]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4377worker1 # [ 65.884291] nix-daemon[934]: reaped child process 1122, status = succeeded4378lb # [ 74.174494] niks3-server[1093]: 2026/09/20 12:21:59 INFO Received uploads request method=POST path=/api/pending_closures4379worker1 # [ 65.908622] nix-grpc-daemon[946]: time=2026-09-20T12:22:00.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4380worker1 # [ 65.910336] nix-grpc-daemon[946]: time=2026-09-20T12:22:00.017Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4381lb # [ 74.234431] niks3-server[1093]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4382worker1 # [ 65.970776] nix-grpc-daemon[946]: time=2026-09-20T12:22:00.080Z level=INFO msg="Uploading 1 narinfos"4383lb # [ 74.249226] niks3-server[1093]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4384lb # [ 74.259354] niks3-server[1093]: 2026/09/20 12:21:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4385lb # [ 74.268264] niks3-server[1093]: 2026/09/20 12:21:59 INFO Signed narinfos id=8 count=14386lb # [ 74.270721] niks3-server[1093]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4387lb # [ 74.283236] niks3-server[1093]: 2026/09/20 12:21:59 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4388lb # [ 74.289375] niks3-server[1093]: 2026/09/20 12:21:59 INFO Completed upload id=84389worker1 # [ 66.010981] nix-grpc-daemon[946]: time=2026-09-20T12:22:00.121Z level=INFO msg="Upload complete. (352ms)"4390lb # [ 74.293163] niks3-server[1093]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4391worker1 # [ 66.025921] nix-grpc-daemon[803]: ts=2026-09-20T12:22:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14392worker1 # [ 66.047617] nix-daemon[934]: reaped child process 1102, status = succeeded4393client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.23 seconds)4394client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4395client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.26 seconds)4396lb: must succeed: curl -sf localhost:9901/clusters4397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4398lb: must succeed: curl -sf localhost:9901/clusters4399lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4400worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4401worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4402(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.92 seconds)4403subtest: two clients wanting the same drv build it once4404client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4405client # [ 75.075562] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4406client # Running as unit: dedup1.service; invocation ID: 2391166d53c84d85af6a66228a4d373f4407client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.07 seconds)4408client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4409client # [ 75.332648] nix[968]: don't know how to build these paths:4410worker1 # [ 66.868576] nix-grpc-daemon[803]: ts=2026-09-20T12:22:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518204411client # [ 75.334963] nix[968]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4412worker1 # [ 66.884160] nix-grpc-daemon[803]: ts=2026-09-20T12:22:00Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04413worker1 # [ 66.889669] nix-grpc-daemon[803]: ts=2026-09-20T12:22:00Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14414worker2 # [ 75.175400] nix-grpc-daemon[871]: ts=2026-09-20T12:22:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084415worker2 # [ 75.192149] nix-grpc-daemon[871]: ts=2026-09-20T12:22:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084416client # [ 75.437512] nix[968]: copying 1 paths to 'lb:50051'...4417client # [ 75.439666] nix[968]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4418worker2 # [ 75.210046] nix-grpc-daemon[871]: ts=2026-09-20T12:22:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600084419worker2 # [ 75.221779] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4420worker2 # [ 75.247532] nix-grpc-daemon[871]: ts=2026-09-20T12:22:00Z level=info event=niks3_push_started pid=9614421worker2 # [ 75.550169] nix-grpc-daemon[961]: time=2026-09-20T12:22:00.807Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4422worker2 # [ 75.772169] nix-daemon[938]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4423worker2 # [ 75.797585] nix-daemon[938]: reaped child process 977, status = succeeded4424client # [ 76.197528] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4425client # Running as unit: dedup2.service; invocation ID: 9cd6d1c18c64466b8b9eefee31496d9e4426client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4427worker2 # [ 76.004637] nix-daemon[938]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4428worker2 # [ 76.026335] nix-daemon[938]: reaped child process 982, status = succeeded4429lb # [ 76.089096] niks3-server[1093]: 2026/09/20 12:22:01 INFO Received uploads request method=POST path=/api/pending_closures4430worker2 # [ 76.055549] nix-grpc-daemon[961]: time=2026-09-20T12:22:01.314Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4431worker2 # [ 76.059743] nix-grpc-daemon[961]: time=2026-09-20T12:22:01.316Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4432lb # [ 76.139540] niks3-server[1093]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4433worker2 # [ 76.114971] nix-grpc-daemon[961]: time=2026-09-20T12:22:01.373Z level=INFO msg="Uploading 1 narinfos"4434lb # [ 76.154361] niks3-server[1093]: 2026/09/20 12:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4435lb # [ 76.165251] niks3-server[1093]: 2026/09/20 12:22:01 INFO Signed narinfos id=9 count=14436lb # [ 76.173075] niks3-server[1093]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4437lb # [ 76.187613] niks3-server[1093]: 2026/09/20 12:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4438lb # [ 76.194905] niks3-server[1093]: 2026/09/20 12:22:01 INFO Completed upload id=94439worker2 # [ 76.149024] nix-grpc-daemon[961]: time=2026-09-20T12:22:01.406Z level=INFO msg="Upload complete. (588ms)"4440worker2 # [ 76.150958] nix-grpc-daemon[871]: ts=2026-09-20T12:22:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_in=4324441lb # [ 76.200542] niks3-server[1093]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4442worker2 # [ 76.167380] nix-grpc-daemon[871]: ts=2026-09-20T12:22:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084443worker2 # [ 76.173602] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4444worker2 # [ 76.183292] nix-daemon[938]: reaped child process 959, status = succeeded4445worker2 # [ 76.201896] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4446client # [ 76.433828] nix[968]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4447client # [ 76.606456] nix[984]: don't know how to build these paths:4448client # [ 76.607465] nix[984]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4449worker1 # [ 68.143600] nix-grpc-daemon[803]: ts=2026-09-20T12:22:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518204450worker1 # [ 68.160277] nix-grpc-daemon[803]: ts=2026-09-20T12:22:02Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04451worker2 # [ 76.402386] nix-grpc-daemon[871]: ts=2026-09-20T12:22:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084452client # [ 76.636376] nix[984]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4453client: must succeed: sleep 24454client: (finished: must succeed: sleep 2, in 2.05 seconds)4455??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4456 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394457client: waiting for success: ! systemctl is-active dedup1 dedup24458??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4459 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394460worker2 # [ 79.866466] nix-daemon[938]: reaped child process 990, status = succeeded4461worker2 # [ 80.066644] nix-daemon[938]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4462worker2 # [ 80.092946] nix-daemon[938]: reaped child process 1018, status = succeeded4463lb # [ 80.155437] niks3-server[1093]: 2026/09/20 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures4464worker2 # [ 80.122471] nix-grpc-daemon[961]: time=2026-09-20T12:22:05.379Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4465worker2 # [ 80.124801] nix-grpc-daemon[961]: time=2026-09-20T12:22:05.379Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4466lb # [ 80.195554] niks3-server[1093]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4467lb # [ 80.202334] niks3-server[1093]: 2026/09/20 12:22:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4468worker2 # [ 80.159794] nix-grpc-daemon[961]: time=2026-09-20T12:22:05.418Z level=INFO msg="Uploading 1 narinfos"4469lb # [ 80.209762] niks3-server[1093]: 2026/09/20 12:22:05 INFO Signed narinfos id=10 count=14470lb # [ 80.211580] niks3-server[1093]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4471lb # [ 80.220523] niks3-server[1093]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4472lb # [ 80.225724] niks3-server[1093]: 2026/09/20 12:22:05 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4473lb # [ 80.233136] niks3-server[1093]: 2026/09/20 12:22:05 INFO Completed upload id=104474worker2 # [ 80.186713] nix-grpc-daemon[961]: time=2026-09-20T12:22:05.445Z level=INFO msg="Upload complete. (365ms)"4475lb # [ 80.235506] niks3-server[1093]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4476worker2 # [ 80.197044] nix-grpc-daemon[871]: ts=2026-09-20T12:22:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14477worker2 # [ 80.208475] nix-grpc-daemon[871]: ts=2026-09-20T12:22:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14478worker2 # [ 80.214707] nix-daemon[938]: reaped child process 988, status = succeeded4479client # [ 80.499898] systemd[1]: dedup2.service: Deactivated successfully.4480client # [ 80.503408] systemd[1]: dedup2.service: Consumed 239ms CPU time over 4.302s wall clock time, 33M memory peak, 7.3K incoming IP traffic, 6.4K outgoing IP traffic.4481client # [ 80.529402] systemd[1]: dedup1.service: Deactivated successfully.4482client # [ 80.532344] systemd[1]: dedup1.service: Consumed 181ms CPU time over 5.456s wall clock time, 38.9M memory peak, 8.5K incoming IP traffic, 9.2K outgoing IP traffic.4483client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4484client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4485client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4486worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4487worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4488worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4489worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4490worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4491worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4492worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4493worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4494(finished: subtest: two clients wanting the same drv build it once, in 6.36 seconds)4495subtest: build hook: nix-daemon with builders = grpc://lb4496client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4497client # [ 81.662188] systemd[1]: Started Nix Daemon.4498client # [ 81.888386] nix-daemon[1026]: accepted connection from pid 1023, user root (trusted)4499worker2 # [ 81.913917] nix-grpc-daemon[871]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600084500worker1 # [ 73.691830] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204501worker1 # [ 73.719138] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04502worker1 # [ 73.727191] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14503worker1 # [ 73.768551] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204504worker1 # [ 73.790259] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204505worker1 # [ 73.811235] nix-grpc-daemon[803]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204506worker1 # [ 73.836342] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4507worker1 # [ 74.104148] nix-daemon[934]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4508worker1 # [ 74.131246] nix-daemon[934]: reaped child process 1161, status = succeeded4509worker1 # [ 74.335659] nix-daemon[934]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4510worker1 # [ 74.357992] nix-daemon[934]: reaped child process 1166, status = succeeded4511lb # [ 82.653853] niks3-server[1093]: 2026/09/20 12:22:07 INFO Received uploads request method=POST path=/api/pending_closures4512worker1 # [ 74.387175] nix-grpc-daemon[946]: time=2026-09-20T12:22:08.496Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4513worker1 # [ 74.388935] nix-grpc-daemon[946]: time=2026-09-20T12:22:08.496Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4514lb # [ 82.697035] niks3-server[1093]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4515lb # [ 82.715410] niks3-server[1093]: 2026/09/20 12:22:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4516worker1 # [ 74.444509] nix-grpc-daemon[946]: time=2026-09-20T12:22:08.554Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 82.727367] niks3-server[1093]: 2026/09/20 12:22:07 INFO Signed narinfos id=11 count=14518lb # [ 82.733616] niks3-server[1093]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4519lb # [ 82.747281] niks3-server[1093]: 2026/09/20 12:22:07 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4520lb # [ 82.754952] niks3-server[1093]: 2026/09/20 12:22:07 INFO Completed upload id=114521worker1 # [ 74.475433] nix-grpc-daemon[946]: time=2026-09-20T12:22:08.585Z level=INFO msg="Upload complete. (612ms)"4522lb # [ 82.758096] niks3-server[1093]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4523worker1 # [ 74.484562] nix-grpc-daemon[803]: ts=2026-09-20T12:22:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=5124524worker1 # [ 74.505093] nix-grpc-daemon[803]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204525worker1 # [ 74.514781] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4526worker1 # [ 74.524482] nix-daemon[934]: reaped child process 1156, status = succeeded4527worker1 # [ 74.543534] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4528lb # [ 83.780481] envoy[851]: conn peer=[2001:db8:1::1]:37914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1728 ms=19404529lb # [ 83.786042] envoy[851]: conn peer=[2001:db8:1::1]:37920 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=38554530lb # [ 83.791381] envoy[851]: conn peer=[2001:db8:1::1]:37918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1374 ms=51544531worker1 # [ 75.804196] nix-daemon[934]: reaped child process 1170, status = succeeded4532worker1 # [ 76.008164] nix-daemon[934]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)4533worker1 # [ 76.028635] nix-daemon[934]: reaped child process 1188, status = succeeded4534lb # [ 84.327319] niks3-server[1093]: 2026/09/20 12:22:09 INFO Received uploads request method=POST path=/api/pending_closures4535worker1 # [ 76.064651] nix-grpc-daemon[946]: time=2026-09-20T12:22:10.171Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4536worker1 # [ 76.066256] nix-grpc-daemon[946]: time=2026-09-20T12:22:10.171Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4537lb # [ 84.395340] niks3-server[1093]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4538worker1 # [ 76.130739] nix-grpc-daemon[946]: time=2026-09-20T12:22:10.240Z level=INFO msg="Uploading 1 narinfos"4539lb # [ 84.408945] niks3-server[1093]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4540lb # [ 84.417875] niks3-server[1093]: 2026/09/20 12:22:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4541lb # [ 84.428621] niks3-server[1093]: 2026/09/20 12:22:09 INFO Signed narinfos id=12 count=14542lb # [ 84.433688] niks3-server[1093]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4543lb # [ 84.443321] niks3-server[1093]: 2026/09/20 12:22:09 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4544lb # [ 84.448857] niks3-server[1093]: 2026/09/20 12:22:09 INFO Completed upload id=124545worker1 # [ 76.170473] nix-grpc-daemon[946]: time=2026-09-20T12:22:10.280Z level=INFO msg="Upload complete. (412ms)"4546lb # [ 84.451530] niks3-server[1093]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4547worker1 # [ 76.188982] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14548worker1 # [ 76.211753] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:518204549worker2 # [ 84.445521] nix-grpc-daemon[871]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600084550worker1 # [ 76.217484] nix-daemon[934]: reaped child process 1168, status = succeeded4551worker2 # [ 84.467475] nix-grpc-daemon[871]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4552worker2 # [ 84.497574] nix-grpc-daemon[871]: ts=2026-09-20T12:22:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_out=1284553worker2 # [ 84.842179] nix-grpc-daemon[871]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600084554worker2 # [ 84.851685] nix-grpc-daemon[871]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084555worker2 # [ 84.860765] nix-grpc-daemon[961]: time=2026-09-20T12:22:10.117Z level=INFO msg="All 1 paths already cached"4556worker1 # [ 76.645694] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04557worker1 # [ 76.654881] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14558worker1 # [ 76.665965] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204559worker1 # [ 76.681708] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204560worker1 # [ 76.698618] nix-grpc-daemon[946]: time=2026-09-20T12:22:10.808Z level=INFO msg="All 1 paths already cached"4561worker1 # [ 76.710927] nix-grpc-daemon[803]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204562worker1 # [ 76.730394] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4563worker1 # [ 76.968219] nix-daemon[934]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4564worker1 # [ 76.995161] nix-daemon[934]: reaped child process 1197, status = succeeded4565worker1 # [ 77.174942] nix-daemon[934]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)4566worker1 # [ 77.199724] nix-daemon[934]: reaped child process 1202, status = succeeded4567lb # [ 85.486097] niks3-server[1093]: 2026/09/20 12:22:10 INFO Received uploads request method=POST path=/api/pending_closures4568worker1 # [ 77.213342] nix-grpc-daemon[946]: time=2026-09-20T12:22:11.322Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4569worker1 # [ 77.215142] nix-grpc-daemon[946]: time=2026-09-20T12:22:11.322Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4570lb # [ 85.508560] niks3-server[1093]: 2026/09/20 12:22:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4571lb # [ 85.514440] niks3-server[1093]: 2026/09/20 12:22:10 INFO Signed narinfos id=13 count=14572worker1 # [ 77.235812] nix-grpc-daemon[946]: time=2026-09-20T12:22:11.345Z level=INFO msg="Uploading 1 narinfos"4573lb # [ 85.531602] niks3-server[1093]: 2026/09/20 12:22:10 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4574lb # [ 85.574138] niks3-server[1093]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4575lb # [ 85.586396] niks3-server[1093]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4576lb # [ 85.590880] niks3-server[1093]: 2026/09/20 12:22:10 INFO Completed upload id=134577worker1 # [ 77.311473] nix-grpc-daemon[946]: time=2026-09-20T12:22:11.421Z level=INFO msg="Upload complete. (561ms)"4578lb # [ 85.594601] niks3-server[1093]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4579worker1 # [ 77.321706] nix-grpc-daemon[803]: ts=2026-09-20T12:22:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=6244580worker1 # [ 77.337748] nix-grpc-daemon[803]: ts=2026-09-20T12:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204581worker1 # [ 77.351993] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4582worker1 # [ 77.358642] nix-daemon[934]: reaped child process 1192, status = succeeded4583worker1 # [ 77.376084] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4584worker1 # [ 78.208172] nix-daemon[934]: reaped child process 1206, status = succeeded4585worker1 # [ 78.373403] nix-daemon[934]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4586worker1 # [ 78.396722] nix-daemon[934]: reaped child process 1224, status = succeeded4587lb # [ 86.693116] niks3-server[1093]: 2026/09/20 12:22:11 INFO Received uploads request method=POST path=/api/pending_closures4588worker1 # [ 78.478542] nix-grpc-daemon[946]: time=2026-09-20T12:22:12.587Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4589worker1 # [ 78.487896] nix-grpc-daemon[946]: time=2026-09-20T12:22:12.592Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4590lb # [ 86.807659] niks3-server[1093]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4591lb # [ 86.818861] niks3-server[1093]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4592lb # [ 86.828677] niks3-server[1093]: 2026/09/20 12:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4593lb # [ 86.835743] niks3-server[1093]: 2026/09/20 12:22:11 INFO Signed narinfos id=14 count=14594worker1 # [ 78.555256] nix-grpc-daemon[946]: time=2026-09-20T12:22:12.664Z level=INFO msg="Uploading 1 narinfos"4595lb # [ 86.845420] niks3-server[1093]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4596lb # [ 86.858842] niks3-server[1093]: 2026/09/20 12:22:12 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4597lb # [ 86.866561] niks3-server[1093]: 2026/09/20 12:22:12 INFO Completed upload id=144598worker1 # [ 78.587692] nix-grpc-daemon[946]: time=2026-09-20T12:22:12.697Z level=INFO msg="Upload complete. (400ms)"4599lb # [ 86.869394] niks3-server[1093]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4600worker1 # [ 78.601727] nix-grpc-daemon[803]: ts=2026-09-20T12:22:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14601worker1 # [ 78.627075] nix-daemon[934]: reaped child process 1204, status = succeeded4602worker2 # [ 86.881223] nix-grpc-daemon[871]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600084603worker2 # [ 86.896464] nix-grpc-daemon[871]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4604worker2 # [ 86.947036] nix-grpc-daemon[871]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600084605worker2 # [ 86.952004] nix-grpc-daemon[871]: ts=2026-09-20T12:22:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_out=1764606worker1 # [ 79.064368] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:518204607worker1 # [ 79.073634] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204608worker1 # [ 79.082317] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.190Z level=INFO msg="All 2 paths already cached"4609worker1 # [ 79.101581] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04610worker1 # [ 79.110372] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14611worker1 # [ 79.152612] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204612worker1 # [ 79.176458] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204613worker1 # [ 79.198887] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.309Z level=INFO msg="All 2 paths already cached"4614worker1 # [ 79.213613] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204615worker1 # [ 79.233843] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4616worker1 # [ 79.493167] nix-daemon[934]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)4617worker1 # [ 79.518168] nix-daemon[934]: reaped child process 1233, status = succeeded4618worker1 # [ 79.720356] nix-daemon[934]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)4619worker1 # [ 79.740912] nix-daemon[934]: reaped child process 1238, status = succeeded4620lb # [ 88.038915] niks3-server[1093]: 2026/09/20 12:22:13 INFO Received uploads request method=POST path=/api/pending_closures4621worker1 # [ 79.771560] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.881Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4622worker1 # [ 79.773362] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.881Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4623lb # [ 88.081154] niks3-server[1093]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4624worker1 # [ 79.818283] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.928Z level=INFO msg="Uploading 1 narinfos"4625lb # [ 88.099577] niks3-server[1093]: 2026/09/20 12:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4626lb # [ 88.107856] niks3-server[1093]: 2026/09/20 12:22:13 INFO Signed narinfos id=15 count=14627lb # [ 88.115767] niks3-server[1093]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4628lb # [ 88.131766] niks3-server[1093]: 2026/09/20 12:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4629lb # [ 88.137282] niks3-server[1093]: 2026/09/20 12:22:13 INFO Completed upload id=154630worker1 # [ 79.857978] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.968Z level=INFO msg="Upload complete. (604ms)"4631worker1 # [ 79.870911] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=7524632worker1 # [ 79.889267] nix-grpc-daemon[946]: time=2026-09-20T12:22:13.972Z level=WARN msg="Failed to register uploaded object" key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo error="server returned 404: object is not pending upload\n"4633worker1 # [ 79.894889] nix-grpc-daemon[803]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204634worker1 # [ 79.905576] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4635worker1 # [ 79.911183] nix-daemon[934]: reaped child process 1228, status = succeeded4636worker1 # [ 79.924854] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4637worker1 # [ 81.158081] nix-daemon[934]: reaped child process 1242, status = succeeded4638worker1 # [ 81.337540] nix-daemon[934]: accepted connection from pid 1257, user nix-grpc-daemon (trusted)4639worker1 # [ 81.360695] nix-daemon[934]: reaped child process 1260, status = succeeded4640lb # [ 89.658627] niks3-server[1093]: 2026/09/20 12:22:14 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 81.400598] nix-grpc-daemon[946]: time=2026-09-20T12:22:15.506Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4642worker1 # [ 81.402200] nix-grpc-daemon[946]: time=2026-09-20T12:22:15.506Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4643lb # [ 89.726802] niks3-server[1093]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4644worker1 # [ 81.460167] nix-grpc-daemon[946]: time=2026-09-20T12:22:15.569Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 89.740145] niks3-server[1093]: 2026/09/20 12:22:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4646lb # [ 89.750919] niks3-server[1093]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4647lb # [ 89.759586] niks3-server[1093]: 2026/09/20 12:22:14 INFO Signed narinfos id=16 count=14648lb # [ 89.763136] niks3-server[1093]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4649lb # [ 89.774301] niks3-server[1093]: 2026/09/20 12:22:14 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4650lb # [ 89.780230] niks3-server[1093]: 2026/09/20 12:22:14 INFO Completed upload id=164651worker1 # [ 81.500354] nix-grpc-daemon[946]: time=2026-09-20T12:22:15.610Z level=INFO msg="Upload complete. (359ms)"4652worker1 # [ 81.518646] nix-grpc-daemon[803]: ts=2026-09-20T12:22:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14653worker1 # [ 81.531946] nix-grpc-daemon[803]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:518204654worker1 # [ 81.542953] nix-daemon[934]: reaped child process 1240, status = succeeded4655worker1 # [ 81.548776] nix-grpc-daemon[803]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:518204656worker1 # [ 81.555816] nix-grpc-daemon[803]: ts=2026-09-20T12:22:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_out=2324657lb # [ 89.973074] niks3-server[1093]: 2026/09/20 12:22:15 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4658client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.78 seconds)4659client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4660client # [ 90.235854] nix-daemon[1026]: reaped child process 1028, status = succeeded4661client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.07 seconds)4662client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4663client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4664client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4665client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4666(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.95 seconds)4667subtest: an input only one worker has still reaches the builder4668client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4669client # this derivation will be built:4670client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4671client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4672client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.61 seconds)4673client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4674client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.24 seconds)4675worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4676worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.27 seconds)4677worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4678worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.05 seconds)4679client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24680client # [ 91.768824] nix-daemon[1026]: accepted connection from pid 1108, user root (trusted)4681client # this derivation will be built:4682client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4683worker1 # [ 83.545423] nix-grpc-daemon[803]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:518204684client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4685worker2 # [ 91.785642] nix-grpc-daemon[871]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084686client # copying 1 paths...4687client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4688worker1 # [ 83.577016] nix-grpc-daemon[803]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204689worker1 # [ 83.598095] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4690worker1 # [ 83.834906] nix-daemon[934]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)4691worker1 # [ 83.860967] nix-daemon[934]: reaped child process 1272, status = succeeded4692lb # [ 92.153293] niks3-server[1093]: 2026/09/20 12:22:17 INFO Received uploads request method=POST path=/api/pending_closures4693worker1 # [ 83.893884] nix-grpc-daemon[946]: time=2026-09-20T12:22:18.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4694worker1 # [ 83.895524] nix-grpc-daemon[946]: time=2026-09-20T12:22:18.003Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4695lb # [ 92.199535] niks3-server[1093]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4696lb # [ 92.216796] niks3-server[1093]: 2026/09/20 12:22:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4697worker1 # [ 83.944402] nix-grpc-daemon[946]: time=2026-09-20T12:22:18.054Z level=INFO msg="Uploading 1 narinfos"4698lb # [ 92.230085] niks3-server[1093]: 2026/09/20 12:22:17 INFO Signed narinfos id=17 count=14699lb # [ 92.236545] niks3-server[1093]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4700lb # [ 92.250707] niks3-server[1093]: 2026/09/20 12:22:17 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4701lb # [ 92.255356] niks3-server[1093]: 2026/09/20 12:22:17 INFO Completed upload id=174702worker1 # [ 83.976968] nix-grpc-daemon[946]: time=2026-09-20T12:22:18.087Z level=INFO msg="Upload complete. (359ms)"4703lb # [ 92.258776] niks3-server[1093]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4704worker1 # [ 83.988593] nix-grpc-daemon[803]: ts=2026-09-20T12:22:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_in=1204705worker1 # [ 84.010558] nix-daemon[934]: reaped child process 1267, status = succeeded4706worker1 # [ 84.014482] nix-grpc-daemon[803]: ts=2026-09-20T12:22:18Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04707worker1 # [ 84.018422] nix-grpc-daemon[803]: ts=2026-09-20T12:22:18Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14708worker2 # [ 92.292856] nix-grpc-daemon[871]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084709worker2 # [ 92.314271] nix-grpc-daemon[871]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084710client # copying 2 paths to 'lb:50051'...4711client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4712client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4713worker2 # [ 92.337487] nix-grpc-daemon[871]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600084714worker2 # [ 92.358920] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4715worker2 # [ 92.626269] nix-daemon[938]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4716worker2 # [ 92.650367] nix-daemon[938]: reaped child process 1039, status = succeeded4717worker2 # [ 92.858137] nix-daemon[938]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4718worker2 # [ 92.879660] nix-daemon[938]: reaped child process 1044, status = succeeded4719lb # [ 92.938943] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received uploads request method=POST path=/api/pending_closures4720lb # [ 92.953121] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received uploads request method=POST path=/api/pending_closures4721worker2 # [ 92.920617] nix-grpc-daemon[961]: time=2026-09-20T12:22:18.178Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4722worker2 # [ 92.922335] nix-grpc-daemon[961]: time=2026-09-20T12:22:18.178Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4723worker2 # [ 92.924742] nix-grpc-daemon[961]: time=2026-09-20T12:22:18.178Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4724lb # [ 93.009466] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4725lb # [ 93.030248] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4726lb # [ 93.051261] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4727worker2 # [ 93.012251] nix-grpc-daemon[961]: time=2026-09-20T12:22:18.270Z level=INFO msg="Uploading 3 narinfos"4728lb # [ 93.059719] niks3-server[1093]: 2026/09/20 12:22:18 INFO Signed narinfos id=18 count=14729lb # [ 93.064684] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4730lb # [ 93.066997] niks3-server[1093]: 2026/09/20 12:22:18 INFO Signed narinfos id=19 count=24731lb # [ 93.073766] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4732lb # [ 93.075904] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4733lb # [ 93.097088] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4734lb # [ 93.102210] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4735lb # [ 93.109706] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4736lb # [ 93.114466] niks3-server[1093]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4737lb # [ 93.117531] niks3-server[1093]: 2026/09/20 12:22:18 INFO Completed upload id=184738lb # [ 93.120264] niks3-server[1093]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4739lb # [ 93.124503] niks3-server[1093]: 2026/09/20 12:22:18 INFO Completed upload id=194740worker2 # [ 93.078856] nix-grpc-daemon[961]: time=2026-09-20T12:22:18.337Z level=INFO msg="Upload complete. (691ms)"4741worker2 # [ 93.100281] nix-grpc-daemon[871]: ts=2026-09-20T12:22:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=2 nar_bytes_in=9844742worker2 # [ 93.121116] nix-grpc-daemon[871]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084743worker2 # [ 93.136414] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4744worker2 # [ 93.145472] nix-daemon[938]: reaped child process 1034, status = succeeded4745worker2 # [ 93.171373] nix-grpc-daemon[871]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4746client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4747client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4748worker2 # [ 93.206523] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4749lb # [ 93.778124] envoy[851]: conn peer=[2001:db8:1::1]:57614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3436 tx=1679 ms=26484750lb # [ 93.783934] envoy[851]: conn peer=[2001:db8:1::1]:57628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1144751lb # [ 93.784329] envoy[851]: conn peer=[2001:db8:1::1]:57632 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3875 tx=1988 ms=21694752lb # [ 93.784589] envoy[851]: conn peer=[2001:db8:1::1]:57640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=694753lb # [ 93.784711] envoy[851]: conn peer=[2001:db8:1::1]:57650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=754754lb # [ 93.784806] envoy[851]: conn peer=[2001:db8:1::1]:57648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4325 tx=2108 ms=25564755worker2 # [ 94.641903] nix-daemon[938]: reaped child process 1049, status = succeeded4756worker2 # [ 94.826656] nix-daemon[938]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4757worker2 # [ 94.850806] nix-daemon[938]: reaped child process 1067, status = succeeded4758lb # [ 94.915195] niks3-server[1093]: 2026/09/20 12:22:20 INFO Received uploads request method=POST path=/api/pending_closures4759worker2 # [ 94.879978] nix-grpc-daemon[961]: time=2026-09-20T12:22:20.138Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4760lb # [ 94.957681] niks3-server[1093]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4761worker2 # [ 94.931584] nix-grpc-daemon[961]: time=2026-09-20T12:22:20.190Z level=INFO msg="Uploading 1 narinfos"4762lb # [ 94.970280] niks3-server[1093]: 2026/09/20 12:22:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4763lb # [ 94.982717] niks3-server[1093]: 2026/09/20 12:22:20 INFO Signed narinfos id=20 count=14764lb # [ 94.989428] niks3-server[1093]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4765lb # [ 95.000148] niks3-server[1093]: 2026/09/20 12:22:20 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4766worker2 # [ 94.960939] nix-grpc-daemon[961]: time=2026-09-20T12:22:20.220Z level=INFO msg="Upload complete. (335ms)"4767lb # [ 95.009933] niks3-server[1093]: 2026/09/20 12:22:20 INFO Completed upload id=204768lb # [ 95.014472] niks3-server[1093]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4769worker2 # [ 94.973489] nix-grpc-daemon[871]: ts=2026-09-20T12:22:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14770worker2 # [ 94.991332] nix-daemon[938]: reaped child process 1046, status = succeeded4771client # copying 1 paths...4772worker2 # [ 95.022474] nix-grpc-daemon[871]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600084773client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4774worker2 # [ 95.068094] nix-grpc-daemon[871]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600084775worker2 # [ 95.070813] nix-grpc-daemon[871]: ts=2026-09-20T12:22:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_out=1204776client # [ 95.408990] nix-daemon[1026]: reaped child process 1111, status = succeeded4777client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.93 seconds)4778client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24779client # [ 95.653621] nix-daemon[1026]: accepted connection from pid 1129, user root (trusted)4780client # this derivation will be built:4781client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4782worker1 # [ 87.437378] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:518204783client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4784worker1 # [ 87.444948] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204785worker1 # [ 87.575776] nix-grpc-daemon[946]: time=2026-09-20T12:22:21.684Z level=INFO msg="All 1 paths already cached"4786worker1 # [ 87.606930] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04787worker1 # [ 87.613476] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14788worker1 # [ 87.622139] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204789worker1 # [ 87.636397] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204790client # copying 1 paths to 'lb:50051'...4791client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4792worker1 # [ 87.666927] nix-grpc-daemon[803]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518204793worker1 # [ 87.687334] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4794worker1 # [ 87.713542] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)4795worker1 # [ 87.799954] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)4796worker1 # [ 87.971629] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 3/5)4797worker1 # [ 88.013097] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5)4798worker1 # [ 88.619468] nix-grpc-daemon[803]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4799worker1 # [ 88.632189] nix-grpc-daemon[803]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4800worker1 # [ 88.904215] nix-daemon[934]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)4801worker1 # [ 88.930329] nix-daemon[934]: reaped child process 1287, status = succeeded4802worker1 # [ 89.145787] nix-daemon[934]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)4803worker1 # [ 89.169367] nix-daemon[934]: reaped child process 1292, status = succeeded4804lb # [ 97.461299] niks3-server[1093]: 2026/09/20 12:22:22 INFO Received uploads request method=POST path=/api/pending_closures4805worker1 # [ 89.198223] nix-grpc-daemon[946]: time=2026-09-20T12:22:23.307Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4806worker1 # [ 89.199774] nix-grpc-daemon[946]: time=2026-09-20T12:22:23.307Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4807lb # [ 97.504589] niks3-server[1093]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4808lb # [ 97.520856] niks3-server[1093]: 2026/09/20 12:22:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4809worker1 # [ 89.248355] nix-grpc-daemon[946]: time=2026-09-20T12:22:23.358Z level=INFO msg="Uploading 1 narinfos"4810lb # [ 97.530696] niks3-server[1093]: 2026/09/20 12:22:22 INFO Signed narinfos id=21 count=14811lb # [ 97.537595] niks3-server[1093]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4812lb # [ 97.549521] niks3-server[1093]: 2026/09/20 12:22:22 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4813worker1 # [ 89.276041] nix-grpc-daemon[946]: time=2026-09-20T12:22:23.386Z level=INFO msg="Upload complete. (610ms)"4814lb # [ 97.557287] niks3-server[1093]: 2026/09/20 12:22:22 INFO Completed upload id=214815worker1 # [ 89.282620] nix-grpc-daemon[803]: ts=2026-09-20T12:22:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=1 paths=1 nar_bytes_in=5684816lb # [ 97.561705] niks3-server[1093]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4817worker1 # [ 89.288848] nix-grpc-daemon[803]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518204818worker1 # [ 89.300541] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4819worker1 # [ 89.305077] nix-daemon[934]: reaped child process 1278, status = succeeded4820client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4821worker1 # [ 89.315931] nix-daemon[934]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4822client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4823worker1 # [ 90.978530] nix-daemon[934]: reaped child process 1296, status = succeeded4824worker1 # [ 91.170608] nix-daemon[934]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)4825worker1 # [ 91.194371] nix-daemon[934]: reaped child process 1314, status = succeeded4826lb # [ 99.483928] niks3-server[1093]: 2026/09/20 12:22:24 INFO Received uploads request method=POST path=/api/pending_closures4827worker1 # [ 91.218755] nix-grpc-daemon[946]: time=2026-09-20T12:22:25.327Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4828lb # [ 99.541112] niks3-server[1093]: 2026/09/20 12:22:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4829worker1 # [ 91.270216] nix-grpc-daemon[946]: time=2026-09-20T12:22:25.380Z level=INFO msg="Uploading 1 narinfos"4830lb # [ 99.554919] niks3-server[1093]: 2026/09/20 12:22:24 INFO Signed narinfos id=22 count=14831lb # [ 99.563857] niks3-server[1093]: 2026/09/20 12:22:24 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4832lb # [ 99.573873] niks3-server[1093]: 2026/09/20 12:22:24 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4833lb # [ 99.581677] niks3-server[1093]: 2026/09/20 12:22:24 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4834lb # [ 99.591721] niks3-server[1093]: 2026/09/20 12:22:24 INFO Completed upload id=224835worker1 # [ 91.312350] nix-grpc-daemon[946]: time=2026-09-20T12:22:25.422Z level=INFO msg="Upload complete. (382ms)"4836lb # [ 99.595189] niks3-server[1093]: 2026/09/20 12:22:24 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4837worker1 # [ 91.329802] nix-grpc-daemon[803]: ts=2026-09-20T12:22:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=2 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14838worker1 # [ 91.351228] nix-daemon[934]: reaped child process 1294, status = succeeded4839client # copying 1 paths...4840worker2 # [ 99.606994] nix-grpc-daemon[871]: ts=2026-09-20T12:22:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600084841worker2 # [ 99.621310] nix-grpc-daemon[871]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4842client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4843worker2 # [ 99.678266] nix-grpc-daemon[871]: ts=2026-09-20T12:22:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600084844worker2 # [ 99.683955] nix-grpc-daemon[871]: ts=2026-09-20T12:22:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_out=1204845client # [ 100.046193] nix-daemon[1026]: reaped child process 1132, status = succeeded4846client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 4.63 seconds)4847client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24848client # [ 100.307865] nix-daemon[1026]: accepted connection from pid 1151, user root (trusted)4849client # this derivation will be built:4850client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4851worker2 # [ 100.311684] nix-grpc-daemon[871]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600084852client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4853worker1 # [ 92.084639] nix-grpc-daemon[803]: ts=2026-09-20T12:22:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518204854worker1 # [ 92.100048] nix-grpc-daemon[946]: time=2026-09-20T12:22:26.210Z level=INFO msg="All 1 paths already cached"4855worker1 # [ 92.122931] nix-grpc-daemon[803]: ts=2026-09-20T12:22:26Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04856worker1 # [ 92.130312] nix-grpc-daemon[803]: ts=2026-09-20T12:22:26Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14857worker2 # [ 100.367384] nix-grpc-daemon[871]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084858worker2 # [ 100.386576] nix-grpc-daemon[871]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600084859worker2 # [ 100.401120] nix-grpc-daemon[961]: time=2026-09-20T12:22:25.659Z level=INFO msg="All 1 paths already cached"4860client # copying 1 paths to 'lb:50051'...4861client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4862worker2 # [ 100.412241] nix-grpc-daemon[871]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600084863worker2 # [ 100.433344] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4864worker2 # [ 100.682982] nix-daemon[938]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4865worker2 # [ 100.704454] nix-daemon[938]: reaped child process 1075, status = succeeded4866worker2 # [ 100.911882] nix-daemon[938]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4867worker2 # [ 100.936425] nix-daemon[938]: reaped child process 1080, status = succeeded4868lb # [ 100.999310] niks3-server[1093]: 2026/09/20 12:22:26 INFO Received uploads request method=POST path=/api/pending_closures4869worker2 # [ 100.963078] nix-grpc-daemon[961]: time=2026-09-20T12:22:26.221Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4870worker2 # [ 100.964878] nix-grpc-daemon[961]: time=2026-09-20T12:22:26.221Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4871lb # [ 101.036635] niks3-server[1093]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4872lb # [ 101.056525] niks3-server[1093]: 2026/09/20 12:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4873worker2 # [ 101.019368] nix-grpc-daemon[961]: time=2026-09-20T12:22:26.277Z level=INFO msg="Uploading 1 narinfos"4874lb # [ 101.068347] niks3-server[1093]: 2026/09/20 12:22:26 INFO Signed narinfos id=23 count=14875lb # [ 101.075921] niks3-server[1093]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4876lb # [ 101.090679] niks3-server[1093]: 2026/09/20 12:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4877lb # [ 101.096981] niks3-server[1093]: 2026/09/20 12:22:26 INFO Completed upload id=234878worker2 # [ 101.050194] nix-grpc-daemon[961]: time=2026-09-20T12:22:26.309Z level=INFO msg="Upload complete. (595ms)"4879worker2 # [ 101.064894] nix-grpc-daemon[871]: ts=2026-09-20T12:22:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=0 paths=1 nar_bytes_in=5684880worker2 # [ 101.081875] nix-grpc-daemon[961]: time=2026-09-20T12:22:26.314Z level=WARN msg="Failed to register uploaded object" key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo error="server returned 404: object is not pending upload\n"4881worker2 # [ 101.094526] nix-grpc-daemon[871]: ts=2026-09-20T12:22:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600084882worker2 # [ 101.102011] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4883worker2 # [ 101.107987] nix-daemon[938]: reaped child process 1070, status = succeeded4884client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4885worker2 # [ 101.118121] nix-daemon[938]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4886client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4887worker2 # [ 102.178578] nix-daemon[938]: reaped child process 1084, status = succeeded4888worker2 # [ 102.385613] nix-daemon[938]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4889worker2 # [ 102.409086] nix-daemon[938]: reaped child process 1102, status = succeeded4890lb # [ 102.471384] niks3-server[1093]: 2026/09/20 12:22:27 INFO Received uploads request method=POST path=/api/pending_closures4891worker2 # [ 102.435820] nix-grpc-daemon[961]: time=2026-09-20T12:22:27.692Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4892worker2 # [ 102.473744] nix-grpc-daemon[961]: time=2026-09-20T12:22:27.732Z level=INFO msg="Uploading 1 narinfos"4893lb # [ 102.509042] niks3-server[1093]: 2026/09/20 12:22:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4894lb # [ 102.523648] niks3-server[1093]: 2026/09/20 12:22:27 INFO Signed narinfos id=24 count=14895lb # [ 102.532074] niks3-server[1093]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4896lb # [ 102.542097] niks3-server[1093]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4897lb # [ 102.553695] niks3-server[1093]: 2026/09/20 12:22:27 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4898lb # [ 102.562658] niks3-server[1093]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4899lb # [ 102.569231] niks3-server[1093]: 2026/09/20 12:22:27 INFO Completed upload id=244900worker2 # [ 102.523861] nix-grpc-daemon[961]: time=2026-09-20T12:22:27.782Z level=INFO msg="Upload complete. (391ms)"4901client # copying 1 paths...4902worker2 # [ 102.541554] nix-grpc-daemon[871]: ts=2026-09-20T12:22:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60008 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14903client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4904worker2 # [ 102.560677] nix-grpc-daemon[871]: ts=2026-09-20T12:22:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600084905worker2 # [ 102.567925] nix-daemon[938]: reaped child process 1082, status = succeeded4906worker1 # [ 94.359303] nix-grpc-daemon[803]: ts=2026-09-20T12:22:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:518204907worker1 # [ 94.382620] nix-grpc-daemon[803]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4908worker1 # [ 94.407930] nix-grpc-daemon[803]: ts=2026-09-20T12:22:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51820 duration_s=0 paths=1 nar_bytes_out=1204909client # [ 103.009114] nix-daemon[1026]: reaped child process 1154, status = succeeded4910client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.96 seconds)4911(finished: subtest: an input only one worker has still reaches the builder, in 12.68 seconds)4912subtest: upload whose reference the worker lacks is completed from the cache4913client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4914client # this derivation will be built:4915client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4916client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4917client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.54 seconds)4918client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4919client # this derivation will be built:4920client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4921lb # [ 103.779960] envoy[851]: conn peer=[2001:db8:1::1]:33668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4226 tx=1796 ms=33564922lb # [ 103.780463] envoy[851]: conn peer=[2001:db8:1::1]:33676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=744923lb # [ 103.780758] envoy[851]: conn peer=[2001:db8:1::1]:33686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=1935 ms=40784924lb # [ 103.780898] envoy[851]: conn peer=[2001:db8:1::1]:33702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=744925lb # [ 103.781060] envoy[851]: conn peer=[2001:db8:1::1]:33714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3698 tx=1806 ms=23944926lb # [ 103.781183] envoy[851]: conn peer=[2001:db8:1::1]:40728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1204927client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4928client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.52 seconds)4929client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4930client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.24 seconds)4931worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4932worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.26 seconds)4933worker2 # [ 104.447351] systemd[1]: Stopping Nix worker-protocol over gRPC...4934worker2 # [ 104.555418] nix-grpc-daemon[871]: ts=2026-09-20T12:22:29Z level=info event=signal_exit4935worker1 # [ 97.777217] nix-grpc-daemon[803]: ts=2026-09-20T12:22:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518064936worker2 # [ 106.014558] nix-grpc-daemon[871]: ts=2026-09-20T12:22:31Z level=info event=scheduler_restarting4937worker1 # [ 97.782271] nix-grpc-daemon[803]: ts=2026-09-20T12:22:31Z level=info event=worker_gone addr=192.168.1.4:500514938worker2 # [ 106.038067] nix-grpc-daemon[871]: ts=2026-09-20T12:22:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4939worker2 # [ 106.075672] nix-daemon[938]: reaped child process 940, status = succeeded4940worker2 # [ 106.900721] nix-grpc-daemon[871]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4941worker2 # [ 106.906125] nix-grpc-daemon[871]: E0000 00:00:1789906952.159456 914 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4942worker2 # [ 106.946685] nix-grpc-daemon[871]: E0000 00:00:1789906952.205626 914 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4943worker2 # [ 107.233216] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4944worker2 # [ 107.241184] systemd[1]: Stopped Nix worker-protocol over gRPC.4945worker2 # [ 107.243573] systemd[1]: nix-grpc-daemon.service: Consumed 2.432s CPU time over 1min 13.239s wall clock time, 48.2M memory peak, 57.4K incoming IP traffic, 160.7K outgoing IP traffic.4946worker2 # [ 107.263405] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4947worker2 # [ 107.269378] systemd[1]: Closed Nix worker-protocol over gRPC.4948worker2 # [ 107.271937] systemd[1]: nix-grpc-daemon.socket: Consumed 933us CPU time over 1min 23.380s wall clock time, 256K memory peak, 66.6K incoming IP traffic, 59.6K outgoing IP traffic.4949lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4950lb: must succeed: curl -sf localhost:9901/clusters4951lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4952lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.12 seconds)4953client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24954worker1 # [ 99.426744] nix-grpc-daemon[803]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518204955client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4956worker1 # [ 99.435676] nix-grpc-daemon[803]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:518204957client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.31 seconds)4958worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4959worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.05 seconds)4960worker2 # [ 107.838854] systemd[1]: Listening on Nix worker-protocol over gRPC.4961worker1 # [ 99.673588] systemd[1]: Stopping Nix worker-protocol over gRPC...4962worker2 # [ 108.222779] systemd[1]: Starting scheduler-token.service...4963worker2 # [ 108.353837] systemd[1]: scheduler-token.service: Deactivated successfully.4964worker2 # [ 108.361123] systemd[1]: Finished scheduler-token.service.4965worker2 # [ 108.362770] systemd[1]: scheduler-token.service: Consumed 64ms CPU time over 135ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4966worker2 # [ 108.374048] systemd[1]: Starting Nix worker-protocol over gRPC...4967worker2 # [ 108.592783] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4968worker2 # [ 108.596157] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4969worker2 # [ 108.600857] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14970worker2 # [ 108.611149] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4971worker2 # [ 108.624467] systemd[1]: Started Nix worker-protocol over gRPC.4972worker2 # [ 108.646069] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:33Z level=info event=scheduler_connected addr=lb:500514973worker1 # [ 100.418363] nix-grpc-daemon[803]: ts=2026-09-20T12:22:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:518064974worker1 # [ 100.426363] nix-grpc-daemon[803]: ts=2026-09-20T12:22:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04975worker1 # [ 100.597367] nix-grpc-daemon[803]: ts=2026-09-20T12:22:34Z level=info event=signal_exit4976worker1 # [ 101.086607] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_yield4977worker2 # [ 109.323367] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:34Z level=info event=scheduler_restarting addr=lb:500514978worker1 # [ 101.093941] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518064979lb # [ 109.377500] niks3-server[1093]: 2026/09/20 12:22:34 INFO lead: released remote=192.168.1.3:441084980worker1 # [ 101.100177] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=worker_gone addr=192.168.1.4:500514981worker1 # [ 101.106348] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_restarting addr=lb:500514982worker1 # [ 101.110570] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518064983worker1 # [ 101.114998] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=worker_gone addr=192.168.1.3:500514984worker2 # [ 109.525341] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:34Z level=info event=scheduler_connected addr=lb:500514985worker2 # [ 109.531002] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4986worker1 # [ 101.301599] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500514987worker1 # [ 101.307248] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4988worker2 # [ 109.732359] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:34Z level=info event=scheduler_connected addr=lb:500514989worker2 # [ 109.734634] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4990worker1 # [ 101.510373] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500514991worker1 # [ 101.514771] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker2 # [ 109.936268] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500514993worker2 # [ 109.941754] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4994worker1 # [ 101.710788] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500514995worker1 # [ 101.716571] nix-grpc-daemon[803]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4996worker2 # [ 110.143278] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500514997worker1 # [ 101.916853] nix-grpc-daemon[803]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500514998worker1 # [ 101.919436] nix-grpc-daemon[803]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker2 # [ 110.149230] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5000worker1 # [ 102.091716] nix-grpc-daemon[803]: ts=2026-09-20T12:22:36Z level=info event=scheduler_restarting5001worker2 # [ 110.350477] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500515002worker2 # [ 110.356288] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker1 # [ 102.146732] nix-grpc-daemon[803]: ts=2026-09-20T12:22:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5004worker1 # [ 102.163944] nix-daemon[934]: reaped child process 936, status = succeeded5005worker2 # [ 110.556951] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 110.559141] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker2 # [ 110.762616] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500515008worker2 # [ 110.767475] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5009worker2 # [ 110.964748] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500515010worker2 # [ 110.971261] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker1 # [ 102.913897] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5012worker1 # [ 102.921247] systemd[1]: Stopped Nix worker-protocol over gRPC.5013worker1 # [ 102.923671] systemd[1]: nix-grpc-daemon.service: Consumed 4.841s CPU time over 1min 17.458s wall clock time, 63.5M memory peak, 105.6K incoming IP traffic, 201.6K outgoing IP traffic.5014worker1 # [ 102.941599] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5015worker2 # [ 111.172845] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 111.177125] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5017worker1 # [ 102.947813] systemd[1]: Closed Nix worker-protocol over gRPC.5018worker1 # [ 102.953729] systemd[1]: nix-grpc-daemon.socket: Consumed 981us CPU time over 1min 20.476s wall clock time, 256K memory peak, 103.7K incoming IP traffic, 80.7K outgoing IP traffic.5019lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5020lb: must succeed: curl -sf localhost:9901/clusters5021lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5022lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.11 seconds)5023client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25024worker2 # [ 111.378449] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500515025worker2 # [ 111.382114] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker2 # [ 111.584172] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_connected addr=lb:500515027worker2 # [ 111.590036] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5028worker2 # [ 111.639787] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396065029worker2 # [ 111.661590] nix-daemon[938]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5030worker2 # [ 111.691710] nix-grpc-daemon[1126]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 1/5)5031worker2 # [ 111.741673] nix-grpc-daemon[1126]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5)5032worker2 # [ 111.793463] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_connected addr=lb:500515033worker2 # [ 111.799067] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5034worker2 # [ 111.835260] nix-grpc-daemon[1126]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 242 ms (attempt 3/5)5035worker2 # [ 111.999881] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_connected addr=lb:500515036worker2 # [ 112.003214] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5037worker2 # [ 112.085827] nix-grpc-daemon[1126]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 551 ms (attempt 4/5)5038worker2 # [ 112.204832] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_connected addr=lb:500515039worker2 # [ 112.211508] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5040worker2 # [ 112.413466] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_connected addr=lb:500515041worker2 # [ 112.419772] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5042worker2 # [ 112.620820] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_connected addr=lb:500515043worker2 # [ 112.624800] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5044worker2 # [ 112.645745] nix-grpc-daemon[1126]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5045client # copying 2 paths...5046client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5047client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5048worker2 # [ 112.688353] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396065049worker2 # [ 112.708756] nix-daemon[938]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5050worker2 # [ 112.745730] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=niks3_push_started pid=11825051worker2 # [ 112.836211] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_connected addr=lb:500515052worker2 # [ 112.837939] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5053worker2 # [ 112.999727] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.258Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5054worker2 # [ 113.031372] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_connected addr=lb:500515055worker2 # [ 113.033951] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5056worker2 # [ 113.212688] nix-daemon[938]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5057worker2 # [ 113.235940] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_connected addr=lb:500515058worker2 # [ 113.238365] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5059worker2 # [ 113.240664] nix-daemon[938]: reaped child process 1198, status = succeeded5060lb # [ 113.307867] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures5061lb # [ 113.320249] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures5062worker2 # [ 113.287537] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.543Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5063worker2 # [ 113.290687] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.544Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5064worker2 # [ 113.294218] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.544Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5065lb # [ 113.382431] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5066lb # [ 113.419138] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5067lb # [ 113.444375] niks3-server[1093]: 2026/09/20 12:22:38 INFO Signed narinfos id=25 count=15068worker2 # [ 113.404124] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.661Z level=INFO msg="Uploading 3 narinfos"5069lb # [ 113.448953] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5070lb # [ 113.453697] niks3-server[1093]: 2026/09/20 12:22:38 INFO Signed narinfos id=26 count=25071worker2 # [ 113.441817] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_connected addr=lb:500515072lb # [ 113.492469] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5073worker2 # [ 113.445627] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5074lb # [ 113.503907] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5075lb # [ 113.509209] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5076lb # [ 113.511652] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5077lb # [ 113.515554] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5078lb # [ 113.519333] niks3-server[1093]: 2026/09/20 12:22:38 INFO Completed upload id=255079lb # [ 113.522835] niks3-server[1093]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5080lb # [ 113.526102] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5081lb # [ 113.529831] niks3-server[1093]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5082lb # [ 113.532675] niks3-server[1093]: 2026/09/20 12:22:38 INFO Completed upload id=265083worker2 # [ 113.487255] nix-grpc-daemon[1182]: time=2026-09-20T12:22:38.746Z level=INFO msg="Upload complete. (482ms)"5084worker2 # [ 113.500674] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39606 duration_s=0 paths=2 nar_bytes_in=3125085worker2 # [ 113.526551] nix-daemon[938]: reaped child process 1180, status = succeeded5086client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.24 seconds)5087worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5088worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.05 seconds)5089lb # [ 113.675745] niks3-server[1093]: 2026/09/20 12:22:38 INFO lead: acquired remote=192.168.1.4:407065090worker2 # [ 113.635020] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_take_over5091worker2 # [ 113.645637] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_connected addr=lb:500515092worker2 # [ 113.651928] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5093worker1 # [ 105.479040] systemd[1]: Listening on Nix worker-protocol over gRPC.5094worker1 # [ 105.488845] systemd[1]: Starting Nix worker-protocol over gRPC...5095lb # [ 113.781935] envoy[851]: conn peer=[2001:db8:1::4]:35634 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=74755 tx=3601 ms=705625096lb # [ 113.787846] envoy[851]: conn peer=[2001:db8:1::1]:40732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=315097lb # [ 113.792898] envoy[851]: conn peer=[2001:db8:1::3]:52114 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16227 tx=4357 ms=755155098lb # [ 113.798094] envoy[851]: conn peer=[2001:db8:1::1]:39812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=304 ms=18825099worker2 # [ 113.855090] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=scheduler_connected addr=lb:500515100worker2 # [ 113.883690] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=oidc_jwks provider=mock keys=15101worker2 # [ 113.889608] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:396165102worker2 # [ 113.895640] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05103worker1 # [ 105.708859] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5104worker1 # [ 105.711022] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5105worker1 # [ 105.715733] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15106worker1 # [ 105.725299] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5107worker1 # [ 105.727109] systemd[1]: Started Nix worker-protocol over gRPC.5108lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5109lb: must succeed: curl -sf localhost:9901/clusters5110worker1 # [ 105.762659] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:39Z level=info event=scheduler_connected addr=lb:500515111worker2 # [ 113.998545] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:396165112worker2 # [ 114.004790] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)5114lb: must succeed: curl -sf localhost:9901/clusters5115lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5116lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.21 seconds)5117(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.40 seconds)5118subtest: scheduler down: the other node takes the lock and keeps it5119lb: must succeed: curl -sf localhost:9901/clusters5120lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5121lb: must succeed: curl -sf localhost:9901/clusters5122lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5123lb: must succeed: curl -sf localhost:9901/clusters5124lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5125lb: must succeed: curl -sf localhost:9901/clusters5126lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5127worker2 # [ 115.535899] systemd[1]: Stopping Nix worker-protocol over gRPC...5128worker2 # [ 115.617480] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:40Z level=info event=signal_exit5129worker2 # [ 115.832459] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_yield5130lb # [ 115.893946] niks3-server[1093]: 2026/09/20 12:22:41 INFO lead: released remote=192.168.1.4:407065131worker1 # [ 107.618327] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:41Z level=info event=scheduler_restarting addr=lb:500515132worker2 # [ 115.854976] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:396165133worker2 # [ 115.863742] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=worker_gone addr=192.168.1.3:500515134worker2 # [ 115.868566] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_restarting addr=lb:500515135worker2 # [ 115.872842] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:396165136worker2 # [ 115.877183] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=worker_gone addr=192.168.1.4:500515137worker1 # [ 107.818716] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515138worker1 # [ 107.824899] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5139worker2 # [ 116.063425] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515140worker2 # [ 116.069570] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5141worker1 # [ 108.025989] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_connected addr=lb:500515142worker1 # [ 108.031462] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5143worker2 # [ 116.270859] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515144worker2 # [ 116.276831] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5145worker1 # [ 108.232557] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_connected addr=lb:500515146worker1 # [ 108.238009] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5147worker2 # [ 116.478126] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515148worker2 # [ 116.483833] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5149worker1 # [ 108.439045] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_connected addr=lb:500515150worker1 # [ 108.445095] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5151worker2 # [ 116.685098] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515152worker2 # [ 116.687688] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5153worker2 # [ 116.839215] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:42Z level=info event=scheduler_restarting5154worker2 # [ 116.873525] nix-daemon[938]: reaped child process 1175, status = succeeded5155worker1 # [ 108.648364] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_connected addr=lb:500515156worker1 # [ 108.652629] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5157worker2 # [ 116.906306] nix-grpc-daemon[1126]: ts=2026-09-20T12:22:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5158worker1 # [ 108.849183] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_connected addr=lb:500515159worker1 # [ 108.855204] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5160worker1 # [ 109.055763] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_connected addr=lb:500515161worker1 # [ 109.061456] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5162worker1 # [ 109.262744] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_connected addr=lb:500515163worker1 # [ 109.268580] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5164worker2 # [ 117.678275] nix-grpc-daemon[1126]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5165worker1 # [ 109.469383] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_connected addr=lb:500515166worker1 # [ 109.475118] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5167worker1 # [ 109.678233] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_connected addr=lb:500515168worker1 # [ 109.682688] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5169worker1 # [ 109.879276] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_connected addr=lb:500515170worker1 # [ 109.885133] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5171worker2 # [ 117.683128] nix-grpc-daemon[1126]: E0000 00:00:1789906962.937028 1169 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5172worker2 # [ 117.683407] nix-grpc-daemon[1126]: E0000 00:00:1789906962.937337 1169 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5173worker1 # [ 110.086138] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_connected addr=lb:500515174worker1 # [ 110.091794] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5175worker1 # [ 110.292979] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_connected addr=lb:500515176worker1 # [ 110.298549] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5177worker2 # [ 118.647746] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5178worker2 # [ 118.656668] systemd[1]: Stopped Nix worker-protocol over gRPC.5179worker2 # [ 118.659325] systemd[1]: nix-grpc-daemon.service: Consumed 612ms CPU time over 10.272s wall clock time, 30.8M memory peak, 24.5K incoming IP traffic, 49.1K outgoing IP traffic.5180worker2 # [ 118.679222] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5181worker2 # [ 118.682188] systemd[1]: Closed Nix worker-protocol over gRPC.5182worker2 # [ 118.686494] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 10.852s wall clock time, 256K memory peak, 20.4K incoming IP traffic, 15.3K outgoing IP traffic.5183worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5184worker1 # [ 110.499906] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_connected addr=lb:500515185worker1 # [ 110.508705] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5186worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.11 seconds)5187worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5188worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5189worker1 # [ 110.708553] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_connected addr=lb:500515190worker1 # [ 110.712734] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5191lb # [ 119.020766] niks3-server[1093]: 2026/09/20 12:22:44 INFO lead: acquired remote=192.168.1.3:421125192worker1 # [ 110.741533] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:44Z level=info event=scheduler_take_over5193worker1 # [ 110.909285] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:45Z level=info event=scheduler_connected addr=lb:500515194worker1 # [ 110.914797] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5195worker1 # [ 111.115347] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:45Z level=info event=scheduler_connected addr=lb:500515196worker1 # [ 111.124784] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:345205197worker1 # [ 111.129813] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05198worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5199worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5200client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25201worker1 # [ 111.971255] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345225202worker1 # [ 111.987485] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5203client # don't know how to build these paths:5204client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5205worker1 # [ 112.014820] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345225206worker1 # [ 112.024248] nix-grpc-daemon[1340]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5207worker1 # [ 112.061654] nix-grpc-daemon[1340]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5)5208worker1 # [ 112.206138] nix-grpc-daemon[1340]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 330 ms (attempt 3/5)5209worker1 # [ 112.544370] nix-grpc-daemon[1340]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 246 ms (attempt 4/5)5210worker1 # [ 112.791378] nix-grpc-daemon[1340]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5211worker1 # [ 112.827011] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:46Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05212worker1 # [ 112.834881] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:46Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15213worker1 # [ 112.875120] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225214worker1 # [ 112.892768] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345225215client # copying 1 paths to 'lb:50051'...5216client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5217worker1 # [ 112.907079] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345225218worker1 # [ 112.923532] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5219worker1 # [ 112.952422] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:47Z level=info event=niks3_push_started pid=14105220worker1 # [ 113.187868] nix-grpc-daemon[1410]: time=2026-09-20T12:22:47.297Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5221worker1 # [ 113.376756] nix-daemon[934]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5222worker1 # [ 113.397839] nix-daemon[934]: reaped child process 1426, status = succeeded5223worker1 # [ 113.581351] nix-daemon[934]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5224worker1 # [ 113.601730] nix-daemon[934]: reaped child process 1431, status = succeeded5225lb # [ 121.892726] niks3-server[1093]: 2026/09/20 12:22:47 INFO Received uploads request method=POST path=/api/pending_closures5226worker1 # [ 113.626400] nix-grpc-daemon[1410]: time=2026-09-20T12:22:47.735Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5227worker1 # [ 113.627966] nix-grpc-daemon[1410]: time=2026-09-20T12:22:47.735Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5228lb # [ 121.926955] niks3-server[1093]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5229worker1 # [ 113.657153] nix-grpc-daemon[1410]: time=2026-09-20T12:22:47.767Z level=INFO msg="Uploading 1 narinfos"5230lb # [ 121.935463] niks3-server[1093]: 2026/09/20 12:22:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5231lb # [ 121.942767] niks3-server[1093]: 2026/09/20 12:22:47 INFO Signed narinfos id=27 count=15232lb # [ 121.946847] niks3-server[1093]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5233lb # [ 121.956182] niks3-server[1093]: 2026/09/20 12:22:47 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5234worker1 # [ 113.682146] nix-grpc-daemon[1410]: time=2026-09-20T12:22:47.792Z level=INFO msg="Upload complete. (489ms)"5235lb # [ 121.962291] niks3-server[1093]: 2026/09/20 12:22:47 INFO Completed upload id=275236lb # [ 121.966437] niks3-server[1093]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5237worker1 # [ 113.688624] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=0 paths=1 nar_bytes_in=5205238worker1 # [ 113.697972] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225239worker1 # [ 113.710185] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5240worker1 # [ 113.714832] nix-daemon[934]: reaped child process 1408, status = succeeded5241client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5242worker1 # [ 113.726382] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5243client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5244client # farm-a> LOG-farm-a-on-standby5245worker1 # [ 115.352986] nix-daemon[934]: reaped child process 1435, status = succeeded5246lb # [ 123.779755] envoy[851]: conn peer=[2001:db8:1::4]:56976 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20956 tx=1101 ms=82605247worker1 # [ 115.528247] nix-daemon[934]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5248worker1 # [ 115.552809] nix-daemon[934]: reaped child process 1453, status = succeeded5249lb # [ 123.843420] niks3-server[1093]: 2026/09/20 12:22:48 INFO Received uploads request method=POST path=/api/pending_closures5250worker1 # [ 115.580564] nix-grpc-daemon[1410]: time=2026-09-20T12:22:49.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5251worker1 # [ 115.582161] nix-grpc-daemon[1410]: time=2026-09-20T12:22:49.689Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5252lb # [ 123.910378] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5253lb # [ 123.921130] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5254lb # [ 123.930185] niks3-server[1093]: 2026/09/20 12:22:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5255lb # [ 123.935709] niks3-server[1093]: 2026/09/20 12:22:49 INFO Signed narinfos id=28 count=15256worker1 # [ 115.658230] nix-grpc-daemon[1410]: time=2026-09-20T12:22:49.766Z level=INFO msg="Uploading 1 narinfos"5257lb # [ 123.940638] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5258lb # [ 123.952729] niks3-server[1093]: 2026/09/20 12:22:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5259lb # [ 123.960369] niks3-server[1093]: 2026/09/20 12:22:49 INFO Completed upload id=285260worker1 # [ 115.681404] nix-grpc-daemon[1410]: time=2026-09-20T12:22:49.791Z level=INFO msg="Upload complete. (340ms)"5261lb # [ 123.963684] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5262worker1 # [ 115.708362] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15263worker1 # [ 115.723181] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05264worker1 # [ 115.726506] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15265worker1 # [ 115.730407] nix-daemon[934]: reaped child process 1433, status = succeeded5266worker1 # [ 115.755164] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225267worker1 # [ 115.769000] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345225268worker1 # [ 115.783508] nix-grpc-daemon[1410]: time=2026-09-20T12:22:49.893Z level=INFO msg="All 1 paths already cached"5269client # copying 1 paths to 'lb:50051'...5270client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5271worker1 # [ 115.794396] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345225272worker1 # [ 115.808196] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5273worker1 # [ 116.009619] nix-daemon[934]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5274worker1 # [ 116.032675] nix-daemon[934]: reaped child process 1462, status = succeeded5275worker1 # [ 116.218428] nix-daemon[934]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5276worker1 # [ 116.239131] nix-daemon[934]: reaped child process 1467, status = succeeded5277lb # [ 124.529894] niks3-server[1093]: 2026/09/20 12:22:49 INFO Received uploads request method=POST path=/api/pending_closures5278worker1 # [ 116.261266] nix-grpc-daemon[1410]: time=2026-09-20T12:22:50.371Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5279worker1 # [ 116.262849] nix-grpc-daemon[1410]: time=2026-09-20T12:22:50.371Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5280lb # [ 124.558970] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5281worker1 # [ 116.296242] nix-grpc-daemon[1410]: time=2026-09-20T12:22:50.406Z level=INFO msg="Uploading 1 narinfos"5282lb # [ 124.574904] niks3-server[1093]: 2026/09/20 12:22:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5283lb # [ 124.587957] niks3-server[1093]: 2026/09/20 12:22:49 INFO Signed narinfos id=29 count=15284lb # [ 124.591431] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5285lb # [ 124.608412] niks3-server[1093]: 2026/09/20 12:22:49 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5286worker1 # [ 116.334857] nix-grpc-daemon[1410]: time=2026-09-20T12:22:50.445Z level=INFO msg="Upload complete. (515ms)"5287lb # [ 124.614033] niks3-server[1093]: 2026/09/20 12:22:49 INFO Completed upload id=295288lb # [ 124.620070] niks3-server[1093]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5289worker1 # [ 116.344530] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=0 paths=1 nar_bytes_in=6405290worker1 # [ 116.353096] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225291worker1 # [ 116.366966] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5292worker1 # [ 116.373049] nix-daemon[934]: reaped child process 1457, status = succeeded5293client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5294client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5295worker1 # [ 116.388622] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5296client # farm-b> LOG-farm-b-on-standby5297worker1 # [ 117.631074] nix-daemon[934]: reaped child process 1471, status = succeeded5298worker1 # [ 117.811977] nix-daemon[934]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5299worker1 # [ 117.833385] nix-daemon[934]: reaped child process 1489, status = succeeded5300lb # [ 126.125658] niks3-server[1093]: 2026/09/20 12:22:51 INFO Received uploads request method=POST path=/api/pending_closures5301worker1 # [ 117.856963] nix-grpc-daemon[1410]: time=2026-09-20T12:22:51.966Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5302worker1 # [ 117.858537] nix-grpc-daemon[1410]: time=2026-09-20T12:22:51.966Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5303lb # [ 126.163254] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5304lb # [ 126.175452] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5305lb # [ 126.185150] niks3-server[1093]: 2026/09/20 12:22:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5306lb # [ 126.191328] niks3-server[1093]: 2026/09/20 12:22:51 INFO Signed narinfos id=30 count=15307worker1 # [ 117.912899] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.022Z level=INFO msg="Uploading 1 narinfos"5308lb # [ 126.197250] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5309lb # [ 126.210336] niks3-server[1093]: 2026/09/20 12:22:51 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5310lb # [ 126.217060] niks3-server[1093]: 2026/09/20 12:22:51 INFO Completed upload id=305311worker1 # [ 117.937557] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.047Z level=INFO msg="Upload complete. (337ms)"5312lb # [ 126.220271] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5313worker1 # [ 117.957168] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15314worker1 # [ 117.971429] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05315worker1 # [ 117.975122] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15316worker1 # [ 117.979858] nix-daemon[934]: reaped child process 1469, status = succeeded5317worker1 # [ 118.002430] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225318worker1 # [ 118.018998] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345225319worker1 # [ 118.034015] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.144Z level=INFO msg="All 2 paths already cached"5320client # copying 1 paths to 'lb:50051'...5321client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5322worker1 # [ 118.046637] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345225323worker1 # [ 118.059013] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5324worker1 # [ 118.262135] nix-daemon[934]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)5325worker1 # [ 118.284261] nix-daemon[934]: reaped child process 1498, status = succeeded5326worker1 # [ 118.479483] nix-daemon[934]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)5327worker1 # [ 118.502064] nix-daemon[934]: reaped child process 1503, status = succeeded5328lb # [ 126.792133] niks3-server[1093]: 2026/09/20 12:22:51 INFO Received uploads request method=POST path=/api/pending_closures5329worker1 # [ 118.518984] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.628Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5330worker1 # [ 118.520758] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.628Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5331lb # [ 126.813709] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5332lb # [ 126.829336] niks3-server[1093]: 2026/09/20 12:22:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5333worker1 # [ 118.556209] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.666Z level=INFO msg="Uploading 1 narinfos"5334lb # [ 126.840954] niks3-server[1093]: 2026/09/20 12:22:51 INFO Signed narinfos id=31 count=15335lb # [ 126.843411] niks3-server[1093]: 2026/09/20 12:22:51 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5336lb # [ 126.854198] niks3-server[1093]: 2026/09/20 12:22:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5337lb # [ 126.862184] niks3-server[1093]: 2026/09/20 12:22:52 INFO Completed upload id=315338worker1 # [ 118.583482] nix-grpc-daemon[1410]: time=2026-09-20T12:22:52.693Z level=INFO msg="Upload complete. (508ms)"5339lb # [ 126.865112] niks3-server[1093]: 2026/09/20 12:22:52 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5340worker1 # [ 118.592648] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=0 paths=1 nar_bytes_in=7685341worker1 # [ 118.603765] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225342worker1 # [ 118.610560] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5343worker1 # [ 118.615728] nix-daemon[934]: reaped child process 1493, status = succeeded5344client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5345client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5346worker1 # [ 118.631199] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5347client # farm-top> LOG-farm-top-on-standby5348worker1 # [ 119.667051] nix-daemon[934]: reaped child process 1507, status = succeeded5349worker1 # [ 119.841803] nix-daemon[934]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)5350worker1 # [ 119.865190] nix-daemon[934]: reaped child process 1525, status = succeeded5351lb # [ 128.157425] niks3-server[1093]: 2026/09/20 12:22:53 INFO Received uploads request method=POST path=/api/pending_closures5352worker1 # [ 119.888536] nix-grpc-daemon[1410]: time=2026-09-20T12:22:53.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5353worker1 # [ 119.890180] nix-grpc-daemon[1410]: time=2026-09-20T12:22:53.997Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5354lb # [ 128.200222] niks3-server[1093]: 2026/09/20 12:22:53 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5355lb # [ 128.216089] niks3-server[1093]: 2026/09/20 12:22:53 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5356worker1 # [ 119.948393] nix-grpc-daemon[1410]: time=2026-09-20T12:22:54.058Z level=INFO msg="Uploading 1 narinfos"5357lb # [ 128.227266] niks3-server[1093]: 2026/09/20 12:22:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5358lb # [ 128.232581] niks3-server[1093]: 2026/09/20 12:22:53 INFO Signed narinfos id=32 count=15359lb # [ 128.239291] niks3-server[1093]: 2026/09/20 12:22:53 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5360lb # [ 128.251732] niks3-server[1093]: 2026/09/20 12:22:53 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5361lb # [ 128.259627] niks3-server[1093]: 2026/09/20 12:22:53 INFO Completed upload id=325362lb # [ 128.262604] niks3-server[1093]: 2026/09/20 12:22:53 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5363worker1 # [ 119.978737] nix-grpc-daemon[1410]: time=2026-09-20T12:22:54.089Z level=INFO msg="Upload complete. (337ms)"5364worker1 # [ 119.993648] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15365worker1 # [ 120.015047] nix-daemon[934]: reaped child process 1505, status = succeeded5366client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.35 seconds)5367client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5368client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.25 seconds)5369worker2 # [ 128.619996] systemd[1]: Listening on Nix worker-protocol over gRPC.5370worker2 # [ 128.623738] systemd[1]: Starting scheduler-token.service...5371worker2 # [ 128.710282] systemd[1]: scheduler-token.service: Deactivated successfully.5372worker2 # [ 128.715258] systemd[1]: Finished scheduler-token.service.5373worker2 # [ 128.718030] systemd[1]: scheduler-token.service: Consumed 29ms CPU time over 86ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5374worker2 # [ 128.721272] systemd[1]: Starting Nix worker-protocol over gRPC...5375worker2 # [ 128.902482] nix-grpc-daemon[1222]: ts=2026-09-20T12:22:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5376worker2 # [ 128.905110] nix-grpc-daemon[1222]: ts=2026-09-20T12:22:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5377worker2 # [ 128.909334] nix-grpc-daemon[1222]: ts=2026-09-20T12:22:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15378worker2 # [ 128.917809] nix-grpc-daemon[1222]: ts=2026-09-20T12:22:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5379worker2 # [ 128.919618] systemd[1]: Started Nix worker-protocol over gRPC.5380lb: must succeed: curl -sf localhost:9901/clusters5381worker2 # [ 128.959849] nix-grpc-daemon[1222]: ts=2026-09-20T12:22:54Z level=info event=scheduler_connected addr=lb:500515382worker1 # [ 120.741884] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:54Z level=info event=oidc_jwks provider=mock keys=15383worker1 # [ 120.747864] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:345205384worker1 # [ 120.754012] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05385lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5386worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5387worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5388lb: must succeed: curl -sf localhost:9901/clusters5389lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5390lb: must succeed: curl -sf localhost:9901/clusters5391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5392worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5393worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5394(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.15 seconds)5395subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5396client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5397client # [ 129.643891] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5398client # Running as unit: rs.service; invocation ID: 1eb457e787884ed79eb74260d9870c105399client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5400client # [ 129.894828] nix[1255]: don't know how to build these paths:5401client # [ 129.895856] nix[1255]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5402worker1 # [ 121.431546] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345225403worker1 # [ 121.446868] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05404worker1 # [ 121.455010] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15405worker1 # [ 121.493617] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225406worker1 # [ 121.510651] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345225407client # [ 129.988211] nix[1255]: copying 1 paths to 'lb:50051'...5408worker1 # [ 121.526955] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345225409client # [ 129.990662] nix[1255]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5410worker1 # [ 121.545110] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5411worker1 # [ 121.752212] nix-daemon[934]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)5412worker1 # [ 121.771194] nix-daemon[934]: reaped child process 1541, status = succeeded5413worker1 # [ 121.950437] nix-daemon[934]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5414worker1 # [ 121.971438] nix-daemon[934]: reaped child process 1546, status = succeeded5415lb # [ 130.262512] niks3-server[1093]: 2026/09/20 12:22:55 INFO Received uploads request method=POST path=/api/pending_closures5416worker1 # [ 122.060227] nix-grpc-daemon[1410]: time=2026-09-20T12:22:56.168Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5417worker1 # [ 122.063676] nix-grpc-daemon[1410]: time=2026-09-20T12:22:56.169Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5418lb # [ 130.360554] niks3-server[1093]: 2026/09/20 12:22:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5419worker1 # [ 122.093785] nix-grpc-daemon[1410]: time=2026-09-20T12:22:56.203Z level=INFO msg="Uploading 1 narinfos"5420lb # [ 130.379670] niks3-server[1093]: 2026/09/20 12:22:55 INFO Signed narinfos id=33 count=15421lb # [ 130.392285] niks3-server[1093]: 2026/09/20 12:22:55 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5422lb # [ 130.410776] niks3-server[1093]: 2026/09/20 12:22:55 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5423worker1 # [ 122.131111] nix-grpc-daemon[1410]: time=2026-09-20T12:22:56.241Z level=INFO msg="Upload complete. (567ms)"5424lb # [ 130.417252] niks3-server[1093]: 2026/09/20 12:22:55 INFO Completed upload id=335425lb # [ 130.420398] niks3-server[1093]: 2026/09/20 12:22:55 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5426worker1 # [ 122.144552] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=0 paths=1 nar_bytes_in=4165427worker1 # [ 122.157501] nix-grpc-daemon[1410]: time=2026-09-20T12:22:56.248Z level=WARN msg="Failed to register uploaded object" key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls error="server returned 404: object is not pending upload\n"5428worker1 # [ 122.178635] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345225429worker1 # [ 122.185516] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5430worker1 # [ 122.188693] nix-daemon[934]: reaped child process 1536, status = succeeded5431client # [ 130.662629] nix[1255]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5432worker1 # [ 122.202643] nix-daemon[934]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5433lb: must succeed: curl -sf localhost:9901/clusters5434lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5435lb: must succeed: curl -sf localhost:9901/clusters5436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5437lb: must succeed: curl -sf localhost:9901/clusters5438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5439lb: must succeed: curl -sf localhost:9901/clusters5440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5441worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5442worker2 # [ 132.429090] systemd-journald[280]: Received client request to rotate journal, rotating.5443worker2 # [ 132.470324] systemd-journald[280]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e9e15eb14d534bf9ae9e1dd09226eec8.5444worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.12 seconds)5445worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5446worker1 # [ 124.259723] systemd[1]: Stopping Nix worker-protocol over gRPC...5447worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5448??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5449 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735450client: must succeed: sleep 5.05451worker1 # [ 124.742683] nix-grpc-daemon[1340]: ts=2026-09-20T12:22:58Z level=info event=draining5452lb # [ 133.781227] envoy[851]: conn peer=[2001:db8:1::1]:39822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=3860 ms=80445453client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5454worker1 # [ 129.645178] nix-daemon[934]: reaped child process 1550, status = succeeded5455worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5456worker1 # [ 129.818772] nix-daemon[934]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)5457worker1 # [ 129.838879] nix-daemon[934]: reaped child process 1593, status = succeeded5458lb # [ 138.134855] niks3-server[1093]: 2026/09/20 12:23:03 INFO Received uploads request method=POST path=/api/pending_closures5459worker1 # [ 129.892528] nix-grpc-daemon[1410]: time=2026-09-20T12:23:04.000Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5460worker1 # [ 129.894400] nix-grpc-daemon[1410]: time=2026-09-20T12:23:04.000Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5461lb # [ 138.217533] niks3-server[1093]: 2026/09/20 12:23:03 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5462worker1 # [ 129.954310] nix-grpc-daemon[1410]: time=2026-09-20T12:23:04.064Z level=INFO msg="Uploading 1 narinfos"5463lb # [ 138.232994] niks3-server[1093]: 2026/09/20 12:23:03 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5464lb # [ 138.242513] niks3-server[1093]: 2026/09/20 12:23:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5465lb # [ 138.253912] niks3-server[1093]: 2026/09/20 12:23:03 INFO Signed narinfos id=34 count=15466lb # [ 138.258552] niks3-server[1093]: 2026/09/20 12:23:03 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5467lb # [ 138.263265] niks3-server[1093]: 2026/09/20 12:23:03 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5468lb # [ 138.266891] niks3-server[1093]: 2026/09/20 12:23:03 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5469worker1 # [ 129.987865] nix-grpc-daemon[1410]: time=2026-09-20T12:23:04.098Z level=INFO msg="Upload complete. (360ms)"5470lb # [ 138.269928] niks3-server[1093]: 2026/09/20 12:23:03 INFO Completed upload id=345471worker1 # [ 130.001953] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34522 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15472worker1 # [ 130.024862] nix-daemon[934]: reaped child process 1548, status = succeeded5473client # [ 138.508862] systemd[1]: rs.service: Deactivated successfully.5474client # [ 138.514161] systemd[1]: rs.service: Consumed 164ms CPU time over 8.868s wall clock time, 38.9M memory peak, 8.5K incoming IP traffic, 9.2K outgoing IP traffic.5475worker1 # [ 130.744392] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:04Z level=info event=signal_exit5476worker1 # [ 130.962308] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_yield5477lb # [ 139.248087] niks3-server[1093]: 2026/09/20 12:23:04 INFO lead: released remote=192.168.1.3:421125478worker2 # [ 139.201517] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:04Z level=info event=scheduler_restarting addr=lb:500515479worker1 # [ 130.969084] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:345205480worker1 # [ 130.974742] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=worker_gone addr=192.168.1.4:500515481worker1 # [ 130.980128] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_restarting addr=lb:500515482worker1 # [ 130.984171] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:345205483worker1 # [ 130.988560] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=worker_gone addr=192.168.1.3:500515484worker2 # [ 139.401588] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:04Z level=info event=scheduler_connected addr=lb:500515485worker2 # [ 139.407222] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker1 # [ 131.177287] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515487worker1 # [ 131.182264] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5488worker2 # [ 139.608704] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:04Z level=info event=scheduler_connected addr=lb:500515489worker2 # [ 139.613963] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490worker1 # [ 131.383294] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515491worker1 # [ 131.388805] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker2 # [ 139.815290] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515493worker2 # [ 139.821262] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494worker1 # [ 131.589854] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 131.595381] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496worker2 # [ 140.022372] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515497worker1 # [ 131.796637] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 140.032494] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 131.802114] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.22 seconds)5501client: waiting for success: ! systemctl is-active rs5502client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5503client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5504worker1 # [ 131.968178] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:06Z level=info event=scheduler_restarting5505worker2 # [ 140.231327] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515506worker2 # [ 140.235747] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5507client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5508client: must fail: journalctl -u rs | grep -q 'reconnecting'5509worker1 # [ 132.018415] nix-daemon[934]: reaped child process 1401, status = succeeded5510worker1 # [ 132.027660] nix-grpc-daemon[1340]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5511worker1 # [ 132.031401] nix-grpc-daemon[1340]: E0000 00:00:1789906986.134167 1383 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5512worker1 # [ 132.036447] nix-grpc-daemon[1340]: ts=2026-09-20T12:23:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5513client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5514lb: must succeed: curl -sf localhost:9901/clusters5515worker1 # [ 132.041033] nix-grpc-daemon[1340]: E0000 00:00:1789906986.151260 1383 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5517worker2 # [ 140.432583] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515518worker2 # [ 140.438200] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker2 # [ 140.639515] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_connected addr=lb:500515520worker2 # [ 140.645108] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker2 # [ 140.846390] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_connected addr=lb:500515522worker2 # [ 140.852105] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5523worker2 # [ 141.053175] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_connected addr=lb:500515524worker2 # [ 141.058526] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker1 # [ 132.931423] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5526worker1 # [ 132.934622] systemd[1]: Stopped Nix worker-protocol over gRPC.5527worker1 # [ 132.940814] systemd[1]: nix-grpc-daemon.service: Consumed 1.918s CPU time over 27.440s wall clock time, 34.3M memory peak, 49.1K incoming IP traffic, 88.5K outgoing IP traffic.5528worker1 # [ 132.958901] systemd[1]: Starting Nix worker-protocol over gRPC...5529worker2 # [ 141.261794] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_connected addr=lb:500515530worker2 # [ 141.266431] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531lb: must succeed: curl -sf localhost:9901/clusters5532worker1 # [ 133.156093] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5533worker1 # [ 133.158768] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5534worker1 # [ 133.162939] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15535worker1 # [ 133.170972] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5536worker1 # [ 133.175170] systemd[1]: Started Nix worker-protocol over gRPC.5537worker1 # [ 133.185653] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_take_over5538lb # [ 141.465440] niks3-server[1093]: 2026/09/20 12:23:06 INFO lead: acquired remote=192.168.1.3:574205539worker1 # [ 133.196967] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515540worker1 # [ 133.198916] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5542worker2 # [ 141.463407] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_connected addr=lb:500515543worker2 # [ 141.469039] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5544worker1 # [ 133.400420] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515545worker1 # [ 133.405962] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5546worker2 # [ 141.670369] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_connected addr=lb:500515547worker2 # [ 141.675962] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5548worker1 # [ 133.606969] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515549worker1 # [ 133.612793] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5550worker2 # [ 141.877350] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515551worker2 # [ 141.882666] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5552worker1 # [ 133.813549] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515553worker1 # [ 133.818863] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5554worker2 # [ 142.083985] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515555worker2 # [ 142.089488] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker1 # [ 134.019641] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=scheduler_connected addr=lb:500515557worker1 # [ 134.029337] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:466045558worker1 # [ 134.035282] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05559worker2 # [ 142.292097] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:07Z level=info event=scheduler_connected addr=lb:500515560worker1 # [ 134.068168] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=oidc_jwks provider=mock keys=15561worker1 # [ 134.073000] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:466045562worker1 # [ 134.078402] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05563lb: must succeed: curl -sf localhost:9901/clusters5564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5565worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5566worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5567(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.25 seconds)5568subtest: scheduler restart mid-build: build finishes, no double build5569client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5570client # [ 142.889231] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5571client # Running as unit: mid.service; invocation ID: a624e67e0af84d88a0754d64a88efcb85572client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5573worker1 # [ 134.685618] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466205574worker1 # [ 134.701245] nix-daemon[934]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)5575client # [ 143.182751] nix[1291]: don't know how to build these paths:5576client # [ 143.184379] nix[1291]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5577worker1 # [ 134.731674] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05578worker1 # [ 134.739235] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15579worker1 # [ 134.779091] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466205580worker1 # [ 134.797040] nix-grpc-daemon[1600]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)5581worker1 # [ 134.828561] nix-grpc-daemon[1600]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)5582worker1 # [ 134.884168] nix-grpc-daemon[1600]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 247 ms (attempt 3/5)5583worker1 # [ 135.131605] nix-grpc-daemon[1600]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 4/5)5584worker1 # [ 135.201358] nix-grpc-daemon[1600]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5585worker1 # [ 135.223288] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466205586client # [ 143.699722] nix[1291]: copying 1 paths to 'lb:50051'...5587client # [ 143.707813] nix[1291]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5588worker1 # [ 135.244769] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466205589worker1 # [ 135.260186] nix-daemon[934]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)5590worker1 # [ 135.291727] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:09Z level=info event=niks3_push_started pid=16655591lb # [ 143.780079] envoy[851]: conn peer=[2001:db8:1::1]:41154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1348 ms=85865592lb # [ 143.780363] envoy[851]: conn peer=[2001:db8:1::3]:39790 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4160 tx=1315 ms=262615593worker1 # [ 135.518230] nix-grpc-daemon[1665]: time=2026-09-20T12:23:09.627Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5594worker1 # [ 135.717034] nix-daemon[934]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)5595worker1 # [ 135.745170] nix-daemon[934]: reaped child process 1685, status = succeeded5596worker1 # [ 135.925152] nix-daemon[934]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)5597worker1 # [ 135.946942] nix-daemon[934]: reaped child process 1690, status = succeeded5598lb # [ 144.239031] niks3-server[1093]: 2026/09/20 12:23:09 INFO Received uploads request method=POST path=/api/pending_closures5599worker1 # [ 135.984087] nix-grpc-daemon[1665]: time=2026-09-20T12:23:10.093Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5600worker1 # [ 135.985725] nix-grpc-daemon[1665]: time=2026-09-20T12:23:10.093Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5601lb # [ 144.292704] niks3-server[1093]: 2026/09/20 12:23:09 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5602worker1 # [ 136.027511] nix-grpc-daemon[1665]: time=2026-09-20T12:23:10.137Z level=INFO msg="Uploading 1 narinfos"5603lb # [ 144.306322] niks3-server[1093]: 2026/09/20 12:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5604lb # [ 144.317252] niks3-server[1093]: 2026/09/20 12:23:09 INFO Signed narinfos id=35 count=15605lb # [ 144.325618] niks3-server[1093]: 2026/09/20 12:23:09 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5606lb # [ 144.336782] niks3-server[1093]: 2026/09/20 12:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5607lb # [ 144.349405] niks3-server[1093]: 2026/09/20 12:23:09 INFO Completed upload id=355608worker1 # [ 136.071017] nix-grpc-daemon[1665]: time=2026-09-20T12:23:10.181Z level=INFO msg="Upload complete. (548ms)"5609lb # [ 144.356840] niks3-server[1093]: 2026/09/20 12:23:09 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5610worker1 # [ 136.084803] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46620 duration_s=0 paths=1 nar_bytes_in=4245611worker1 # [ 136.100100] nix-grpc-daemon[1600]: ts=2026-09-20T12:23:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466205612worker1 # [ 136.108191] nix-daemon[934]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)5613worker1 # [ 136.114417] nix-daemon[934]: reaped child process 1663, status = succeeded5614client # [ 144.588937] nix[1291]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5615worker1 # [ 136.127582] nix-daemon[934]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5618lb: must succeed: curl -sf localhost:9901/clusters5619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5620worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5621worker1 # [ 138.496751] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1600 (nix-grpc-daemon) on client request.5622worker1 # [ 138.498387] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5623lb # [ 146.798831] niks3-server[1093]: 2026/09/20 12:23:11 INFO lead: released remote=192.168.1.3:574205624client # [ 146.991097] nix[1291]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5625worker2 # [ 146.761661] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:12Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5626worker1 # [ 138.537534] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5627worker1 # [ 138.539188] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5628client # [ 146.997945] nix[1291]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5629worker1 # [ 138.549712] systemd[1]: nix-grpc-daemon.service: Consumed 543ms CPU time over 5.577s wall clock time, 31.5M memory peak, 14.4K incoming IP traffic, 16.8K outgoing IP traffic.5630worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.10 seconds)5631client: must succeed: sleep 5.05632worker1 # [ 138.563331] nix-daemon[934]: reaped child process 1656, status = succeeded5633worker1 # [ 138.565363] nix-daemon[934]: reaped child process 1692, status = succeeded5634worker1 # [ 138.572641] nix-daemon[1694]: unexpected Nix daemon error: error: interrupted by the user5635worker1 # [ 138.579532] nix-daemon[934]: reaped child process 1694, status = failed with exit code 15636worker2 # [ 146.962026] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:12Z level=info event=scheduler_connected addr=lb:500515637worker1 # [ 138.733708] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5638worker1 # [ 138.745944] systemd[1]: Starting Nix worker-protocol over gRPC...5639worker1 # [ 138.957627] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5640worker1 # [ 138.960282] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5641worker1 # [ 138.964348] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15642worker1 # [ 138.972064] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5643worker1 # [ 138.973813] systemd[1]: Started Nix worker-protocol over gRPC.5644lb # [ 147.264224] niks3-server[1093]: 2026/09/20 12:23:12 INFO lead: acquired remote=192.168.1.3:574925645worker1 # [ 138.989750] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=scheduler_take_over5646worker1 # [ 139.005362] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=scheduler_connected addr=lb:500515647worker1 # [ 139.009411] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=oidc_jwks provider=mock keys=15648worker1 # [ 139.012428] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:466285649worker1 # [ 139.014104] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05650worker1 # [ 139.015912] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:466285651worker1 # [ 139.017814] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05652worker1 # [ 139.037540] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05653worker1 # [ 139.040366] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15654worker1 # [ 139.088979] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466585655worker1 # [ 139.100095] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5656worker1 # [ 139.116264] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5657worker1 # [ 139.127269] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5658client # [ 147.588683] nix[1291]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5659client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5660client: waiting for success: ! systemctl is-active mid5661worker1 # [ 143.948875] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:18Z level=info event=niks3_push_started pid=18095662worker1 # [ 143.981371] nix-daemon[934]: reaped child process 1785, status = succeeded5663worker1 # [ 144.180420] nix-grpc-daemon[1809]: time=2026-09-20T12:23:18.290Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5664worker1 # [ 144.361151] nix-daemon[934]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)5665worker1 # [ 144.382689] nix-daemon[934]: reaped child process 1825, status = succeeded5666lb # [ 152.677168] niks3-server[1093]: 2026/09/20 12:23:17 INFO Received uploads request method=POST path=/api/pending_closures5667worker1 # [ 144.410724] nix-grpc-daemon[1809]: time=2026-09-20T12:23:18.519Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5668worker1 # [ 144.413805] nix-grpc-daemon[1809]: time=2026-09-20T12:23:18.519Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5669lb # [ 152.726087] niks3-server[1093]: 2026/09/20 12:23:17 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5670worker1 # [ 144.469128] nix-grpc-daemon[1809]: time=2026-09-20T12:23:18.578Z level=INFO msg="Uploading 1 narinfos"5671lb # [ 152.746660] niks3-server[1093]: 2026/09/20 12:23:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5672lb # [ 152.758107] niks3-server[1093]: 2026/09/20 12:23:17 INFO Signed narinfos id=36 count=15673lb # [ 152.773757] niks3-server[1093]: 2026/09/20 12:23:17 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5674lb # [ 152.784282] niks3-server[1093]: 2026/09/20 12:23:17 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5675lb # [ 152.796422] niks3-server[1093]: 2026/09/20 12:23:17 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5676lb # [ 152.799539] niks3-server[1093]: 2026/09/20 12:23:17 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5677lb # [ 152.814121] niks3-server[1093]: 2026/09/20 12:23:17 INFO Completed upload id=365678worker1 # [ 144.535571] nix-grpc-daemon[1809]: time=2026-09-20T12:23:18.645Z level=INFO msg="Upload complete. (350ms)"5679worker1 # [ 144.552529] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46658 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15680worker1 # [ 144.574455] nix-daemon[934]: reaped child process 1783, status = succeeded5681client # [ 153.057273] systemd[1]: mid.service: Deactivated successfully.5682client # [ 153.061991] systemd[1]: mid.service: Consumed 181ms CPU time over 10.172s wall clock time, 37.2M memory peak, 9.8K incoming IP traffic, 10.8K outgoing IP traffic.5683client: (finished: waiting for success: ! systemctl is-active mid, in 1.15 seconds)5684client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5685client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5686(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.79 seconds)5687subtest: interrupting the client stops the build on the worker5688client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5689client # [ 153.648369] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5690client # Running as unit: intr.service; invocation ID: 99b665bc05624f568ead6baf7f737f495691client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5692worker2 # [ 153.671347] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479705693worker2 # [ 153.687311] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5694client # [ 153.937425] nix[1327]: don't know how to build these paths:5695client # [ 153.939081] nix[1327]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5696worker1 # [ 145.495102] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:19Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05697lb # [ 153.780494] envoy[851]: conn peer=[2001:db8:1::3]:56498 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=411 ms=53435698worker1 # [ 145.501690] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:19Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15699lb # [ 153.780891] envoy[851]: conn peer=[2001:db8:1::1]:47200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3884 tx=1672 ms=98855700worker2 # [ 153.775594] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705701worker2 # [ 153.796099] nix-grpc-daemon[1222]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)5702worker2 # [ 153.904299] nix-grpc-daemon[1222]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 2/5)5703worker2 # [ 154.074518] nix-grpc-daemon[1222]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 3/5)5704worker2 # [ 154.234708] nix-grpc-daemon[1222]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 4/5)5705worker2 # [ 154.247204] nix-grpc-daemon[1222]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5706worker2 # [ 154.273042] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479705707client # [ 154.517156] nix[1327]: copying 1 paths to 'lb:50051'...5708client # [ 154.524257] nix[1327]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5709worker2 # [ 154.295132] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479705710worker2 # [ 154.310846] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5711worker2 # [ 154.348104] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:19Z level=info event=niks3_push_started pid=13605712worker2 # [ 154.575748] nix-grpc-daemon[1360]: time=2026-09-20T12:23:19.834Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5713worker2 # [ 154.769505] nix-daemon[938]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)5714worker2 # [ 154.801702] nix-daemon[938]: reaped child process 1380, status = succeeded5715worker2 # [ 154.980122] nix-daemon[938]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5716worker2 # [ 155.000667] nix-daemon[938]: reaped child process 1385, status = succeeded5717lb # [ 155.059295] niks3-server[1093]: 2026/09/20 12:23:20 INFO Received uploads request method=POST path=/api/pending_closures5718worker2 # [ 155.027755] nix-grpc-daemon[1360]: time=2026-09-20T12:23:20.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5719worker2 # [ 155.030312] nix-grpc-daemon[1360]: time=2026-09-20T12:23:20.285Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5720lb # [ 155.111125] niks3-server[1093]: 2026/09/20 12:23:20 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5721worker2 # [ 155.077242] nix-grpc-daemon[1360]: time=2026-09-20T12:23:20.334Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 155.121983] niks3-server[1093]: 2026/09/20 12:23:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5723lb # [ 155.128494] niks3-server[1093]: 2026/09/20 12:23:20 INFO Signed narinfos id=37 count=15724lb # [ 155.136452] niks3-server[1093]: 2026/09/20 12:23:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5725lb # [ 155.148705] niks3-server[1093]: 2026/09/20 12:23:20 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5726lb # [ 155.158944] niks3-server[1093]: 2026/09/20 12:23:20 INFO Completed upload id=375727worker2 # [ 155.113064] nix-grpc-daemon[1360]: time=2026-09-20T12:23:20.372Z level=INFO msg="Upload complete. (532ms)"5728lb # [ 155.162649] niks3-server[1093]: 2026/09/20 12:23:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5729worker2 # [ 155.123319] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=4245730worker2 # [ 155.143390] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705731worker2 # [ 155.152389] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5732worker2 # [ 155.157535] nix-daemon[938]: reaped child process 1358, status = succeeded5733client # [ 155.399782] nix[1327]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5734worker2 # [ 155.171109] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5735client: must succeed: systemctl kill -s INT intr5736worker2 # [ 156.021896] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:21Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5737client # [ 156.250141] systemd[1]: intr.service: Sent signal SIGINT to main process 1327 (nix) on client request.5738client # [ 156.259356] nix[1327]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5739client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5740client # [ 156.269522] nix[1327]: error: interrupted by the user5741client # [ 156.305648] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5742client # [ 156.309746] systemd[1]: intr.service: Failed with result 'exit-code'.5743client # [ 156.314451] systemd[1]: intr.service: Consumed 157ms CPU time over 2.660s wall clock time, 39.2M memory peak, 7.9K incoming IP traffic, 8.7K outgoing IP traffic.5744worker2 # [ 156.207957] nix-daemon[938]: reaped child process 1387, status = succeeded5745worker2 # [ 156.216732] nix-daemon[1389]: unexpected Nix daemon error: error: interrupted by the user5746worker2 # [ 156.222271] nix-daemon[938]: reaped child process 1389, status = failed with exit code 15747(finished: subtest: interrupting the client stops the build on the worker, in 4.00 seconds)5748subtest: a deploy mid-build drains: the build finishes, the new generation takes over5749client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5750client # [ 157.669991] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5751client # Running as unit: sw.service; invocation ID: c9b6d7eb1a3646708d1e9b441e9f030a5752client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5753client # [ 157.910160] nix[1348]: don't know how to build these paths:5754client # [ 157.911184] nix[1348]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5755worker2 # [ 157.680644] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479705756worker1 # [ 149.466301] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:23Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05757worker1 # [ 149.466690] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:23Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15758worker1 # [ 149.510298] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466585759worker1 # [ 149.526648] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5760worker1 # [ 149.592947] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)5761worker1 # [ 149.617545] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 3/5)5762worker1 # [ 149.635081] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 4/5)5763worker1 # [ 149.748640] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5764worker1 # [ 149.770609] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466585765client # [ 158.248196] nix[1348]: copying 1 paths to 'lb:50051'...5766worker1 # [ 149.787279] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466585767client # [ 158.250627] nix[1348]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5768worker1 # [ 149.806230] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5769worker1 # [ 150.011792] nix-daemon[934]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)5770worker1 # [ 150.033584] nix-daemon[934]: reaped child process 1863, status = succeeded5771worker1 # [ 150.204574] nix-daemon[934]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)5772worker1 # [ 150.225035] nix-daemon[934]: reaped child process 1868, status = succeeded5773lb # [ 158.516900] niks3-server[1093]: 2026/09/20 12:23:23 INFO Received uploads request method=POST path=/api/pending_closures5774worker1 # [ 150.252597] nix-grpc-daemon[1809]: time=2026-09-20T12:23:24.362Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5775worker1 # [ 150.254705] nix-grpc-daemon[1809]: time=2026-09-20T12:23:24.364Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5776lb # [ 158.555560] niks3-server[1093]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5777worker1 # [ 150.288427] nix-grpc-daemon[1809]: time=2026-09-20T12:23:24.398Z level=INFO msg="Uploading 1 narinfos"5778lb # [ 158.569926] niks3-server[1093]: 2026/09/20 12:23:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5779lb # [ 158.579562] niks3-server[1093]: 2026/09/20 12:23:23 INFO Signed narinfos id=38 count=15780lb # [ 158.591239] niks3-server[1093]: 2026/09/20 12:23:23 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5781worker1 # [ 150.320370] nix-grpc-daemon[1809]: time=2026-09-20T12:23:24.430Z level=INFO msg="Upload complete. (495ms)"5782lb # [ 158.598093] niks3-server[1093]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5783lb # [ 158.602707] niks3-server[1093]: 2026/09/20 12:23:23 INFO Completed upload id=385784lb # [ 158.605367] niks3-server[1093]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5785worker1 # [ 150.328240] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46658 duration_s=0 paths=1 nar_bytes_in=4165786worker1 # [ 150.344439] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466585787worker1 # [ 150.350909] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5788worker1 # [ 150.356270] nix-daemon[934]: reaped child process 1858, status = succeeded5789client # [ 158.828712] nix[1348]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5790worker1 # [ 150.371253] nix-daemon[934]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5791worker1: must succeed: systemctl reload nix-grpc-daemon.service5792worker1 # [ 151.899880] systemd[1]: Reloading Nix worker-protocol over gRPC...5793worker1 # [ 151.954263] systemd[1]: Reloaded Nix worker-protocol over gRPC.5794worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5795worker1: must succeed: pgrep -f 'read -t [9]0 x'5796worker1 # [ 151.987775] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:26Z level=info event=draining5797worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5798client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25799client # don't know how to build these paths:5800client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5801worker2 # [ 160.498517] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479705802worker1 # [ 152.291209] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:26Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05803worker1 # [ 152.296248] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:26Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15804worker2 # [ 160.506692] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479705805worker2 # [ 160.570778] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705806worker2 # [ 160.591351] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479705807client # copying 1 paths to 'lb:50051'...5808worker2 # [ 160.607665] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479705809client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5810worker2 # [ 160.632103] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5811worker2 # [ 160.843750] nix-daemon[938]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5812worker2 # [ 160.864791] nix-daemon[938]: reaped child process 1438, status = succeeded5813worker2 # [ 161.040138] nix-daemon[938]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5814worker2 # [ 161.060292] nix-daemon[938]: reaped child process 1443, status = succeeded5815lb # [ 161.118690] niks3-server[1093]: 2026/09/20 12:23:26 INFO Received uploads request method=POST path=/api/pending_closures5816worker2 # [ 161.084283] nix-grpc-daemon[1360]: time=2026-09-20T12:23:26.342Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5817worker2 # [ 161.085821] nix-grpc-daemon[1360]: time=2026-09-20T12:23:26.342Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5818lb # [ 161.152659] niks3-server[1093]: 2026/09/20 12:23:26 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5819lb # [ 161.167079] niks3-server[1093]: 2026/09/20 12:23:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5820worker2 # [ 161.125877] nix-grpc-daemon[1360]: time=2026-09-20T12:23:26.384Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 161.175721] niks3-server[1093]: 2026/09/20 12:23:26 INFO Signed narinfos id=39 count=15822lb # [ 161.180300] niks3-server[1093]: 2026/09/20 12:23:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5823lb # [ 161.187654] niks3-server[1093]: 2026/09/20 12:23:26 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5824lb # [ 161.194218] niks3-server[1093]: 2026/09/20 12:23:26 INFO Completed upload id=395825worker2 # [ 161.147348] nix-grpc-daemon[1360]: time=2026-09-20T12:23:26.406Z level=INFO msg="Upload complete. (497ms)"5826lb # [ 161.196489] niks3-server[1093]: 2026/09/20 12:23:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5827worker2 # [ 161.156971] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=5285828worker2 # [ 161.164261] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705829worker2 # [ 161.176963] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5830worker2 # [ 161.179599] nix-daemon[938]: reaped child process 1433, status = succeeded5831client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5832client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5833worker2 # [ 161.192442] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5834client # farm-a> LOG-farm-a-during-drain5835worker2 # [ 162.641670] nix-daemon[938]: reaped child process 1447, status = succeeded5836worker2 # [ 162.817423] nix-daemon[938]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5837worker2 # [ 162.836115] nix-daemon[938]: reaped child process 1465, status = succeeded5838lb # [ 162.894335] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received uploads request method=POST path=/api/pending_closures5839worker2 # [ 162.860543] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.118Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5840worker2 # [ 162.866331] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.118Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5841lb # [ 162.934106] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5842lb # [ 162.952048] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5843worker2 # [ 162.920142] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.179Z level=INFO msg="Uploading 1 narinfos"5844lb # [ 162.966032] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5845lb # [ 162.973605] niks3-server[1093]: 2026/09/20 12:23:28 INFO Signed narinfos id=40 count=15846lb # [ 162.978878] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5847lb # [ 162.992296] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5848lb # [ 162.997966] niks3-server[1093]: 2026/09/20 12:23:28 INFO Completed upload id=405849worker2 # [ 162.952091] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.211Z level=INFO msg="Upload complete. (346ms)"5850lb # [ 163.001244] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5851worker1 # [ 154.733220] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:28Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05852worker1 # [ 154.736228] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:28Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15853worker2 # [ 162.965395] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15854worker2 # [ 162.986000] nix-daemon[938]: reaped child process 1445, status = succeeded5855worker2 # [ 163.020742] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705856worker2 # [ 163.036864] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479705857worker2 # [ 163.052972] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.312Z level=INFO msg="All 1 paths already cached"5858client # copying 1 paths to 'lb:50051'...5859worker2 # [ 163.064950] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479705860client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5861worker2 # [ 163.094688] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5862worker2 # [ 163.310785] nix-daemon[938]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)5863worker2 # [ 163.328897] nix-daemon[938]: reaped child process 1472, status = succeeded5864worker2 # [ 163.504147] nix-daemon[938]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)5865worker2 # [ 163.521046] nix-daemon[938]: reaped child process 1477, status = succeeded5866lb # [ 163.580427] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received uploads request method=POST path=/api/pending_closures5867worker2 # [ 163.545093] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.803Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5868worker2 # [ 163.546559] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.803Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5869lb # [ 163.612962] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5870worker2 # [ 163.578603] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.837Z level=INFO msg="Uploading 1 narinfos"5871lb # [ 163.628581] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5872lb # [ 163.638557] niks3-server[1093]: 2026/09/20 12:23:28 INFO Signed narinfos id=41 count=15873lb # [ 163.642268] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5874lb # [ 163.651704] niks3-server[1093]: 2026/09/20 12:23:28 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5875worker2 # [ 163.605025] nix-grpc-daemon[1360]: time=2026-09-20T12:23:28.864Z level=INFO msg="Upload complete. (491ms)"5876lb # [ 163.657250] niks3-server[1093]: 2026/09/20 12:23:28 INFO Completed upload id=415877lb # [ 163.660599] niks3-server[1093]: 2026/09/20 12:23:28 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5878worker2 # [ 163.614963] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=6405879worker2 # [ 163.626216] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705880worker2 # [ 163.632373] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5881worker2 # [ 163.637203] nix-daemon[938]: reaped child process 1467, status = succeeded5882client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5883client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5884worker2 # [ 163.650971] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5885lb # [ 163.777847] envoy[851]: conn peer=[2001:db8:1::1]:50548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=757 ms=23965886client # farm-b> LOG-farm-b-during-drain5887worker2 # [ 164.882096] nix-daemon[938]: reaped child process 1481, status = succeeded5888worker2 # [ 165.048928] nix-daemon[938]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5889worker2 # [ 165.072342] nix-daemon[938]: reaped child process 1499, status = succeeded5890lb # [ 165.133105] niks3-server[1093]: 2026/09/20 12:23:30 INFO Received uploads request method=POST path=/api/pending_closures5891worker2 # [ 165.521461] nix-grpc-daemon[1360]: time=2026-09-20T12:23:30.776Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5892worker2 # [ 165.526166] nix-grpc-daemon[1360]: time=2026-09-20T12:23:30.777Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5893lb # [ 165.605685] niks3-server[1093]: 2026/09/20 12:23:30 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5894worker2 # [ 165.572890] nix-grpc-daemon[1360]: time=2026-09-20T12:23:30.831Z level=INFO msg="Uploading 1 narinfos"5895lb # [ 165.620712] niks3-server[1093]: 2026/09/20 12:23:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5896lb # [ 165.637335] niks3-server[1093]: 2026/09/20 12:23:30 INFO Signed narinfos id=42 count=15897lb # [ 165.645810] niks3-server[1093]: 2026/09/20 12:23:30 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5898lb # [ 165.655108] niks3-server[1093]: 2026/09/20 12:23:30 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5899lb # [ 165.670519] niks3-server[1093]: 2026/09/20 12:23:30 INFO Completed upload id=425900worker2 # [ 165.627023] nix-grpc-daemon[1360]: time=2026-09-20T12:23:30.886Z level=INFO msg="Upload complete. (766ms)"5901worker1 # [ 157.407501] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:31Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05902worker1 # [ 157.409652] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:31Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15903lb # [ 165.687496] niks3-server[1093]: 2026/09/20 12:23:30 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5904worker2 # [ 165.642736] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=2 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15905lb # [ 165.700568] niks3-server[1093]: 2026/09/20 12:23:30 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5906worker2 # [ 165.661611] nix-daemon[938]: reaped child process 1479, status = succeeded5907worker2 # [ 165.693072] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705908worker2 # [ 165.709796] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479705909worker2 # [ 165.732859] nix-grpc-daemon[1360]: time=2026-09-20T12:23:30.991Z level=INFO msg="All 2 paths already cached"5910client # copying 1 paths to 'lb:50051'...5911worker2 # [ 165.742549] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479705912client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5913worker2 # [ 165.763249] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5914worker2 # [ 165.985864] nix-daemon[938]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)5915worker2 # [ 166.005196] nix-daemon[938]: reaped child process 1506, status = succeeded5916worker2 # [ 166.182522] nix-daemon[938]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)5917worker2 # [ 166.201101] nix-daemon[938]: reaped child process 1511, status = succeeded5918lb # [ 166.259045] niks3-server[1093]: 2026/09/20 12:23:31 INFO Received uploads request method=POST path=/api/pending_closures5919worker2 # [ 166.222423] nix-grpc-daemon[1360]: time=2026-09-20T12:23:31.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5920worker2 # [ 166.223906] nix-grpc-daemon[1360]: time=2026-09-20T12:23:31.481Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5921lb # [ 166.288892] niks3-server[1093]: 2026/09/20 12:23:31 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5922worker2 # [ 166.259305] nix-grpc-daemon[1360]: time=2026-09-20T12:23:31.518Z level=INFO msg="Uploading 1 narinfos"5923lb # [ 166.305248] niks3-server[1093]: 2026/09/20 12:23:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5924lb # [ 166.312651] niks3-server[1093]: 2026/09/20 12:23:31 INFO Signed narinfos id=43 count=15925lb # [ 166.315927] niks3-server[1093]: 2026/09/20 12:23:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5926lb # [ 166.330417] niks3-server[1093]: 2026/09/20 12:23:31 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5927lb # [ 166.340220] niks3-server[1093]: 2026/09/20 12:23:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5928lb # [ 166.345158] niks3-server[1093]: 2026/09/20 12:23:31 INFO Completed upload id=435929worker2 # [ 166.298698] nix-grpc-daemon[1360]: time=2026-09-20T12:23:31.557Z level=INFO msg="Upload complete. (513ms)"5930worker2 # [ 166.307924] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=7685931worker2 # [ 166.324534] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479705932worker2 # [ 166.330684] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5933worker2 # [ 166.335866] nix-daemon[938]: reaped child process 1501, status = succeeded5934client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5935worker2 # [ 166.348992] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5936client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5937client # farm-top> LOG-farm-top-during-drain5938worker2 # [ 167.597911] nix-daemon[938]: reaped child process 1515, status = succeeded5939worker2 # [ 167.769871] nix-daemon[938]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)5940worker2 # [ 167.788321] nix-daemon[938]: reaped child process 1533, status = succeeded5941lb # [ 167.846946] niks3-server[1093]: 2026/09/20 12:23:32 INFO Received uploads request method=POST path=/api/pending_closures5942worker2 # [ 167.812428] nix-grpc-daemon[1360]: time=2026-09-20T12:23:33.070Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5943worker2 # [ 167.813963] nix-grpc-daemon[1360]: time=2026-09-20T12:23:33.070Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5944lb # [ 167.888451] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5945lb # [ 167.901474] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5946lb # [ 167.911022] niks3-server[1093]: 2026/09/20 12:23:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5947worker2 # [ 167.870286] nix-grpc-daemon[1360]: time=2026-09-20T12:23:33.128Z level=INFO msg="Uploading 1 narinfos"5948lb # [ 167.919310] niks3-server[1093]: 2026/09/20 12:23:33 INFO Signed narinfos id=44 count=15949lb # [ 167.924230] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5950lb # [ 167.930513] niks3-server[1093]: 2026/09/20 12:23:33 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5951lb # [ 167.935941] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5952lb # [ 167.940428] niks3-server[1093]: 2026/09/20 12:23:33 INFO Completed upload id=445953worker2 # [ 167.894495] nix-grpc-daemon[1360]: time=2026-09-20T12:23:33.153Z level=INFO msg="Upload complete. (331ms)"5954worker2 # [ 167.905288] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15955worker2 # [ 167.923805] nix-daemon[938]: reaped child process 1513, status = succeeded5956client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 7.70 seconds)5957client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5958client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5959worker1: must succeed: pkill -f 'read -t [9]0 x'5960worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5961client: waiting for success: ! systemctl is-active sw5962worker1 # [ 160.229031] nix-daemon[934]: reaped child process 1872, status = succeeded5963worker1 # [ 160.407671] nix-daemon[934]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)5964worker1 # [ 160.428684] nix-daemon[934]: reaped child process 1921, status = succeeded5965lb # [ 168.719761] niks3-server[1093]: 2026/09/20 12:23:33 INFO Received uploads request method=POST path=/api/pending_closures5966worker1 # [ 160.452493] nix-grpc-daemon[1809]: time=2026-09-20T12:23:34.562Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5967worker1 # [ 160.454384] nix-grpc-daemon[1809]: time=2026-09-20T12:23:34.562Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5968lb # [ 168.757752] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5969lb # [ 168.771189] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5970lb # [ 168.788501] niks3-server[1093]: 2026/09/20 12:23:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5971worker1 # [ 160.518018] nix-grpc-daemon[1809]: time=2026-09-20T12:23:34.626Z level=INFO msg="Uploading 1 narinfos"5972lb # [ 168.801813] niks3-server[1093]: 2026/09/20 12:23:33 INFO Signed narinfos id=45 count=15973lb # [ 168.808256] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5974lb # [ 168.816569] niks3-server[1093]: 2026/09/20 12:23:33 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5975lb # [ 168.823202] niks3-server[1093]: 2026/09/20 12:23:33 INFO Completed upload id=455976worker1 # [ 160.543857] nix-grpc-daemon[1809]: time=2026-09-20T12:23:34.654Z level=INFO msg="Upload complete. (353ms)"5977lb # [ 168.825874] niks3-server[1093]: 2026/09/20 12:23:33 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5978worker1 # [ 160.557234] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46658 duration_s=10 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15979worker1 # [ 160.577823] nix-daemon[934]: reaped child process 1870, status = succeeded5980client # [ 169.058744] systemd[1]: sw.service: Deactivated successfully.5981client # [ 169.063326] systemd[1]: sw.service: Consumed 160ms CPU time over 11.393s wall clock time, 39.2M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.5982worker1 # [ 160.992419] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=signal_exit5983client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)5984client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5985client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5986worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5987worker1 # [ 161.210219] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_yield5988lb # [ 169.493363] niks3-server[1093]: 2026/09/20 12:23:34 INFO lead: released remote=192.168.1.3:574925989worker2 # [ 169.450451] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:34Z level=info event=scheduler_restarting addr=lb:500515990worker1 # [ 161.220710] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:466285991worker1 # [ 161.233462] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=worker_gone addr=192.168.1.4:500515992worker1 # [ 161.238363] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_restarting addr=lb:500515993worker1 # [ 161.243500] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:466285994worker1 # [ 161.250719] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=worker_gone addr=192.168.1.3:500515995worker2 # [ 169.650835] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:34Z level=info event=scheduler_connected addr=lb:500515996worker2 # [ 169.656274] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5997worker1 # [ 161.430812] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500515998worker1 # [ 161.436221] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5999worker2 # [ 169.857596] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516000worker2 # [ 169.862954] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6001worker1 # [ 161.638932] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516002worker1 # [ 161.643225] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6003worker2 # [ 170.064000] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516004worker2 # [ 170.069307] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6005worker1 # [ 161.839940] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516006worker1 # [ 161.845181] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6007worker2 # [ 170.270421] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516008worker2 # [ 170.276457] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6009worker1 # [ 162.046336] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516010worker1 # [ 162.051642] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6011worker1 # [ 162.211557] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:36Z level=info event=scheduler_restarting6012worker1 # [ 162.243183] nix-daemon[934]: reaped child process 1781, status = succeeded6013worker2 # [ 170.479572] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516014worker2 # [ 170.483861] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6015worker1 # [ 162.269736] nix-grpc-daemon[1730]: ts=2026-09-20T12:23:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6016worker2 # [ 170.680642] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_connected addr=lb:500516017worker2 # [ 170.685816] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6018worker2 # [ 170.886936] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516019worker2 # [ 170.892309] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6020worker2 # [ 171.093331] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516021worker2 # [ 171.098616] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6022worker1 # [ 163.020278] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6023worker1 # [ 163.026603] systemd[1]: nix-grpc-daemon.service: Consumed 901ms CPU time over 24.485s wall clock time, 32M memory peak, 25.8K incoming IP traffic, 33K outgoing IP traffic.6024worker1 # [ 163.050750] systemd[1]: Starting Nix worker-protocol over gRPC...6025worker2 # [ 171.299725] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516026worker2 # [ 171.305062] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6027worker1 # [ 163.254010] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6028worker1 # [ 163.256627] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6029worker1 # [ 163.260689] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16030worker1 # [ 163.268435] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6031worker1 # [ 163.270227] systemd[1]: Started Nix worker-protocol over gRPC.6032worker2 # [ 171.508156] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516033worker2 # [ 171.512462] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6034worker1 # [ 163.286668] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_take_over6035worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.10 seconds)6036lb # [ 171.568945] niks3-server[1093]: 2026/09/20 12:23:36 INFO lead: acquired remote=192.168.1.3:495686037lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6038lb: must succeed: curl -sf localhost:9901/clusters6039worker1 # [ 163.306404] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516040worker1 # [ 163.308261] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6041lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6042lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds)6043lb: must succeed: curl -sf localhost:9901/clusters6044lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6045worker2 # [ 171.709124] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_connected addr=lb:500516046worker2 # [ 171.714399] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6047worker1 # [ 163.509580] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516048worker1 # [ 163.515020] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6049worker2 # [ 171.915568] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516050worker2 # [ 171.921019] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6051worker1 # [ 163.715888] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516052worker1 # [ 163.721346] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6053worker2 # [ 172.121830] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516054worker2 # [ 172.127060] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6055worker1 # [ 163.922037] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=scheduler_connected addr=lb:500516056worker1 # [ 163.927450] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6057worker2 # [ 172.328220] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516058worker2 # [ 172.333296] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6059worker1 # [ 164.128383] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=scheduler_connected addr=lb:500516060worker1 # [ 164.133937] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6061worker2 # [ 172.535608] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:37Z level=info event=scheduler_connected addr=lb:500516062worker1 # [ 164.317205] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=oidc_jwks provider=mock keys=16063worker1 # [ 164.322481] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:535106064worker1 # [ 164.327828] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06065worker1 # [ 164.334437] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=scheduler_connected addr=lb:500516066worker1 # [ 164.339686] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:535106067worker1 # [ 164.344632] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06068lb: must succeed: curl -sf localhost:9901/clusters6069lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6070worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6071worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6072(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.47 seconds)6073subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6074client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16075worker1 # [ 164.861697] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535126076worker1 # [ 164.877115] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6077worker1 # [ 164.897076] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126078worker1 # [ 164.914193] nix-grpc-daemon[1935]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)6079worker1 # [ 164.940407] nix-grpc-daemon[1935]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 2/5)6080worker1 # [ 165.135088] nix-grpc-daemon[1935]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 3/5)6081worker1 # [ 165.278835] nix-grpc-daemon[1935]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 397 ms (attempt 4/5)6082lb # [ 173.779489] envoy[851]: conn peer=[2001:db8:1::1]:50568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=3898 ms=74216083lb # [ 173.784689] envoy[851]: conn peer=[2001:db8:1::1]:50560 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1351 ms=111226084lb # [ 173.790070] envoy[851]: conn peer=[2001:db8:1::3]:56510 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1426 tx=539 ms=232626085worker1 # [ 165.685426] nix-grpc-daemon[1935]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6086worker1 # [ 165.741890] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06087worker1 # [ 165.748942] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16088worker1 # [ 165.755300] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06089worker1 # [ 165.760754] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06090worker1 # [ 165.790278] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126091worker1 # [ 165.807014] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126092worker1 # [ 165.822109] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126093worker1 # [ 165.837106] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6094worker1 # [ 165.863165] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:39Z level=info event=niks3_push_started pid=19966095worker1 # [ 166.079020] nix-grpc-daemon[1996]: time=2026-09-20T12:23:40.188Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""6096worker1 # [ 166.258240] nix-daemon[934]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)6097worker1 # [ 166.280384] nix-daemon[934]: reaped child process 2012, status = succeeded6098worker1 # [ 166.461404] nix-daemon[934]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6099worker1 # [ 166.479908] nix-daemon[934]: reaped child process 2017, status = succeeded6100lb # [ 174.771192] niks3-server[1093]: 2026/09/20 12:23:39 INFO Received uploads request method=POST path=/api/pending_closures6101worker1 # [ 166.503183] nix-grpc-daemon[1996]: time=2026-09-20T12:23:40.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6102worker1 # [ 166.505053] nix-grpc-daemon[1996]: time=2026-09-20T12:23:40.612Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6103lb # [ 174.816676] niks3-server[1093]: 2026/09/20 12:23:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6104worker1 # [ 166.546697] nix-grpc-daemon[1996]: time=2026-09-20T12:23:40.656Z level=INFO msg="Uploading 1 narinfos"6105lb # [ 174.831590] niks3-server[1093]: 2026/09/20 12:23:39 INFO Signed narinfos id=46 count=16106lb # [ 174.843972] niks3-server[1093]: 2026/09/20 12:23:39 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6107lb # [ 174.855285] niks3-server[1093]: 2026/09/20 12:23:40 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6108worker1 # [ 166.582717] nix-grpc-daemon[1996]: time=2026-09-20T12:23:40.693Z level=INFO msg="Upload complete. (499ms)"6109lb # [ 174.864789] niks3-server[1093]: 2026/09/20 12:23:40 INFO Completed upload id=466110lb # [ 174.868510] niks3-server[1093]: 2026/09/20 12:23:40 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6111worker1 # [ 166.592279] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=5126112lb # [ 174.876709] niks3-server[1093]: 2026/09/20 12:23:40 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6113worker1 # [ 166.602017] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126114worker1 # [ 166.614627] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6115worker1 # [ 166.621284] nix-daemon[934]: reaped child process 1994, status = succeeded6116worker1 # [ 166.631998] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6117worker1 # [ 168.268873] nix-daemon[934]: reaped child process 2021, status = succeeded6118worker1 # [ 168.437395] nix-daemon[934]: accepted connection from pid 2036, user nix-grpc-daemon (trusted)6119worker1 # [ 168.456979] nix-daemon[934]: reaped child process 2039, status = succeeded6120lb # [ 176.749895] niks3-server[1093]: 2026/09/20 12:23:41 INFO Received uploads request method=POST path=/api/pending_closures6121worker1 # [ 168.480587] nix-grpc-daemon[1996]: time=2026-09-20T12:23:42.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6122worker1 # [ 168.482128] nix-grpc-daemon[1996]: time=2026-09-20T12:23:42.589Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6123lb # [ 176.782114] niks3-server[1093]: 2026/09/20 12:23:41 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6124lb # [ 176.803200] niks3-server[1093]: 2026/09/20 12:23:41 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6125worker1 # [ 168.536211] nix-grpc-daemon[1996]: time=2026-09-20T12:23:42.645Z level=INFO msg="Uploading 1 narinfos"6126lb # [ 176.816970] niks3-server[1093]: 2026/09/20 12:23:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6127lb # [ 176.823629] niks3-server[1093]: 2026/09/20 12:23:41 INFO Signed narinfos id=47 count=16128lb # [ 176.830317] niks3-server[1093]: 2026/09/20 12:23:41 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6129worker1 # [ 168.562177] nix-grpc-daemon[1996]: time=2026-09-20T12:23:42.672Z level=INFO msg="Upload complete. (312ms)"6130lb # [ 176.842976] niks3-server[1093]: 2026/09/20 12:23:41 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6131lb # [ 176.848679] niks3-server[1093]: 2026/09/20 12:23:41 INFO Completed upload id=476132lb # [ 176.852798] niks3-server[1093]: 2026/09/20 12:23:41 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6133worker1 # [ 168.581381] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16134worker1 # [ 168.595220] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16135worker1 # [ 168.604445] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126136worker1 # [ 168.607438] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06137worker1 # [ 168.613694] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126138worker1 # [ 168.618128] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126139worker1 # [ 168.629183] nix-daemon[934]: reaped child process 2019, status = succeeded6140worker1 # [ 168.633765] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6141worker1 # [ 168.814294] nix-daemon[934]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)6142worker1 # [ 168.834535] nix-daemon[934]: reaped child process 2048, status = succeeded6143worker1 # [ 169.009074] nix-daemon[934]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6144worker1 # [ 169.028135] nix-daemon[934]: reaped child process 2053, status = succeeded6145lb # [ 177.318037] niks3-server[1093]: 2026/09/20 12:23:42 INFO Received uploads request method=POST path=/api/pending_closures6146worker1 # [ 169.048141] nix-grpc-daemon[1996]: time=2026-09-20T12:23:43.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6147worker1 # [ 169.049690] nix-grpc-daemon[1996]: time=2026-09-20T12:23:43.158Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6148lb # [ 177.349156] niks3-server[1093]: 2026/09/20 12:23:42 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6149worker1 # [ 169.084409] nix-grpc-daemon[1996]: time=2026-09-20T12:23:43.194Z level=INFO msg="Uploading 1 narinfos"6150lb # [ 177.367190] niks3-server[1093]: 2026/09/20 12:23:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6151lb # [ 177.378079] niks3-server[1093]: 2026/09/20 12:23:42 INFO Signed narinfos id=48 count=16152worker1 # [ 169.107074] nix-grpc-daemon[1996]: time=2026-09-20T12:23:43.217Z level=INFO msg="Upload complete. (466ms)"6153lb # [ 177.382356] niks3-server[1093]: 2026/09/20 12:23:42 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6154worker1 # [ 169.112344] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=5126155lb # [ 177.393059] niks3-server[1093]: 2026/09/20 12:23:42 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6156lb # [ 177.397524] niks3-server[1093]: 2026/09/20 12:23:42 INFO Completed upload id=486157lb # [ 177.399678] niks3-server[1093]: 2026/09/20 12:23:42 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6158worker1 # [ 169.127116] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126159worker1 # [ 169.132776] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6160worker1 # [ 169.137842] nix-daemon[934]: reaped child process 2043, status = succeeded6161worker1 # [ 169.151302] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6162worker1 # [ 170.595519] nix-daemon[934]: reaped child process 2057, status = succeeded6163worker1 # [ 170.773952] nix-daemon[934]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)6164worker1 # [ 170.792161] nix-daemon[934]: reaped child process 2075, status = succeeded6165lb # [ 179.082898] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received uploads request method=POST path=/api/pending_closures6166worker1 # [ 170.815100] nix-grpc-daemon[1996]: time=2026-09-20T12:23:44.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6167worker1 # [ 170.816821] nix-grpc-daemon[1996]: time=2026-09-20T12:23:44.924Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6168lb # [ 179.125495] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6169lb # [ 179.136466] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6170lb # [ 179.148749] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6171worker1 # [ 170.870548] nix-grpc-daemon[1996]: time=2026-09-20T12:23:44.980Z level=INFO msg="Uploading 1 narinfos"6172lb # [ 179.155821] niks3-server[1093]: 2026/09/20 12:23:44 INFO Signed narinfos id=49 count=16173lb # [ 179.160456] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6174lb # [ 179.168877] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6175lb # [ 179.173254] niks3-server[1093]: 2026/09/20 12:23:44 INFO Completed upload id=496176worker1 # [ 170.893890] nix-grpc-daemon[1996]: time=2026-09-20T12:23:45.004Z level=INFO msg="Upload complete. (332ms)"6177lb # [ 179.175935] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6178worker1 # [ 170.911389] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16179worker1 # [ 170.929576] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16180worker1 # [ 170.939792] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126181worker1 # [ 170.944590] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06182worker1 # [ 170.948844] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126183worker1 # [ 170.953310] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126184worker1 # [ 170.963627] nix-daemon[934]: reaped child process 2055, status = succeeded6185worker1 # [ 170.967581] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6186worker1 # [ 171.153105] nix-daemon[934]: accepted connection from pid 2081, user nix-grpc-daemon (trusted)6187worker1 # [ 171.172301] nix-daemon[934]: reaped child process 2084, status = succeeded6188worker1 # [ 171.348855] nix-daemon[934]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6189worker1 # [ 171.367698] nix-daemon[934]: reaped child process 2089, status = succeeded6190lb # [ 179.657322] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received uploads request method=POST path=/api/pending_closures6191worker1 # [ 171.387224] nix-grpc-daemon[1996]: time=2026-09-20T12:23:45.497Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6192worker1 # [ 171.388908] nix-grpc-daemon[1996]: time=2026-09-20T12:23:45.497Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6193lb # [ 179.683376] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6194worker1 # [ 171.418614] nix-grpc-daemon[1996]: time=2026-09-20T12:23:45.528Z level=INFO msg="Uploading 1 narinfos"6195lb # [ 179.698238] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6196lb # [ 179.710940] niks3-server[1093]: 2026/09/20 12:23:44 INFO Signed narinfos id=50 count=16197lb # [ 179.717397] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6198lb # [ 179.724662] niks3-server[1093]: 2026/09/20 12:23:44 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6199lb # [ 179.736558] niks3-server[1093]: 2026/09/20 12:23:44 INFO Completed upload id=506200worker1 # [ 171.457414] nix-grpc-daemon[1996]: time=2026-09-20T12:23:45.567Z level=INFO msg="Upload complete. (483ms)"6201lb # [ 179.739785] niks3-server[1093]: 2026/09/20 12:23:44 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6202worker1 # [ 171.466347] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=5126203worker1 # [ 171.479703] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126204worker1 # [ 171.486076] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6205worker1 # [ 171.491601] nix-daemon[934]: reaped child process 2079, status = succeeded6206worker1 # [ 171.505341] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6207worker1 # [ 172.536312] nix-daemon[934]: reaped child process 2093, status = succeeded6208worker1 # [ 172.701330] nix-daemon[934]: accepted connection from pid 2108, user nix-grpc-daemon (trusted)6209worker1 # [ 172.721146] nix-daemon[934]: reaped child process 2111, status = succeeded6210lb # [ 181.017044] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received uploads request method=POST path=/api/pending_closures6211worker1 # [ 172.751304] nix-grpc-daemon[1996]: time=2026-09-20T12:23:46.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6212worker1 # [ 172.753000] nix-grpc-daemon[1996]: time=2026-09-20T12:23:46.860Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6213lb # [ 181.052906] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6214lb # [ 181.064175] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6215worker1 # [ 172.803548] nix-grpc-daemon[1996]: time=2026-09-20T12:23:46.913Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 181.077572] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6217lb # [ 181.085328] niks3-server[1093]: 2026/09/20 12:23:46 INFO Signed narinfos id=51 count=16218lb # [ 181.091041] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6219lb # [ 181.101904] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6220worker1 # [ 172.827989] nix-grpc-daemon[1996]: time=2026-09-20T12:23:46.938Z level=INFO msg="Upload complete. (312ms)"6221lb # [ 181.107735] niks3-server[1093]: 2026/09/20 12:23:46 INFO Completed upload id=516222lb # [ 181.110880] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6223worker1 # [ 172.843723] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16224worker1 # [ 172.865089] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:46Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16225worker1 # [ 172.869159] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126226worker1 # [ 172.874469] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:46Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06227worker1 # [ 172.877869] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126228worker1 # [ 172.882905] nix-daemon[934]: reaped child process 2091, status = succeeded6229worker1 # [ 172.886724] nix-grpc-daemon[1996]: time=2026-09-20T12:23:46.997Z level=INFO msg="All 1 paths already cached"6230worker1 # [ 172.894839] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126231worker1 # [ 172.905573] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6232worker1 # [ 173.098704] nix-daemon[934]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)6233worker1 # [ 173.119052] nix-daemon[934]: reaped child process 2120, status = succeeded6234worker1 # [ 173.296128] nix-daemon[934]: accepted connection from pid 2122, user nix-grpc-daemon (trusted)6235worker1 # [ 173.315803] nix-daemon[934]: reaped child process 2125, status = succeeded6236lb # [ 181.606431] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 173.336732] nix-grpc-daemon[1996]: time=2026-09-20T12:23:47.446Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6238worker1 # [ 173.338256] nix-grpc-daemon[1996]: time=2026-09-20T12:23:47.446Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6239lb # [ 181.635765] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6240worker1 # [ 173.370074] nix-grpc-daemon[1996]: time=2026-09-20T12:23:47.480Z level=INFO msg="Uploading 1 narinfos"6241lb # [ 181.650821] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6242lb # [ 181.659840] niks3-server[1093]: 2026/09/20 12:23:46 INFO Signed narinfos id=52 count=16243lb # [ 181.665784] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6244lb # [ 181.672821] niks3-server[1093]: 2026/09/20 12:23:46 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6245worker1 # [ 173.400444] nix-grpc-daemon[1996]: time=2026-09-20T12:23:47.510Z level=INFO msg="Upload complete. (482ms)"6246lb # [ 181.679311] niks3-server[1093]: 2026/09/20 12:23:46 INFO Completed upload id=526247lb # [ 181.682466] niks3-server[1093]: 2026/09/20 12:23:46 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6248worker1 # [ 173.408142] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=6246249worker1 # [ 173.420608] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126250worker1 # [ 173.427317] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6251worker1 # [ 173.432873] nix-daemon[934]: reaped child process 2115, status = succeeded6252worker1 # [ 173.446352] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6253worker1 # [ 174.680269] nix-daemon[934]: reaped child process 2129, status = succeeded6254worker1 # [ 174.865464] nix-daemon[934]: accepted connection from pid 2144, user nix-grpc-daemon (trusted)6255worker1 # [ 174.884223] nix-daemon[934]: reaped child process 2147, status = succeeded6256lb # [ 183.175696] niks3-server[1093]: 2026/09/20 12:23:48 INFO Received uploads request method=POST path=/api/pending_closures6257worker1 # [ 174.908447] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.016Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6258worker1 # [ 174.909980] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.016Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6259lb # [ 183.215355] niks3-server[1093]: 2026/09/20 12:23:48 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6260lb # [ 183.228943] niks3-server[1093]: 2026/09/20 12:23:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6261worker1 # [ 174.955379] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.065Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 183.238062] niks3-server[1093]: 2026/09/20 12:23:48 INFO Signed narinfos id=53 count=16263lb # [ 183.252563] niks3-server[1093]: 2026/09/20 12:23:48 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6264lb # [ 183.321788] niks3-server[1093]: 2026/09/20 12:23:48 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6265lb # [ 183.336422] niks3-server[1093]: 2026/09/20 12:23:48 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6266lb # [ 183.368127] niks3-server[1093]: 2026/09/20 12:23:48 INFO Completed upload id=536267worker1 # [ 175.089376] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.199Z level=INFO msg="Upload complete. (432ms)"6268lb # [ 183.371921] niks3-server[1093]: 2026/09/20 12:23:48 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6269worker1 # [ 175.109660] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16270worker1 # [ 175.133303] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16271worker1 # [ 175.138531] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126272worker1 # [ 175.145545] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06273worker1 # [ 175.151367] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126274worker1 # [ 175.155240] nix-daemon[934]: reaped child process 2127, status = succeeded6275worker1 # [ 175.158612] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.268Z level=INFO msg="All 1 paths already cached"6276worker1 # [ 175.162716] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126277worker1 # [ 175.177341] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6278worker1 # [ 175.375043] nix-daemon[934]: accepted connection from pid 2153, user nix-grpc-daemon (trusted)6279worker1 # [ 175.393711] nix-daemon[934]: reaped child process 2156, status = succeeded6280worker1 # [ 175.570777] nix-daemon[934]: accepted connection from pid 2158, user nix-grpc-daemon (trusted)6281worker1 # [ 175.589206] nix-daemon[934]: reaped child process 2161, status = succeeded6282lb # [ 183.879135] niks3-server[1093]: 2026/09/20 12:23:49 INFO Received uploads request method=POST path=/api/pending_closures6283worker1 # [ 175.611081] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.721Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6284worker1 # [ 175.612727] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.721Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6285lb # [ 183.908365] niks3-server[1093]: 2026/09/20 12:23:49 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6286lb # [ 183.922673] niks3-server[1093]: 2026/09/20 12:23:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6287worker1 # [ 175.650750] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.760Z level=INFO msg="Uploading 1 narinfos"6288lb # [ 183.934275] niks3-server[1093]: 2026/09/20 12:23:49 INFO Signed narinfos id=54 count=16289lb # [ 183.941711] niks3-server[1093]: 2026/09/20 12:23:49 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6290worker1 # [ 175.672263] nix-grpc-daemon[1996]: time=2026-09-20T12:23:49.782Z level=INFO msg="Upload complete. (480ms)"6291lb # [ 183.952487] niks3-server[1093]: 2026/09/20 12:23:49 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6292lb # [ 183.957245] niks3-server[1093]: 2026/09/20 12:23:49 INFO Completed upload id=546293worker1 # [ 175.678277] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=6246294lb # [ 183.960108] niks3-server[1093]: 2026/09/20 12:23:49 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6295worker1 # [ 175.692669] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126296worker1 # [ 175.699414] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6297worker1 # [ 175.704873] nix-daemon[934]: reaped child process 2151, status = succeeded6298worker1 # [ 175.718466] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6299worker1 # [ 176.755682] nix-daemon[934]: reaped child process 2165, status = succeeded6300worker1 # [ 176.927724] nix-daemon[934]: accepted connection from pid 2180, user nix-grpc-daemon (trusted)6301worker1 # [ 176.946506] nix-daemon[934]: reaped child process 2183, status = succeeded6302lb # [ 185.240804] niks3-server[1093]: 2026/09/20 12:23:50 INFO Received uploads request method=POST path=/api/pending_closures6303worker1 # [ 176.972439] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.081Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6304worker1 # [ 176.973971] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.081Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6305lb # [ 185.286453] niks3-server[1093]: 2026/09/20 12:23:50 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6306worker1 # [ 177.021067] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.130Z level=INFO msg="Uploading 1 narinfos"6307lb # [ 185.303293] niks3-server[1093]: 2026/09/20 12:23:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6308lb # [ 185.311917] niks3-server[1093]: 2026/09/20 12:23:50 INFO Signed narinfos id=55 count=16309lb # [ 185.319702] niks3-server[1093]: 2026/09/20 12:23:50 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6310lb # [ 185.323335] niks3-server[1093]: 2026/09/20 12:23:50 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6311worker1 # [ 177.047774] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.158Z level=INFO msg="Upload complete. (317ms)"6312lb # [ 185.327121] niks3-server[1093]: 2026/09/20 12:23:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6313lb # [ 185.330814] niks3-server[1093]: 2026/09/20 12:23:50 INFO Completed upload id=556314lb # [ 185.333651] niks3-server[1093]: 2026/09/20 12:23:50 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6315worker1 # [ 177.063363] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16316worker1 # [ 177.076458] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16317worker1 # [ 177.085998] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126318worker1 # [ 177.091344] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06319worker1 # [ 177.096822] nix-daemon[934]: reaped child process 2163, status = succeeded6320worker1 # [ 177.098626] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126321worker1 # [ 177.113443] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.223Z level=INFO msg="All 1 paths already cached"6322worker1 # [ 177.123078] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126323worker1 # [ 177.133954] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6324worker1 # [ 177.327954] nix-daemon[934]: accepted connection from pid 2189, user nix-grpc-daemon (trusted)6325worker1 # [ 177.349483] nix-daemon[934]: reaped child process 2192, status = succeeded6326worker1 # [ 177.527935] nix-daemon[934]: accepted connection from pid 2194, user nix-grpc-daemon (trusted)6327worker1 # [ 177.545868] nix-daemon[934]: reaped child process 2197, status = succeeded6328lb # [ 185.838087] niks3-server[1093]: 2026/09/20 12:23:50 INFO Received uploads request method=POST path=/api/pending_closures6329worker1 # [ 177.570879] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.680Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6330worker1 # [ 177.572599] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.680Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6331lb # [ 185.869018] niks3-server[1093]: 2026/09/20 12:23:51 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6332lb # [ 185.879383] niks3-server[1093]: 2026/09/20 12:23:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6333worker1 # [ 177.606020] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.716Z level=INFO msg="Uploading 1 narinfos"6334lb # [ 185.884970] niks3-server[1093]: 2026/09/20 12:23:51 INFO Signed narinfos id=56 count=16335lb # [ 185.888890] niks3-server[1093]: 2026/09/20 12:23:51 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6336lb # [ 185.898675] niks3-server[1093]: 2026/09/20 12:23:51 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6337lb # [ 185.903950] niks3-server[1093]: 2026/09/20 12:23:51 INFO Completed upload id=566338worker1 # [ 177.624431] nix-grpc-daemon[1996]: time=2026-09-20T12:23:51.734Z level=INFO msg="Upload complete. (477ms)"6339lb # [ 185.906240] niks3-server[1093]: 2026/09/20 12:23:51 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6340worker1 # [ 177.632221] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=6246341worker1 # [ 177.645688] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126342worker1 # [ 177.651387] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6343worker1 # [ 177.656251] nix-daemon[934]: reaped child process 2187, status = succeeded6344worker1 # [ 177.669089] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6345worker1 # [ 178.494518] nix-daemon[934]: reaped child process 2201, status = succeeded6346worker1 # [ 178.659830] nix-daemon[934]: accepted connection from pid 2216, user nix-grpc-daemon (trusted)6347worker1 # [ 178.681822] nix-daemon[934]: reaped child process 2219, status = succeeded6348lb # [ 186.972734] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received uploads request method=POST path=/api/pending_closures6349worker1 # [ 178.704227] nix-grpc-daemon[1996]: time=2026-09-20T12:23:52.814Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6350worker1 # [ 178.708850] nix-grpc-daemon[1996]: time=2026-09-20T12:23:52.816Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6351lb # [ 187.016663] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6352lb # [ 187.028418] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6353lb # [ 187.039385] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6354worker1 # [ 178.766018] nix-grpc-daemon[1996]: time=2026-09-20T12:23:52.876Z level=INFO msg="Uploading 1 narinfos"6355lb # [ 187.046414] niks3-server[1093]: 2026/09/20 12:23:52 INFO Signed narinfos id=57 count=16356lb # [ 187.051987] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6357lb # [ 187.063122] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6358lb # [ 187.083215] niks3-server[1093]: 2026/09/20 12:23:52 INFO Completed upload id=576359worker1 # [ 178.807449] nix-grpc-daemon[1996]: time=2026-09-20T12:23:52.917Z level=INFO msg="Upload complete. (329ms)"6360lb # [ 187.086721] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6361worker1 # [ 178.826252] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16362worker1 # [ 178.844388] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:52Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16363worker1 # [ 178.855888] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126364worker1 # [ 178.861023] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:52Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06365worker1 # [ 178.866026] nix-daemon[934]: reaped child process 2199, status = succeeded6366worker1 # [ 178.870030] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126367worker1 # [ 178.884230] nix-grpc-daemon[1996]: time=2026-09-20T12:23:52.994Z level=INFO msg="All 2 paths already cached"6368worker1 # [ 178.894034] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126369worker1 # [ 178.905832] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6370worker1 # [ 179.102717] nix-daemon[934]: accepted connection from pid 2225, user nix-grpc-daemon (trusted)6371worker1 # [ 179.123939] nix-daemon[934]: reaped child process 2228, status = succeeded6372worker1 # [ 179.297912] nix-daemon[934]: accepted connection from pid 2230, user nix-grpc-daemon (trusted)6373worker1 # [ 179.317718] nix-daemon[934]: reaped child process 2233, status = succeeded6374lb # [ 187.610689] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received uploads request method=POST path=/api/pending_closures6375worker1 # [ 179.439568] nix-grpc-daemon[1996]: time=2026-09-20T12:23:53.549Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6376worker1 # [ 179.444594] nix-grpc-daemon[1996]: time=2026-09-20T12:23:53.549Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6377lb # [ 187.743988] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6378lb # [ 187.752560] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6379lb # [ 187.763190] niks3-server[1093]: 2026/09/20 12:23:52 INFO Signed narinfos id=58 count=16380worker1 # [ 179.484408] nix-grpc-daemon[1996]: time=2026-09-20T12:23:53.594Z level=INFO msg="Uploading 1 narinfos"6381lb # [ 187.771185] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6382lb # [ 187.785365] niks3-server[1093]: 2026/09/20 12:23:52 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6383lb # [ 187.795262] niks3-server[1093]: 2026/09/20 12:23:52 INFO Completed upload id=586384worker1 # [ 179.516437] nix-grpc-daemon[1996]: time=2026-09-20T12:23:53.626Z level=INFO msg="Upload complete. (595ms)"6385lb # [ 187.799105] niks3-server[1093]: 2026/09/20 12:23:52 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6386worker1 # [ 179.529547] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=7526387worker1 # [ 179.544525] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126388worker1 # [ 179.558984] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6389worker1 # [ 179.567226] nix-daemon[934]: reaped child process 2223, status = succeeded6390worker1 # [ 179.584181] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6391worker1 # [ 180.437691] nix-daemon[934]: reaped child process 2237, status = succeeded6392worker1 # [ 180.616450] nix-daemon[934]: accepted connection from pid 2252, user nix-grpc-daemon (trusted)6393worker1 # [ 180.638022] nix-daemon[934]: reaped child process 2255, status = succeeded6394lb # [ 188.929691] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received uploads request method=POST path=/api/pending_closures6395worker1 # [ 180.664579] nix-grpc-daemon[1996]: time=2026-09-20T12:23:54.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6396worker1 # [ 180.666719] nix-grpc-daemon[1996]: time=2026-09-20T12:23:54.771Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6397lb # [ 188.972160] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6398worker1 # [ 180.704534] nix-grpc-daemon[1996]: time=2026-09-20T12:23:54.814Z level=INFO msg="Uploading 1 narinfos"6399lb # [ 188.989523] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6400lb # [ 188.998875] niks3-server[1093]: 2026/09/20 12:23:54 INFO Signed narinfos id=59 count=16401lb # [ 189.003471] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6402worker1 # [ 180.730935] nix-grpc-daemon[1996]: time=2026-09-20T12:23:54.841Z level=INFO msg="Upload complete. (329ms)"6403lb # [ 189.010962] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6404lb # [ 189.014662] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6405lb # [ 189.018871] niks3-server[1093]: 2026/09/20 12:23:54 INFO Completed upload id=596406lb # [ 189.021282] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6407worker1 # [ 180.747178] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16408worker1 # [ 180.761058] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:54Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16409worker1 # [ 180.772572] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126410worker1 # [ 180.775672] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126411worker1 # [ 180.780353] nix-daemon[934]: reaped child process 2235, status = succeeded6412worker1 # [ 180.788471] nix-grpc-daemon[1996]: time=2026-09-20T12:23:54.898Z level=INFO msg="All 2 paths already cached"6413worker1 # [ 180.796125] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126414worker1 # [ 180.808120] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6415worker1 # [ 181.009171] nix-daemon[934]: accepted connection from pid 2259, user nix-grpc-daemon (trusted)6416worker1 # [ 181.028372] nix-daemon[934]: reaped child process 2262, status = succeeded6417worker1 # [ 181.210805] nix-daemon[934]: accepted connection from pid 2264, user nix-grpc-daemon (trusted)6418worker1 # [ 181.230074] nix-daemon[934]: reaped child process 2267, status = succeeded6419lb # [ 189.521951] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received uploads request method=POST path=/api/pending_closures6420worker1 # [ 181.252910] nix-grpc-daemon[1996]: time=2026-09-20T12:23:55.362Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6421worker1 # [ 181.254457] nix-grpc-daemon[1996]: time=2026-09-20T12:23:55.363Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6422lb # [ 189.556570] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6423worker1 # [ 181.287583] nix-grpc-daemon[1996]: time=2026-09-20T12:23:55.397Z level=INFO msg="Uploading 1 narinfos"6424lb # [ 189.571012] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6425worker1 # [ 181.305789] nix-grpc-daemon[1996]: time=2026-09-20T12:23:55.416Z level=INFO msg="Upload complete. (483ms)"6426lb # [ 189.583960] niks3-server[1093]: 2026/09/20 12:23:54 INFO Signed narinfos id=60 count=16427lb # [ 189.587491] niks3-server[1093]: 2026/09/20 12:23:54 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6428worker1 # [ 181.312322] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=7526429lb # [ 189.593427] niks3-server[1093]: 2026/09/20 12:23:54 INFO Completed upload id=606430lb # [ 189.597685] niks3-server[1093]: 2026/09/20 12:23:54 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6431worker1 # [ 181.327371] nix-grpc-daemon[1996]: time=2026-09-20T12:23:55.427Z level=WARN msg="Failed to register uploaded object" key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo error="server returned 404: object is not pending upload\n"6432worker1 # [ 181.333354] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126433worker1 # [ 181.340632] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6434worker1 # [ 181.343148] nix-daemon[934]: reaped child process 2257, status = succeeded6435worker1 # [ 181.353475] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6436worker1 # [ 182.411159] nix-daemon[934]: reaped child process 2271, status = succeeded6437worker1 # [ 182.589370] nix-daemon[934]: accepted connection from pid 2286, user nix-grpc-daemon (trusted)6438worker1 # [ 182.609157] nix-daemon[934]: reaped child process 2289, status = succeeded6439lb # [ 190.901697] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received uploads request method=POST path=/api/pending_closures6440worker1 # [ 182.634272] nix-grpc-daemon[1996]: time=2026-09-20T12:23:56.742Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6441worker1 # [ 182.636312] nix-grpc-daemon[1996]: time=2026-09-20T12:23:56.742Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6442lb # [ 190.934465] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6443lb # [ 190.951532] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6444worker1 # [ 182.691953] nix-grpc-daemon[1996]: time=2026-09-20T12:23:56.801Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 190.970243] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6446lb # [ 190.980354] niks3-server[1093]: 2026/09/20 12:23:56 INFO Signed narinfos id=61 count=16447lb # [ 190.991136] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6448lb # [ 191.000127] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6449lb # [ 191.004750] niks3-server[1093]: 2026/09/20 12:23:56 INFO Completed upload id=616450worker1 # [ 182.726003] nix-grpc-daemon[1996]: time=2026-09-20T12:23:56.836Z level=INFO msg="Upload complete. (355ms)"6451lb # [ 191.007481] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6452worker1 # [ 182.741321] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16453worker1 # [ 182.760428] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:56Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16454worker1 # [ 182.771535] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126455worker1 # [ 182.775725] nix-daemon[934]: reaped child process 2269, status = succeeded6456worker1 # [ 182.786153] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126457worker1 # [ 182.801262] nix-grpc-daemon[1996]: time=2026-09-20T12:23:56.911Z level=INFO msg="All 2 paths already cached"6458worker1 # [ 182.811826] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535126459worker1 # [ 182.827509] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6460worker1 # [ 183.042389] nix-daemon[934]: accepted connection from pid 2293, user nix-grpc-daemon (trusted)6461worker1 # [ 183.062784] nix-daemon[934]: reaped child process 2296, status = succeeded6462worker1 # [ 183.244214] nix-daemon[934]: accepted connection from pid 2298, user nix-grpc-daemon (trusted)6463worker1 # [ 183.262687] nix-daemon[934]: reaped child process 2301, status = succeeded6464lb # [ 191.555712] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received uploads request method=POST path=/api/pending_closures6465worker1 # [ 183.287398] nix-grpc-daemon[1996]: time=2026-09-20T12:23:57.397Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6466worker1 # [ 183.289079] nix-grpc-daemon[1996]: time=2026-09-20T12:23:57.397Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6467lb # [ 191.586119] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6468worker1 # [ 183.326206] nix-grpc-daemon[1996]: time=2026-09-20T12:23:57.436Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 191.598457] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6470lb # [ 191.609247] niks3-server[1093]: 2026/09/20 12:23:56 INFO Signed narinfos id=62 count=16471lb # [ 191.616836] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6472lb # [ 191.623906] niks3-server[1093]: 2026/09/20 12:23:56 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6473lb # [ 191.630348] niks3-server[1093]: 2026/09/20 12:23:56 INFO Completed upload id=626474worker1 # [ 183.351721] nix-grpc-daemon[1996]: time=2026-09-20T12:23:57.462Z level=INFO msg="Upload complete. (505ms)"6475lb # [ 191.633102] niks3-server[1093]: 2026/09/20 12:23:56 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6476worker1 # [ 183.360304] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=0 paths=1 nar_bytes_in=7526477worker1 # [ 183.368702] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535126478worker1 # [ 183.378951] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6479worker1 # [ 183.383722] nix-daemon[934]: reaped child process 2291, status = succeeded6480worker1 # [ 183.397577] nix-daemon[934]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6481worker1 # [ 184.846266] nix-daemon[934]: reaped child process 2305, status = succeeded6482worker1 # [ 185.025081] nix-daemon[934]: accepted connection from pid 2320, user nix-grpc-daemon (trusted)6483worker1 # [ 185.045513] nix-daemon[934]: reaped child process 2323, status = succeeded6484lb # [ 193.340609] niks3-server[1093]: 2026/09/20 12:23:58 INFO Received uploads request method=POST path=/api/pending_closures6485worker1 # [ 185.076483] nix-grpc-daemon[1996]: time=2026-09-20T12:23:59.184Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6486worker1 # [ 185.078032] nix-grpc-daemon[1996]: time=2026-09-20T12:23:59.184Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6487lb # [ 193.388862] niks3-server[1093]: 2026/09/20 12:23:58 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6488lb # [ 193.399522] niks3-server[1093]: 2026/09/20 12:23:58 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6489lb # [ 193.407991] niks3-server[1093]: 2026/09/20 12:23:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6490worker1 # [ 185.134006] nix-grpc-daemon[1996]: time=2026-09-20T12:23:59.244Z level=INFO msg="Uploading 1 narinfos"6491lb # [ 193.416126] niks3-server[1093]: 2026/09/20 12:23:58 INFO Signed narinfos id=63 count=16492lb # [ 193.421423] niks3-server[1093]: 2026/09/20 12:23:58 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6493worker1 # [ 185.154143] nix-grpc-daemon[1996]: time=2026-09-20T12:23:59.264Z level=INFO msg="Upload complete. (345ms)"6494lb # [ 193.434083] niks3-server[1093]: 2026/09/20 12:23:58 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6495lb # [ 193.438412] niks3-server[1093]: 2026/09/20 12:23:58 INFO Completed upload id=636496lb # [ 193.442906] niks3-server[1093]: 2026/09/20 12:23:58 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6497worker1 # [ 185.169691] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53512 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16498worker1 # [ 185.185268] nix-daemon[934]: reaped child process 2303, status = succeeded6499client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.64 seconds)6500worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6501worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6502client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16503lb # [ 193.779680] envoy[851]: conn peer=[2001:db8:1::1]:57780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27144 tx=10676 ms=203316504worker1 # [ 185.585687] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535126505worker2 # [ 193.831801] nix-grpc-daemon[1222]: ts=2026-09-20T12:23:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479706506worker1 # [ 185.622691] nix-grpc-daemon[1935]: ts=2026-09-20T12:23:59Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06507client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.34 seconds)6508(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.06 seconds)6509subtest: low disk drains a worker and builds go to the other6510worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6511worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)6512worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6513worker1 # [ 217.313752] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:31Z level=info event=unhealthy reason=min_free6514worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.16 seconds)6515client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26516client # don't know how to build these paths:6517client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6518worker1 # [ 218.230066] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535126519worker1 # [ 218.238164] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535126520worker1 # [ 218.262898] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:32Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06521worker1 # [ 218.274824] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:32Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16522worker2 # [ 226.546070] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706523worker2 # [ 226.564639] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479706524client # copying 1 paths to 'lb:50051'...6525client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6526worker2 # [ 226.582502] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479706527worker2 # [ 226.599660] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6528worker2 # [ 226.825235] nix-daemon[938]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6529worker2 # [ 226.845126] nix-daemon[938]: reaped child process 1540, status = succeeded6530worker2 # [ 227.038732] nix-daemon[938]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6531worker2 # [ 227.058429] nix-daemon[938]: reaped child process 1545, status = succeeded6532lb # [ 227.120736] niks3-server[1093]: 2026/09/20 12:24:32 INFO Received uploads request method=POST path=/api/pending_closures6533worker2 # [ 227.083545] nix-grpc-daemon[1360]: time=2026-09-20T12:24:32.341Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker2 # [ 227.085257] nix-grpc-daemon[1360]: time=2026-09-20T12:24:32.341Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6535lb # [ 227.155675] niks3-server[1093]: 2026/09/20 12:24:32 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6536lb # [ 227.177075] niks3-server[1093]: 2026/09/20 12:24:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6537worker2 # [ 227.137405] nix-grpc-daemon[1360]: time=2026-09-20T12:24:32.396Z level=INFO msg="Uploading 1 narinfos"6538lb # [ 227.185856] niks3-server[1093]: 2026/09/20 12:24:32 INFO Signed narinfos id=64 count=16539lb # [ 227.194468] niks3-server[1093]: 2026/09/20 12:24:32 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6540lb # [ 227.207627] niks3-server[1093]: 2026/09/20 12:24:32 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6541worker2 # [ 227.167297] nix-grpc-daemon[1360]: time=2026-09-20T12:24:32.426Z level=INFO msg="Upload complete. (547ms)"6542lb # [ 227.216180] niks3-server[1093]: 2026/09/20 12:24:32 INFO Completed upload id=646543lb # [ 227.221946] niks3-server[1093]: 2026/09/20 12:24:32 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6544worker2 # [ 227.180672] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=5126545worker2 # [ 227.192610] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706546worker2 # [ 227.205116] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6547worker2 # [ 227.210677] nix-daemon[938]: reaped child process 1535, status = succeeded6548client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6549client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6550worker2 # [ 227.226786] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6551client # farm-a> LOG-farm-a-drain6552worker2 # [ 228.457565] nix-daemon[938]: reaped child process 1549, status = succeeded6553worker2 # [ 228.647640] nix-daemon[938]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6554worker2 # [ 228.668537] nix-daemon[938]: reaped child process 1567, status = succeeded6555lb # [ 228.732796] niks3-server[1093]: 2026/09/20 12:24:33 INFO Received uploads request method=POST path=/api/pending_closures6556worker2 # [ 228.700516] nix-grpc-daemon[1360]: time=2026-09-20T12:24:33.956Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6557worker2 # [ 228.702065] nix-grpc-daemon[1360]: time=2026-09-20T12:24:33.956Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6558lb # [ 228.771709] niks3-server[1093]: 2026/09/20 12:24:33 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6559lb # [ 228.779891] niks3-server[1093]: 2026/09/20 12:24:33 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6560lb # [ 228.790781] niks3-server[1093]: 2026/09/20 12:24:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6561worker2 # [ 228.748123] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.006Z level=INFO msg="Uploading 1 narinfos"6562lb # [ 228.796636] niks3-server[1093]: 2026/09/20 12:24:33 INFO Signed narinfos id=65 count=16563lb # [ 228.801344] niks3-server[1093]: 2026/09/20 12:24:33 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6564lb # [ 228.814872] niks3-server[1093]: 2026/09/20 12:24:33 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6565lb # [ 228.820588] niks3-server[1093]: 2026/09/20 12:24:33 INFO Completed upload id=656566worker2 # [ 228.774450] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.033Z level=INFO msg="Upload complete. (334ms)"6567lb # [ 228.824763] niks3-server[1093]: 2026/09/20 12:24:33 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6568worker2 # [ 228.788885] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16569worker1 # [ 220.563972] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:34Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06570worker1 # [ 220.570755] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:34Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16571worker2 # [ 228.807330] nix-daemon[938]: reaped child process 1547, status = succeeded6572worker2 # [ 228.844284] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706573worker2 # [ 228.863991] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479706574worker2 # [ 228.880681] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.139Z level=INFO msg="All 1 paths already cached"6575client # copying 1 paths to 'lb:50051'...6576client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6577worker2 # [ 228.893997] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479706578worker2 # [ 228.910931] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6579worker2 # [ 229.176162] nix-daemon[938]: accepted connection from pid 1571, user nix-grpc-daemon (trusted)6580worker2 # [ 229.199725] nix-daemon[938]: reaped child process 1574, status = succeeded6581worker2 # [ 229.394561] nix-daemon[938]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6582worker2 # [ 229.418300] nix-daemon[938]: reaped child process 1579, status = succeeded6583lb # [ 229.475013] niks3-server[1093]: 2026/09/20 12:24:34 INFO Received uploads request method=POST path=/api/pending_closures6584worker2 # [ 229.436938] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.695Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6585worker2 # [ 229.438471] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.695Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6586lb # [ 229.500953] niks3-server[1093]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6587lb # [ 229.509889] niks3-server[1093]: 2026/09/20 12:24:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6588worker2 # [ 229.469087] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.728Z level=INFO msg="Uploading 1 narinfos"6589lb # [ 229.515426] niks3-server[1093]: 2026/09/20 12:24:34 INFO Signed narinfos id=66 count=16590lb # [ 229.519393] niks3-server[1093]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6591lb # [ 229.531577] niks3-server[1093]: 2026/09/20 12:24:34 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6592worker2 # [ 229.490312] nix-grpc-daemon[1360]: time=2026-09-20T12:24:34.749Z level=INFO msg="Upload complete. (556ms)"6593lb # [ 229.537304] niks3-server[1093]: 2026/09/20 12:24:34 INFO Completed upload id=666594lb # [ 229.540997] niks3-server[1093]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6595worker2 # [ 229.496256] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=6326596worker2 # [ 229.504593] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706597worker2 # [ 229.518092] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6598worker2 # [ 229.520366] nix-daemon[938]: reaped child process 1569, status = succeeded6599client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6600worker2 # [ 229.533560] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6601client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6602client # farm-b> LOG-farm-b-drain6603worker2 # [ 230.598171] nix-daemon[938]: reaped child process 1583, status = succeeded6604worker2 # [ 230.768472] nix-daemon[938]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)6605worker2 # [ 230.788326] nix-daemon[938]: reaped child process 1601, status = succeeded6606lb # [ 230.846749] niks3-server[1093]: 2026/09/20 12:24:35 INFO Received uploads request method=POST path=/api/pending_closures6607worker2 # [ 230.812390] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.069Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6608worker2 # [ 230.813915] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.069Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6609lb # [ 230.888235] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6610lb # [ 230.901884] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6611lb # [ 230.912223] niks3-server[1093]: 2026/09/20 12:24:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6612worker2 # [ 230.872870] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.131Z level=INFO msg="Uploading 1 narinfos"6613lb # [ 230.918526] niks3-server[1093]: 2026/09/20 12:24:36 INFO Signed narinfos id=67 count=16614lb # [ 230.923209] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6615lb # [ 230.933890] niks3-server[1093]: 2026/09/20 12:24:36 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6616worker2 # [ 230.893508] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.152Z level=INFO msg="Upload complete. (348ms)"6617lb # [ 230.941162] niks3-server[1093]: 2026/09/20 12:24:36 INFO Completed upload id=676618lb # [ 230.944188] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6619worker2 # [ 230.905108] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16620worker1 # [ 222.681193] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:36Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06621worker1 # [ 222.681622] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:36Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16622worker2 # [ 230.923305] nix-daemon[938]: reaped child process 1581, status = succeeded6623worker2 # [ 230.958004] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706624worker2 # [ 230.975033] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479706625worker2 # [ 230.990334] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.249Z level=INFO msg="All 2 paths already cached"6626client # copying 1 paths to 'lb:50051'...6627client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6628worker2 # [ 231.000997] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479706629worker2 # [ 231.014881] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6630worker2 # [ 231.215555] nix-daemon[938]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)6631worker2 # [ 231.235684] nix-daemon[938]: reaped child process 1608, status = succeeded6632worker2 # [ 231.407342] nix-daemon[938]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)6633worker2 # [ 231.426333] nix-daemon[938]: reaped child process 1613, status = succeeded6634lb # [ 231.483785] niks3-server[1093]: 2026/09/20 12:24:36 INFO Received uploads request method=POST path=/api/pending_closures6635worker2 # [ 231.446966] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.705Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6636worker2 # [ 231.448673] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.705Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6637worker2 # [ 231.467264] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.726Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 231.513156] niks3-server[1093]: 2026/09/20 12:24:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6639lb # [ 231.522847] niks3-server[1093]: 2026/09/20 12:24:36 INFO Signed narinfos id=68 count=16640lb # [ 231.533406] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6641lb # [ 231.542020] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6642lb # [ 231.550941] niks3-server[1093]: 2026/09/20 12:24:36 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6643worker2 # [ 231.509130] nix-grpc-daemon[1360]: time=2026-09-20T12:24:36.768Z level=INFO msg="Upload complete. (477ms)"6644lb # [ 231.555882] niks3-server[1093]: 2026/09/20 12:24:36 INFO Completed upload id=686645lb # [ 231.559926] niks3-server[1093]: 2026/09/20 12:24:36 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6646worker2 # [ 231.516326] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=0 paths=1 nar_bytes_in=7606647worker2 # [ 231.524742] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479706648worker2 # [ 231.534166] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6649worker2 # [ 231.538605] nix-daemon[938]: reaped child process 1603, status = succeeded6650client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6651worker2 # [ 231.551314] nix-daemon[938]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)6652client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6653client # farm-top> LOG-farm-top-drain6654worker2 # [ 232.597538] nix-daemon[938]: reaped child process 1617, status = succeeded6655worker2 # [ 232.776881] nix-daemon[938]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6656worker2 # [ 232.794946] nix-daemon[938]: reaped child process 1635, status = succeeded6657lb # [ 232.855469] niks3-server[1093]: 2026/09/20 12:24:38 INFO Received uploads request method=POST path=/api/pending_closures6658worker2 # [ 232.819788] nix-grpc-daemon[1360]: time=2026-09-20T12:24:38.077Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6659worker2 # [ 232.821551] nix-grpc-daemon[1360]: time=2026-09-20T12:24:38.077Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6660lb # [ 232.891137] niks3-server[1093]: 2026/09/20 12:24:38 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6661worker2 # [ 232.861648] nix-grpc-daemon[1360]: time=2026-09-20T12:24:38.120Z level=INFO msg="Uploading 1 narinfos"6662lb # [ 232.907331] niks3-server[1093]: 2026/09/20 12:24:38 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6663lb # [ 232.916304] niks3-server[1093]: 2026/09/20 12:24:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6664lb # [ 232.919881] niks3-server[1093]: 2026/09/20 12:24:38 INFO Signed narinfos id=69 count=16665lb # [ 232.929379] niks3-server[1093]: 2026/09/20 12:24:38 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6666lb # [ 232.932711] niks3-server[1093]: 2026/09/20 12:24:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6667lb # [ 232.936460] niks3-server[1093]: 2026/09/20 12:24:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6668lb # [ 232.940187] niks3-server[1093]: 2026/09/20 12:24:38 INFO Completed upload id=696669worker2 # [ 232.894337] nix-grpc-daemon[1360]: time=2026-09-20T12:24:38.153Z level=INFO msg="Upload complete. (332ms)"6670worker2 # [ 232.905105] nix-grpc-daemon[1222]: ts=2026-09-20T12:24:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47970 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16671worker2 # [ 232.922170] nix-daemon[938]: reaped child process 1615, status = succeeded6672client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.81 seconds)6673client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6674client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6675worker1: must succeed: rm /nix/.rw-store/fill6676worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6677worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6678worker1 # [ 225.316470] nix-grpc-daemon[1935]: ts=2026-09-20T12:24:39Z level=info event=healthy reason=min_free6679lb # [ 233.778832] envoy[851]: conn peer=[2001:db8:1::1]:34698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300576680lb # [ 233.784097] envoy[851]: conn peer=[2001:db8:1::1]:49572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=3881 ms=65016681worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.13 seconds)6682(finished: subtest: low disk drains a worker and builds go to the other, in 10.73 seconds)6683(finished: run the VM test script, in 236.29 seconds)6684test script finished in 236.59s6685cleanup6686kill QemuMachine (pid 49)6687client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6688kill QemuMachine (pid 51)6689lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6690kill QemuMachine (pid 52)6691worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6692kill QemuMachine (pid 50)6693worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6694(finished: cleanup, in 1.24 seconds)6695additionally exposed symbols:6696 client, lb, worker1, worker2,6697 vlan1,6698 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh