vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #473
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14client # Formatting '/build/vm-state-client/tmp.54WwShvgWr', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16lb: QEMU running (pid 50)17client # Discarding device blocks: 0/262144 done18lb # Disk image does not exist, creating the virtualisation disk image...19lb # Formatting '/build/vm-state-lb/tmp.0DSgKUmmbQ', fmt=raw size=107374182420client # Creating filesystem with 262144 4k blocks and 65536 inodes21lb # mke2fs 1.47.4 (6-Mar-2025)22client # Filesystem UUID: b7150ad9-3f38-4d9a-bad3-d91924cea87a23lb # Discarding device blocks: 0/262144 done24worker1 # Disk image does not exist, creating the virtualisation disk image...25worker1: QEMU running (pid 51)26client # Superblock backups stored on blocks:27worker1 # Formatting '/build/vm-state-worker1/tmp.X5x7BVmtWi', fmt=raw size=107374182428lb # Creating filesystem with 262144 4k blocks and 65536 inodes29lb # Filesystem UUID: 456f51ef-4fe2-4291-a68d-213e15070be530worker1 # mke2fs 1.47.4 (6-Mar-2025)31client # 32768, 98304, 163840, 22937632worker2: QEMU running (pid 52)33worker2 # Disk image does not exist, creating the virtualisation disk image...34worker1 # Discarding device blocks: 0/262144 done35lb # Superblock backups stored on blocks:36worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes37(finished: start all VMs, in 3.91 seconds)38client # 39worker1 # Filesystem UUID: 294d8c26-b609-45e4-ba9d-cdbfb2f7ce9740client: waiting for unit network-addresses-eth1.service41client # Allocating group tables: 0/8 done42worker1 # Superblock backups stored on blocks:43client: waiting for the VM to finish booting44client # Writing inode tables: 0/8 done45worker1 # 32768, 98304, 163840, 22937646client # Creating journal (8192 blocks): done47worker1 # 48lb # 32768, 98304, 163840, 22937649worker2 # Formatting '/build/vm-state-worker2/tmp.XwkIzwkALC', fmt=raw size=107374182450client # Writing superblocks and filesystem accounting information: 0/8 done51worker1 # Allocating group tables: 0/8 done52lb # 53worker2 # mke2fs 1.47.4 (6-Mar-2025)54client # 55worker1 # Writing inode tables: 0/8 done56lb # Allocating group tables: 0/8 done57worker2 # Discarding device blocks: 0/262144 done58client # Virtualisation disk image created.59worker1 # Creating journal (8192 blocks): done60lb # Writing inode tables: 0/8 done61worker1 # Writing superblocks and filesystem accounting information: 0/8 done62lb # Creating journal (8192 blocks): done63worker1 # 64lb # Writing superblocks and filesystem accounting information: 0/8 done65worker1 # Virtualisation disk image created.66lb # 67worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes68lb # Virtualisation disk image created.69worker2 # Filesystem UUID: b68673fc-7eaf-4c9a-8591-acdd17f6d05970worker2 # Superblock backups stored on blocks:71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681client # [ 0.000000] KASLR enabled82client # [ 0.000000] random: crng init done83client # [ 0.000000] Machine model: linux,dummy-virt84client # [ 0.000000] efi: UEFI not found.85client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]87client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]88client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]89client # [ 0.000000] Zone ranges:90lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202691client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]92lb # [ 0.000000] KASLR enabled93client # [ 0.000000] DMA32 empty94lb # [ 0.000000] random: crng init done95client # [ 0.000000] Normal empty96lb # [ 0.000000] Machine model: linux,dummy-virt97client # [ 0.000000] Device empty98lb # [ 0.000000] efi: UEFI not found.99client # [ 0.000000] Movable zone start for each node100client # [ 0.000000] Early memory node ranges101lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT102client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]103lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]104client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]105lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]106lb # [ 0.000000] Zone ranges:107client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000108client # [ 0.000000] psci: probing for conduit method from DT.109lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]110lb # [ 0.000000] DMA32 empty111client # [ 0.000000] psci: PSCIv1.3 detected in firmware.112lb # [ 0.000000] Normal empty113lb # [ 0.000000] Device empty114client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs115lb # [ 0.000000] Movable zone start for each node116client # [ 0.000000] psci: Trusted OS migration not required117lb # [ 0.000000] Early memory node ranges118client # [ 0.000000] psci: SMC Calling Convention v1.1119lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]120lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]121lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000122client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)123client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296124client # [ 0.000000] Detected PIPT I-cache on CPU0125lb # [ 0.000000] psci: probing for conduit method from DT.126lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.127client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)128lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs129client # [ 0.000000] CPU features: detected: GICv3 CPU interface130client # [ 0.000000] CPU features: detected: Spectre-v4131lb # [ 0.000000] psci: Trusted OS migration not required132client # [ 0.000000] CPU features: detected: Spectre-BHB133lb # [ 0.000000] psci: SMC Calling Convention v1.1134client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38135lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)136client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23137lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296138client # [ 0.000000] alternatives: applying boot alternatives139lb # [ 0.000000] Detected PIPT I-cache on CPU0140lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)141lb # [ 0.000000] CPU features: detected: GICv3 CPU interface142lb # [ 0.000000] CPU features: detected: Spectre-v4143lb # [ 0.000000] CPU features: detected: Spectre-BHB144lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38145client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test/init regInfo=/nix/store/pdj8wc854xn099x1vdf9pc3a2nsrcvf7-closure-info/registration console=ttyAMA0,115200n8 console=tty0146lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23147lb # [ 0.000000] alternatives: applying boot alternatives148client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pdj8wc854xn099x1vdf9pc3a2nsrcvf7-closure-info/registration", will be passed to user space.149client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes150client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)151client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)152lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f4b0lk4znyrmdfpm6f1kvcry9mqdiq05-nixos-system-lb-test/init regInfo=/nix/store/lvhvpzdlnfy9xyrdr3lx689a20qywaqb-closure-info/registration console=ttyAMA0,115200n8 console=tty0153client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB154client # [ 0.000000] software IO TLB: area num 1.155client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB156lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lvhvpzdlnfy9xyrdr3lx689a20qywaqb-closure-info/registration", will be passed to user space.157client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)158lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes159client # [ 0.000000] Fallback order for Node 0: 0160lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)161client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216162client # [ 0.000000] Policy zone: DMA163lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)164client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off165lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB166client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1167lb # [ 0.000000] software IO TLB: area num 1.168client # [ 0.000000] allocated 3145728 bytes of page_ext169lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB170client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages171client # [ 0.000000] ftrace: allocated 294 pages with 4 groups172lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)173client # [ 0.000000] rcu: Hierarchical RCU implementation.174lb # [ 0.000000] Fallback order for Node 0: 0175client # [ 0.000000] rcu: RCU event tracing is enabled.176lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216177lb # [ 0.000000] Policy zone: DMA178client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.179client # [ 0.000000] Trampoline variant of Tasks RCU enabled.180lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off181client # [ 0.000000] Rude variant of Tasks RCU enabled.182lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1183client # [ 0.000000] Tracing variant of Tasks RCU enabled.184lb # [ 0.000000] allocated 3145728 bytes of page_ext185client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.186lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages187lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups188client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1189lb # [ 0.000000] rcu: Hierarchical RCU implementation.190lb # [ 0.000000] rcu: RCU event tracing is enabled.191client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.193client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.194lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.195lb # [ 0.000000] Rude variant of Tasks RCU enabled.196client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.197lb # [ 0.000000] Tracing variant of Tasks RCU enabled.198client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0199client # [ 0.000000] GICv3: 256 SPIs implemented200lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.201client # [ 0.000000] GICv3: 0 Extended SPIs implemented202lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1203client # [ 0.000000] Root IRQ handler: gic_handle_irq204client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI205lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.206client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0207client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000208lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.209client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]210lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.211client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)212lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0213lb # [ 0.000000] GICv3: 256 SPIs implemented214client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)215lb # [ 0.000000] GICv3: 0 Extended SPIs implemented216lb # [ 0.000000] Root IRQ handler: gic_handle_irq217client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000218lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI219client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000220lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0221client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.222lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000223lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]224client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns225client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).226lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)227lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)228client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns229lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000230client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns231client # [ 0.000047] arm-pv: using stolen time PV232lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000233lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.234client # [ 0.001528] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)235client # [ 0.001779] Console: colour dummy device 80x25236lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns237client # [ 0.001788] printk: legacy console [tty0] enabled238lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).239client # [ 0.002041] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)240lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns241client # [ 0.002049] pid_max: default: 32768 minimum: 301242client # [ 0.002142] LSM: initializing lsm=capability,landlock,yama,bpf,ima243lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns244client # [ 0.002393] landlock: Up and running.245lb # [ 0.000065] arm-pv: using stolen time PV246client # [ 0.002396] Yama: becoming mindful.247client # [ 0.003459] LSM support for eBPF active248lb # [ 0.001862] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)249client # [ 0.003669] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)250lb # [ 0.002065] Console: colour dummy device 80x25251lb # [ 0.002073] printk: legacy console [tty0] enabled252client # [ 0.003710] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)253lb # [ 0.002306] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)254lb # [ 0.002313] pid_max: default: 32768 minimum: 301255lb # [ 0.002409] LSM: initializing lsm=capability,landlock,yama,bpf,ima256lb # [ 0.002912] landlock: Up and running.257lb # [ 0.002915] Yama: becoming mindful.258lb # [ 0.004809] LSM support for eBPF active259client # [ 0.011117] cacheinfo: Unable to detect cache hierarchy for CPU 0260client # [ 0.013172] rcu: Hierarchical SRCU implementation.261lb # [ 0.005043] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)262client # [ 0.013179] rcu: Max phase no-delay instances is 1000.263lb # [ 0.005085] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)264client # [ 0.014585] fsl-mc MSI: its@8080000 domain created265client # [ 0.014695] EFI services will not be available.266lb # [ 0.006378] cacheinfo: Unable to detect cache hierarchy for CPU 0267client # [ 0.014767] smp: Bringing up secondary CPUs ...268lb # [ 0.008337] rcu: Hierarchical SRCU implementation.269client # [ 0.014776] smp: Brought up 1 node, 1 CPU270lb # [ 0.008341] rcu: Max phase no-delay instances is 1000.271client # [ 0.014780] SMP: Total of 1 processors activated.272lb # [ 0.009594] fsl-mc MSI: its@8080000 domain created273client # [ 0.014785] CPU: All CPU(s) started at EL1274lb # [ 0.009711] EFI services will not be available.275client # [ 0.014816] CPU features: detected: Branch Target Identification276lb # [ 0.009792] smp: Bringing up secondary CPUs ...277lb # [ 0.009801] smp: Brought up 1 node, 1 CPU278client # [ 0.014824] CPU features: detected: ARMv8.4 Translation Table Level279lb # [ 0.009804] SMP: Total of 1 processors activated.280lb # [ 0.009810] CPU: All CPU(s) started at EL1281client # [ 0.014827] CPU features: detected: Instruction cache invalidation not required for I/D coherence282lb # [ 0.009826] CPU features: detected: Branch Target Identification283client # [ 0.014831] CPU features: detected: Data cache clean to the PoU not required for I/D coherence284lb # [ 0.009834] CPU features: detected: ARMv8.4 Translation Table Level285client # [ 0.014834] CPU features: detected: Common not Private translations286lb # [ 0.009838] CPU features: detected: Instruction cache invalidation not required for I/D coherence287client # [ 0.014838] CPU features: detected: CRC32 instructions288client # [ 0.014841] CPU features: detected: Data cache clean to Point of Deep Persistence289lb # [ 0.009841] CPU features: detected: Data cache clean to the PoU not required for I/D coherence290client # [ 0.014844] CPU features: detected: Data cache clean to Point of Persistence291client # [ 0.014848] CPU features: detected: Data independent timing control (DIT)292client # [ 0.014851] CPU features: detected: E0PD293client # [ 0.014853] CPU features: detected: Enhanced Counter Virtualization294lb # [ 0.009845] CPU features: detected: Common not Private translations295lb # [ 0.009848] CPU features: detected: CRC32 instructions296client # [ 0.014856] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)297client # [ 0.014859] CPU features: detected: Enhanced Virtualization Traps298lb # [ 0.009851] CPU features: detected: Data cache clean to Point of Deep Persistence299client # [ 0.014862] CPU features: detected: Fine Grained Traps300lb # [ 0.009855] CPU features: detected: Data cache clean to Point of Persistence301client # [ 0.014866] CPU features: detected: Generic authentication (architected QARMA5 algorithm)302lb # [ 0.009858] CPU features: detected: Data independent timing control (DIT)303lb # [ 0.009861] CPU features: detected: E0PD304client # [ 0.014871] CPU features: detected: RCpc load-acquire (LDAPR)305client # [ 0.014874] CPU features: detected: LSE atomic instructions306lb # [ 0.009864] CPU features: detected: Enhanced Counter Virtualization307client # [ 0.014878] CPU features: detected: Privileged Access Never308lb # [ 0.009867] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)309client # [ 0.014880] CPU features: detected: PMUv3310lb # [ 0.009870] CPU features: detected: Enhanced Virtualization Traps311client # [ 0.014883] CPU features: detected: RAS Extension Support312lb # [ 0.009873] CPU features: detected: Fine Grained Traps313client # [ 0.014886] CPU features: detected: RASv1p1 Extension Support314lb # [ 0.009877] CPU features: detected: Generic authentication (architected QARMA5 algorithm)315client # [ 0.014888] CPU features: detected: Random Number Generator316client # [ 0.014891] CPU features: detected: Speculation barrier (SB)317lb # [ 0.009882] CPU features: detected: RCpc load-acquire (LDAPR)318client # [ 0.014894] CPU features: detected: Stage-2 Force Write-Back319lb # [ 0.009885] CPU features: detected: LSE atomic instructions320lb # [ 0.009888] CPU features: detected: Privileged Access Never321client # [ 0.014897] CPU features: detected: TLB range maintenance instructions322lb # [ 0.009891] CPU features: detected: PMUv3323lb # [ 0.009894] CPU features: detected: RAS Extension Support324lb # [ 0.009896] CPU features: detected: RASv1p1 Extension Support325client # [ 0.014902] CPU features: detected: Speculative Store Bypassing Safe (SSBS)326lb # [ 0.009899] CPU features: detected: Random Number Generator327client # [ 0.014979] alternatives: applying system-wide alternatives328lb # [ 0.009902] CPU features: detected: Speculation barrier (SB)329client # [ 0.019128] CPU features: detected: BBM Level 2 without TLB conflict abort330lb # [ 0.009905] CPU features: detected: Stage-2 Force Write-Back331lb # [ 0.009908] CPU features: detected: TLB range maintenance instructions332client # [ 0.019423] Memory: 1406552K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)333lb # [ 0.009913] CPU features: detected: Speculative Store Bypassing Safe (SSBS)334client # [ 0.022160] devtmpfs: initialized335lb # [ 0.009961] alternatives: applying system-wide alternatives336client # [ 0.024294] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)337lb # [ 0.013523] CPU features: detected: BBM Level 2 without TLB conflict abort338client # [ 0.024318] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).339client # [ 0.024617] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL340client # [ 0.024624] 0 pages in range for non-PLT usage341client # [ 0.024625] 508288 pages in range for PLT usage342lb # [ 0.013779] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)343client # [ 0.024760] pinctrl core: initialized pinctrl subsystem344lb # [ 0.015602] devtmpfs: initialized345client # [ 0.025671] DMI not present or invalid.346lb # [ 0.017775] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)347client # [ 0.033090] NET: Registered PF_NETLINK/PF_ROUTE protocol family348client # [ 0.035855] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations349lb # [ 0.017800] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).350lb # [ 0.018185] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL351client # [ 0.036071] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations352lb # [ 0.018192] 0 pages in range for non-PLT usage353lb # [ 0.018193] 508288 pages in range for PLT usage354client # [ 0.036290] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations355lb # [ 0.018335] pinctrl core: initialized pinctrl subsystem356client # [ 0.036330] audit: initializing netlink subsys (disabled)357lb # [ 0.019298] DMI not present or invalid.358client # [ 0.037058] thermal_sys: Registered thermal governor 'fair_share'359lb # [ 0.024801] NET: Registered PF_NETLINK/PF_ROUTE protocol family360client # [ 0.037060] thermal_sys: Registered thermal governor 'bang_bang'361lb # [ 0.027516] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations362client # [ 0.037064] thermal_sys: Registered thermal governor 'step_wise'363client # [ 0.037069] thermal_sys: Registered thermal governor 'user_space'364lb # [ 0.027723] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations365client # [ 0.037072] thermal_sys: Registered thermal governor 'power_allocator'366lb # [ 0.027944] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations367lb # [ 0.027980] audit: initializing netlink subsys (disabled)368client # [ 0.037099] audit: type=2000 audit(0.032:1): state=initialized audit_enabled=0 res=1369client # [ 0.037110] cpuidle: using governor ladder370lb # [ 0.028710] thermal_sys: Registered thermal governor 'fair_share'371client # [ 0.037116] cpuidle: using governor menu372lb # [ 0.028712] thermal_sys: Registered thermal governor 'bang_bang'373client # [ 0.037335] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.374lb # [ 0.028716] thermal_sys: Registered thermal governor 'step_wise'375lb # [ 0.028721] thermal_sys: Registered thermal governor 'user_space'376lb # [ 0.028724] thermal_sys: Registered thermal governor 'power_allocator'377client # [ 0.037356] ASID allocator initialised with 65536 entries378client # [ 0.038812] Serial: AMBA PL011 UART driver379lb # [ 0.028751] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1380lb # [ 0.028762] cpuidle: using governor ladder381client # [ 0.044708] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1382lb # [ 0.028768] cpuidle: using governor menu383client # [ 0.044895] printk: console [ttyAMA0] enabled384lb # [ 0.029003] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.385lb # [ 0.029022] ASID allocator initialised with 65536 entries386lb # [ 0.030498] Serial: AMBA PL011 UART driver387lb # [ 0.036189] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1388lb # [ 0.036374] printk: console [ttyAMA0] enabled389lb # [ 0.167131] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages390lb # [ 0.167155] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page391client # [ 0.181923] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages392lb # [ 0.167160] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages393client # [ 0.181947] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page394lb # [ 0.167165] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page395client # [ 0.181953] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages396lb # [ 0.167169] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages397client # [ 0.181957] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page398lb # [ 0.167173] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page399client # [ 0.181961] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages400lb # [ 0.167177] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages401client # [ 0.181965] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page402lb # [ 0.167182] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page403client # [ 0.181970] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages404client # [ 0.181974] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page405lb # [ 0.175260] fbcon: Taking over console406lb # [ 0.175284] ACPI: Interpreter disabled.407client # [ 0.189891] fbcon: Taking over console408client # [ 0.189911] ACPI: Interpreter disabled.409client # [ 0.199103] iommu: Default domain type: Translated410lb # [ 0.183578] iommu: Default domain type: Translated411client # [ 0.199117] iommu: DMA domain TLB invalidation policy: strict mode412lb # [ 0.183593] iommu: DMA domain TLB invalidation policy: strict mode413client # [ 0.200833] SCSI subsystem initialized414lb # [ 0.184051] SCSI subsystem initialized415client # [ 0.201516] usbcore: registered new interface driver usbfs416lb # [ 0.186077] usbcore: registered new interface driver usbfs417client # [ 0.201554] usbcore: registered new interface driver hub418lb # [ 0.186115] usbcore: registered new interface driver hub419client # [ 0.201577] usbcore: registered new device driver usb420lb # [ 0.186135] usbcore: registered new device driver usb421client # [ 0.201900] pps_core: LinuxPPS API ver. 1 registered422lb # [ 0.186467] pps_core: LinuxPPS API ver. 1 registered423client # [ 0.201907] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>424lb # [ 0.186474] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>425client # [ 0.201917] PTP clock support registered426lb # [ 0.186484] PTP clock support registered427client # [ 0.201989] EDAC MC: Ver: 3.0.0428lb # [ 0.186548] EDAC MC: Ver: 3.0.0429lb # [ 0.191052] scmi_core: SCMI protocol bus registered430lb # [ 0.192645] FPGA manager framework431lb # [ 0.193700] vgaarb: loaded432client # [ 0.207918] scmi_core: SCMI protocol bus registered433lb # [ 0.197961] clocksource: Switched to clocksource arch_sys_counter434lb # [ 0.198778] VFS: Disk quotas dquot_6.6.0435client # [ 0.212528] FPGA manager framework436client # [ 0.213634] vgaarb: loaded437lb # [ 0.198819] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)438lb # [ 0.201343] netfs: FS-Cache loaded439client # [ 0.214588] clocksource: Switched to clocksource arch_sys_counter440lb # [ 0.201485] pnp: PnP ACPI: disabled441client # [ 0.219057] VFS: Disk quotas dquot_6.6.0442client # [ 0.219114] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)443client # [ 0.221345] netfs: FS-Cache loaded444client # [ 0.221500] pnp: PnP ACPI: disabled445client # [ 0.225758] NET: Registered PF_INET protocol family446client # [ 0.226025] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)447lb # [ 0.238201] NET: Registered PF_INET protocol family448lb # [ 0.238495] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)449client # [ 0.279052] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)450client # [ 0.279124] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)451client # [ 0.279162] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)452client # [ 0.279254] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)453client # [ 0.279398] TCP: Hash tables configured (established 16384 bind 16384)454client # [ 0.279517] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)455client # [ 0.279568] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)456client # [ 0.279609] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)457client # [ 0.279712] NET: Registered PF_UNIX/PF_LOCAL protocol family458client # [ 0.279753] NET: Registered PF_XDP protocol family459client # [ 0.279786] PCI: CLS 0 bytes, default 64460client # [ 0.280060] Trying to unpack rootfs image as initramfs...461client # [ 0.296565] kvm [1]: HYP mode not available462lb # [ 0.294372] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)463lb # [ 0.294443] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)464lb # [ 0.294481] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)465lb # [ 0.294578] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)466lb # [ 0.294722] TCP: Hash tables configured (established 16384 bind 16384)467lb # [ 0.294837] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)468lb # [ 0.294881] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)469lb # [ 0.294921] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)470lb # [ 0.295040] NET: Registered PF_UNIX/PF_LOCAL protocol family471lb # [ 0.295066] NET: Registered PF_XDP protocol family472lb # [ 0.295088] PCI: CLS 0 bytes, default 64473lb # [ 0.295375] Trying to unpack rootfs image as initramfs...474lb # [ 0.318236] kvm [1]: HYP mode not available475worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]476worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026477worker1 # [ 0.000000] KASLR enabled478worker1 # [ 0.000000] random: crng init done479worker1 # [ 0.000000] Machine model: linux,dummy-virt480worker1 # [ 0.000000] efi: UEFI not found.481worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT482worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]483worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]484worker1 # [ 0.000000] Zone ranges:485worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]486worker1 # [ 0.000000] DMA32 empty487worker1 # [ 0.000000] Normal empty488worker1 # [ 0.000000] Device empty489worker1 # [ 0.000000] Movable zone start for each node490worker1 # [ 0.000000] Early memory node ranges491worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]492worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]493worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000494worker1 # [ 0.000000] psci: probing for conduit method from DT.495worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.496worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs497worker1 # [ 0.000000] psci: Trusted OS migration not required498worker1 # [ 0.000000] psci: SMC Calling Convention v1.1499worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)500client # [ 0.463814] Initialise system trusted keyrings501worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296502worker1 # [ 0.000000] Detected PIPT I-cache on CPU0503client # [ 0.464638] workingset: timestamp_bits=42 max_order=19 bucket_order=0504worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)505client # [ 0.466020] squashfs: version 4.0 (2009/01/31) Phillip Lougher506worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface507worker1 # [ 0.000000] CPU features: detected: Spectre-v4508worker1 # [ 0.000000] CPU features: detected: Spectre-BHB509worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38510worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23511worker1 # [ 0.000000] alternatives: applying boot alternatives512client # [ 0.466901] 9p: Installing v9fs 9p2000 file system support513worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9d8fygwq37kz8yh1yl6h683p1r23agf6-nixos-system-worker1-test/init regInfo=/nix/store/bdhfwc4gyq5x9cvh3llg8w2cn0zr2la0-closure-info/registration console=ttyAMA0,115200n8 console=tty0514worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bdhfwc4gyq5x9cvh3llg8w2cn0zr2la0-closure-info/registration", will be passed to user space.515worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes516worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)517worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)518worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB519worker1 # [ 0.000000] software IO TLB: area num 1.520worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB521worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)522worker1 # [ 0.000000] Fallback order for Node 0: 0523worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216524client # [ 0.495245] Key type asymmetric registered525client # [ 0.495281] Asymmetric key parser 'x509' registered526client # [ 0.495378] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)527client # [ 0.497597] io scheduler mq-deadline registered528client # [ 0.497612] io scheduler kyber registered529worker1 # [ 0.000000] Policy zone: DMA530worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off531worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1532worker1 # [ 0.000000] allocated 3145728 bytes of page_ext533worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages534worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups535worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.536worker1 # [ 0.000000] rcu: RCU event tracing is enabled.537worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.538worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.539worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.540worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.541worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.542worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1543worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.544lb # [ 0.498619] Initialise system trusted keyrings545worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546lb # [ 0.499454] workingset: timestamp_bits=42 max_order=19 bucket_order=0547worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.548worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0549worker1 # [ 0.000000] GICv3: 256 SPIs implemented550worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented551worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq552worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI553worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0554worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000555worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]556client # [ 0.511524] pl061_gpio 9030000.pl061: PL061 GPIO chip registered557client # [ 0.512175] ledtrig-cpu: registered to indicate activity on CPUs558worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)559lb # [ 0.506584] squashfs: version 4.0 (2009/01/31) Phillip Lougher560client # [ 0.512590] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:561worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)562client # [ 0.512610] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000563worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000564client # [ 0.512622] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000565worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000566worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.567client # [ 0.512631] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000568client # [ 0.512651] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits569worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns570worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).571client # [ 0.512674] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]572client # [ 0.512756] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00573worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns574lb # [ 0.507425] 9p: Installing v9fs 9p2000 file system support575client # [ 0.512770] pci_bus 0000:00: root bus resource [bus 00-ff]576worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns577client # [ 0.512776] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]578worker1 # [ 0.000191] arm-pv: using stolen time PV579client # [ 0.512781] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]580worker1 # [ 0.009954] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)581client # [ 0.512787] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]582worker1 # [ 0.010266] Console: colour dummy device 80x25583worker1 # [ 0.010275] printk: legacy console [tty0] enabled584client # [ 0.512856] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint585client # [ 0.513325] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint586worker1 # [ 0.010505] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)587client # [ 0.513579] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]588worker1 # [ 0.010512] pid_max: default: 32768 minimum: 301589client # [ 0.513598] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]590worker1 # [ 0.010609] LSM: initializing lsm=capability,landlock,yama,bpf,ima591worker1 # [ 0.011908] landlock: Up and running.592client # [ 0.513630] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]593worker1 # [ 0.011911] Yama: becoming mindful.594worker1 # [ 0.020404] LSM support for eBPF active595client # [ 0.513648] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]596worker1 # [ 0.021330] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)597client # [ 0.514129] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint598client # [ 0.514345] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]599worker1 # [ 0.021372] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)600worker1 # [ 0.022669] cacheinfo: Unable to detect cache hierarchy for CPU 0601worker1 # [ 0.023580] rcu: Hierarchical SRCU implementation.602worker1 # [ 0.023584] rcu: Max phase no-delay instances is 1000.603worker1 # [ 0.024930] fsl-mc MSI: its@8080000 domain created604worker1 # [ 0.025038] EFI services will not be available.605worker1 # [ 0.025122] smp: Bringing up secondary CPUs ...606client # [ 0.514363] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]607lb # [ 0.527419] Key type asymmetric registered608worker1 # [ 0.025134] smp: Brought up 1 node, 1 CPU609lb # [ 0.527453] Asymmetric key parser 'x509' registered610worker1 # [ 0.025138] SMP: Total of 1 processors activated.611client # [ 0.514395] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]612worker1 # [ 0.025143] CPU: All CPU(s) started at EL1613lb # [ 0.527550] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)614client # [ 0.514927] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint615worker1 # [ 0.025170] CPU features: detected: Branch Target Identification616lb # [ 0.529712] io scheduler mq-deadline registered617client # [ 0.515118] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]618worker1 # [ 0.025178] CPU features: detected: ARMv8.4 Translation Table Level619lb # [ 0.529726] io scheduler kyber registered620client # [ 0.515135] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]621worker1 # [ 0.025181] CPU features: detected: Instruction cache invalidation not required for I/D coherence622client # [ 0.515166] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]623worker1 # [ 0.025185] CPU features: detected: Data cache clean to the PoU not required for I/D coherence624client # [ 0.515644] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint625client # [ 0.515842] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]626worker1 # [ 0.025188] CPU features: detected: Common not Private translations627worker1 # [ 0.025192] CPU features: detected: CRC32 instructions628client # [ 0.515859] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]629client # [ 0.515893] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]630worker1 # [ 0.025194] CPU features: detected: Data cache clean to Point of Deep Persistence631worker1 # [ 0.025198] CPU features: detected: Data cache clean to Point of Persistence632client # [ 0.516390] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint633client # [ 0.516582] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]634worker1 # [ 0.025202] CPU features: detected: Data independent timing control (DIT)635worker1 # [ 0.025204] CPU features: detected: E0PD636client # [ 0.516599] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]637worker1 # [ 0.025207] CPU features: detected: Enhanced Counter Virtualization638client # [ 0.516630] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]639worker1 # [ 0.025210] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)640client # [ 0.517108] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint641worker1 # [ 0.025213] CPU features: detected: Enhanced Virtualization Traps642client # [ 0.517297] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]643worker1 # [ 0.025216] CPU features: detected: Fine Grained Traps644client # [ 0.517315] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]645worker1 # [ 0.025220] CPU features: detected: Generic authentication (architected QARMA5 algorithm)646client # [ 0.517346] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]647worker1 # [ 0.025226] CPU features: detected: RCpc load-acquire (LDAPR)648client # [ 0.517826] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint649worker1 # [ 0.025229] CPU features: detected: LSE atomic instructions650client # [ 0.518020] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]651worker1 # [ 0.025232] CPU features: detected: Privileged Access Never652client # [ 0.518038] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]653worker1 # [ 0.025235] CPU features: detected: PMUv3654worker1 # [ 0.025238] CPU features: detected: RAS Extension Support655client # [ 0.518069] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]656worker1 # [ 0.025240] CPU features: detected: RASv1p1 Extension Support657client # [ 0.518086] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]658worker1 # [ 0.025243] CPU features: detected: Random Number Generator659worker1 # [ 0.025246] CPU features: detected: Speculation barrier (SB)660worker1 # [ 0.025249] CPU features: detected: Stage-2 Force Write-Back661worker1 # [ 0.025252] CPU features: detected: TLB range maintenance instructions662worker1 # [ 0.025257] CPU features: detected: Speculative Store Bypassing Safe (SSBS)663worker1 # [ 0.025386] alternatives: applying system-wide alternatives664worker1 # [ 0.030123] CPU features: detected: BBM Level 2 without TLB conflict abort665worker1 # [ 0.030484] Memory: 1406564K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)666worker1 # [ 0.034021] devtmpfs: initialized667worker1 # [ 0.036064] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)668lb # [ 0.546169] pl061_gpio 9030000.pl061: PL061 GPIO chip registered669worker1 # [ 0.036087] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).670lb # [ 0.546938] ledtrig-cpu: registered to indicate activity on CPUs671worker1 # [ 0.036486] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL672worker1 # [ 0.036492] 0 pages in range for non-PLT usage673lb # [ 0.547377] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:674worker1 # [ 0.036493] 508288 pages in range for PLT usage675worker1 # [ 0.036637] pinctrl core: initialized pinctrl subsystem676lb # [ 0.547395] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000677worker1 # [ 0.037568] DMI not present or invalid.678client # [ 0.566734] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint679lb # [ 0.547410] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000680worker1 # [ 0.053352] NET: Registered PF_NETLINK/PF_ROUTE protocol family681client # [ 0.566970] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]682worker1 # [ 0.055774] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations683lb # [ 0.547419] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000684client # [ 0.567005] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]685worker1 # [ 0.056000] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations686lb # [ 0.547444] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits687client # [ 0.567590] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint688worker1 # [ 0.056217] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations689client # [ 0.567789] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]690lb # [ 0.547483] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]691worker1 # [ 0.056267] audit: initializing netlink subsys (disabled)692client # [ 0.567821] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]693lb # [ 0.547587] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00694worker1 # [ 0.057001] thermal_sys: Registered thermal governor 'fair_share'695lb # [ 0.547598] pci_bus 0000:00: root bus resource [bus 00-ff]696client # [ 0.568252] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint697worker1 # [ 0.057003] thermal_sys: Registered thermal governor 'bang_bang'698lb # [ 0.547605] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]699client # [ 0.568441] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]700worker1 # [ 0.057006] thermal_sys: Registered thermal governor 'step_wise'701lb # [ 0.547611] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]702worker1 # [ 0.057011] thermal_sys: Registered thermal governor 'user_space'703client # [ 0.568699] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint704lb # [ 0.547617] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]705client # [ 0.568995] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]706worker1 # [ 0.057014] thermal_sys: Registered thermal governor 'power_allocator'707client # [ 0.569014] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]708lb # [ 0.547757] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint709worker1 # [ 0.057043] audit: type=2000 audit(0.044:1): state=initialized audit_enabled=0 res=1710worker1 # [ 0.057054] cpuidle: using governor ladder711client # [ 0.569046] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]712worker1 # [ 0.057061] cpuidle: using governor menu713lb # [ 0.548229] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint714client # [ 0.569537] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint715lb # [ 0.548434] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]716worker1 # [ 0.057278] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.717client # [ 0.569728] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]718lb # [ 0.548452] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]719worker1 # [ 0.057298] ASID allocator initialised with 65536 entries720client # [ 0.569745] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]721worker1 # [ 0.058687] Serial: AMBA PL011 UART driver722lb # [ 0.548483] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]723client # [ 0.569777] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]724lb # [ 0.548499] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]725worker1 # [ 0.066555] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1726worker1 # [ 0.067378] printk: console [ttyAMA0] enabled727client # [ 0.570425] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned728lb # [ 0.548972] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint729client # [ 0.570437] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned730lb # [ 0.549193] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]731lb # [ 0.549210] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]732client # [ 0.570443] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned733lb # [ 0.549240] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]734client # [ 0.570492] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned735lb # [ 0.549707] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint736client # [ 0.570541] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned737lb # [ 0.549893] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]738lb # [ 0.549910] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]739lb # [ 0.549940] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]740lb # [ 0.550440] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint741lb # [ 0.550626] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]742client # [ 0.588281] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned743lb # [ 0.550643] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]744client # [ 0.588335] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned745lb # [ 0.550676] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]746client # [ 0.588385] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned747lb # [ 0.551155] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint748client # [ 0.588436] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned749lb # [ 0.551340] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]750worker1 # [ 0.213738] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages751client # [ 0.588486] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned752lb # [ 0.551357] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]753worker1 # [ 0.213764] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page754lb # [ 0.551387] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]755worker1 # [ 0.213769] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages756client # [ 0.588536] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned757worker1 # [ 0.213774] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page758lb # [ 0.551847] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint759client # [ 0.588586] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned760lb # [ 0.552033] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]761worker1 # [ 0.213778] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages762client # [ 0.588674] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned763lb # [ 0.552049] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]764worker1 # [ 0.213782] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page765client # [ 0.588723] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned766worker1 # [ 0.213786] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages767lb # [ 0.552080] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]768client # [ 0.588745] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned769worker1 # [ 0.213790] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page770lb # [ 0.552543] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint771client # [ 0.588768] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned772lb # [ 0.552730] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]773client # [ 0.588791] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned774lb # [ 0.552747] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]775client # [ 0.588817] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned776lb # [ 0.552777] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]777client # [ 0.588840] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned778lb # [ 0.552794] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]779client # [ 0.588863] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned780lb # [ 0.553313] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint781client # [ 0.588887] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned782lb # [ 0.553503] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]783client # [ 0.588910] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned784lb # [ 0.553534] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]785client # [ 0.588934] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned786client # [ 0.588957] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned787client # [ 0.588981] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned788worker1 # [ 0.226848] fbcon: Taking over console789worker1 # [ 0.226869] ACPI: Interpreter disabled.790client # [ 0.589005] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned791worker1 # [ 0.228848] iommu: Default domain type: Translated792client # [ 0.589028] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned793worker1 # [ 0.228860] iommu: DMA domain TLB invalidation policy: strict mode794client # [ 0.589051] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned795worker1 # [ 0.230715] SCSI subsystem initialized796client # [ 0.589074] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned797client # [ 0.589097] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned798worker1 # [ 0.231430] usbcore: registered new interface driver usbfs799client # [ 0.589121] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned800worker1 # [ 0.231478] usbcore: registered new interface driver hub801client # [ 0.589144] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned802worker1 # [ 0.231501] usbcore: registered new device driver usb803client # [ 0.589167] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned804worker1 # [ 0.231815] pps_core: LinuxPPS API ver. 1 registered805client # [ 0.589190] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned806client # [ 0.589222] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]807worker1 # [ 0.231821] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>808lb # [ 0.598055] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint809worker1 # [ 0.231832] PTP clock support registered810client # [ 0.589232] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]811lb # [ 0.598279] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]812worker1 # [ 0.231886] EDAC MC: Ver: 3.0.0813client # [ 0.589237] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]814lb # [ 0.598310] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]815client # [ 0.590491] pci 0000:00:0a.0: enabling device (0000 -> 0002)816lb # [ 0.598764] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint817lb # [ 0.598947] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]818worker1 # [ 0.236754] scmi_core: SCMI protocol bus registered819lb # [ 0.599201] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint820lb # [ 0.599485] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]821lb # [ 0.599503] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]822worker1 # [ 0.241255] FPGA manager framework823lb # [ 0.599534] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]824worker1 # [ 0.242325] vgaarb: loaded825lb # [ 0.600004] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint826lb # [ 0.600190] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]827worker1 # [ 0.243023] clocksource: Switched to clocksource arch_sys_counter828lb # [ 0.600206] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]829lb # [ 0.600236] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]830lb # [ 0.600855] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned831lb # [ 0.600868] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned832worker1 # [ 0.247051] VFS: Disk quotas dquot_6.6.0833lb # [ 0.600875] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned834worker1 # [ 0.247095] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)835lb # [ 0.600921] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned836worker1 # [ 0.249161] netfs: FS-Cache loaded837worker1 # [ 0.249317] pnp: PnP ACPI: disabled838lb # [ 0.600970] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned839client # [ 0.627206] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 35858 usecs840lb # [ 0.601036] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned841lb # [ 0.601084] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned842lb # [ 0.601132] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned843lb # [ 0.601182] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned844lb # [ 0.601232] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned845worker1 # [ 0.253460] NET: Registered PF_INET protocol family846lb # [ 0.601281] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned847worker1 # [ 0.253744] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)848lb # [ 0.601330] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned849lb # [ 0.601406] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned850lb # [ 0.601453] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned851lb # [ 0.601476] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned852lb # [ 0.601498] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned853lb # [ 0.601521] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned854lb # [ 0.601547] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned855lb # [ 0.601570] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned856lb # [ 0.601592] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned857lb # [ 0.601616] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned858lb # [ 0.601639] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned859lb # [ 0.601662] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned860lb # [ 0.601686] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned861lb # [ 0.601710] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned862lb # [ 0.601732] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned863lb # [ 0.601755] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned864lb # [ 0.601777] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned865lb # [ 0.601799] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned866lb # [ 0.601822] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned867lb # [ 0.601845] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned868lb # [ 0.601868] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned869lb # [ 0.601890] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned870lb # [ 0.601913] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned871lb # [ 0.601942] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]872client # [ 0.652089] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)873lb # [ 0.601952] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]874client # [ 0.654527] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)875lb # [ 0.650000] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]876client # [ 0.665292] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)877lb # [ 0.651238] pci 0000:00:0a.0: enabling device (0000 -> 0002)878client # [ 0.675416] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)879worker1 # [ 0.297819] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)880worker1 # [ 0.297921] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)881worker1 # [ 0.297958] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)882worker1 # [ 0.298058] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)883worker1 # [ 0.298204] TCP: Hash tables configured (established 16384 bind 16384)884worker1 # [ 0.298319] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)885worker1 # [ 0.298360] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)886worker1 # [ 0.298401] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)887client # [ 0.687286] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)888worker1 # [ 0.298519] NET: Registered PF_UNIX/PF_LOCAL protocol family889client # [ 0.689397] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)890worker1 # [ 0.298545] NET: Registered PF_XDP protocol family891worker1 # [ 0.298565] PCI: CLS 0 bytes, default 64892client # [ 0.691544] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)893worker1 # [ 0.298890] Trying to unpack rootfs image as initramfs...894client # [ 0.693821] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)895worker1 # [ 0.317142] kvm [1]: HYP mode not available896lb # [ 0.682505] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)897lb # [ 0.685105] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)898lb # [ 0.687341] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)899client # [ 0.704892] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)900lb # [ 0.689689] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)901lb # [ 0.699525] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)902client # [ 0.710861] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)903client # [ 0.713228] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)904client # [ 0.736288] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled905client # [ 0.740646] msm_serial: driver initialized906client # [ 0.740825] SuperH (H)SCI(F) driver initialized907client # [ 0.740884] STM32 USART driver initialized908lb # [ 0.715908] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)909lb # [ 0.718199] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)910lb # [ 0.720292] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)911lb # [ 0.723088] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)912lb # [ 0.724895] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)913lb # [ 0.728953] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)914lb # [ 0.746935] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled915lb # [ 0.750162] msm_serial: driver initialized916lb # [ 0.750342] SuperH (H)SCI(F) driver initialized917lb # [ 0.750405] STM32 USART driver initialized918client # [ 0.780032] loop: module loaded919client # [ 0.780264] virtio_blk virtio5: 1/0/0 default/read/poll queues920client # [ 0.781134] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)921lb # [ 0.784196] loop: module loaded922lb # [ 0.784432] virtio_blk virtio5: 1/0/0 default/read/poll queues923lb # [ 0.785278] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)924client # [ 0.795437] megasas: 07.734.00.00-rc1925client # [ 0.796241] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]926client # [ 0.798389] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000927client # [ 0.798441] Intel/Sharp Extended Query Table at 0x0031928client # [ 0.800287] Using buffer write method929client # [ 0.800389] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]930client # [ 0.802295] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000931client # [ 0.802337] Intel/Sharp Extended Query Table at 0x0031932lb # [ 0.798839] megasas: 07.734.00.00-rc1933lb # [ 0.799689] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]934client # [ 0.822731] Using buffer write method935client # [ 0.822778] Concatenating MTD devices:936client # [ 0.822782] (0): "0.flash"937client # [ 0.822786] (1): "0.flash"938client # [ 0.822790] into device "0.flash"939lb # [ 0.818771] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000940lb # [ 0.818855] Intel/Sharp Extended Query Table at 0x0031941lb # [ 0.820993] Using buffer write method942lb # [ 0.821101] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]943lb # [ 0.835953] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000944lb # [ 0.836008] Intel/Sharp Extended Query Table at 0x0031945lb # [ 0.837618] Using buffer write method946lb # [ 0.837658] Concatenating MTD devices:947lb # [ 0.837662] (0): "0.flash"948lb # [ 0.837666] (1): "0.flash"949lb # [ 0.837670] into device "0.flash"950worker1 # [ 0.497739] Initialise system trusted keyrings951worker1 # [ 0.498585] workingset: timestamp_bits=42 max_order=19 bucket_order=0952worker1 # [ 0.500724] squashfs: version 4.0 (2009/01/31) Phillip Lougher953worker1 # [ 0.507077] 9p: Installing v9fs 9p2000 file system support954worker1 # [ 0.527542] Key type asymmetric registered955worker1 # [ 0.527582] Asymmetric key parser 'x509' registered956worker1 # [ 0.527745] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)957worker1 # [ 0.535131] io scheduler mq-deadline registered958worker1 # [ 0.535160] io scheduler kyber registered959worker1 # [ 0.547248] pl061_gpio 9030000.pl061: PL061 GPIO chip registered960worker1 # [ 0.548004] ledtrig-cpu: registered to indicate activity on CPUs961worker1 # [ 0.548426] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:962worker1 # [ 0.548445] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000963worker1 # [ 0.548458] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000964worker1 # [ 0.548466] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000965worker1 # [ 0.548489] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966worker1 # [ 0.548517] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]967worker1 # [ 0.548598] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00968worker1 # [ 0.548610] pci_bus 0000:00: root bus resource [bus 00-ff]969worker1 # [ 0.548616] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]970worker1 # [ 0.548621] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]971worker1 # [ 0.548626] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]972worker1 # [ 0.548755] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint973worker1 # [ 0.549217] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint974worker1 # [ 0.549420] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]975worker1 # [ 0.549437] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]976worker1 # [ 0.549467] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]977worker1 # [ 0.549484] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]978worker1 # [ 0.549976] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint979worker1 # [ 0.550161] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]980worker1 # [ 0.550178] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]981worker1 # [ 0.550208] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]982worker1 # [ 0.550658] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint983worker1 # [ 0.550840] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]984worker1 # [ 0.550856] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]985worker1 # [ 0.550885] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986worker1 # [ 0.551374] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint987worker1 # [ 0.551557] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]988worker1 # [ 0.551573] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]989worker1 # [ 0.551606] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker1 # [ 0.552072] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint991worker1 # [ 0.552253] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]992worker1 # [ 0.552270] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]993worker1 # [ 0.552300] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.552748] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint995worker1 # [ 0.552930] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]996worker1 # [ 0.552945] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]997worker1 # [ 0.552974] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998worker1 # [ 0.553429] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint999worker1 # [ 0.553610] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1000worker1 # [ 0.553626] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1001worker1 # [ 0.553654] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002worker1 # [ 0.553670] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1003worker1 # [ 0.554215] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1004worker1 # [ 0.554404] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1005worker1 # [ 0.554434] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker1 # [ 0.554887] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1007worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]1008worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 20261009worker2 # [ 0.000000] KASLR enabled1010worker2 # [ 0.000000] random: crng init done1011worker2 # [ 0.000000] Machine model: linux,dummy-virt1012worker2 # [ 0.000000] efi: UEFI not found.1013worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT1014worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]1015worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]1016worker2 # [ 0.000000] Zone ranges:1017worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]1018worker2 # [ 0.000000] DMA32 empty1019worker2 # [ 0.000000] Normal empty1020worker2 # [ 0.000000] Device empty1021worker2 # [ 0.000000] Movable zone start for each node1022worker2 # [ 0.000000] Early memory node ranges1023worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]1024worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]1025worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c4000001026worker2 # [ 0.000000] psci: probing for conduit method from DT.1027worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.1028worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs1029worker2 # [ 0.000000] psci: Trusted OS migration not required1030worker2 # [ 0.000000] psci: SMC Calling Convention v1.11031worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)1032worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u3112961033worker2 # [ 0.000000] Detected PIPT I-cache on CPU01034worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)1035worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface1036worker1 # [ 0.611172] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1037worker2 # [ 0.000000] CPU features: detected: Spectre-v41038worker2 # [ 0.000000] CPU features: detected: Spectre-BHB1039worker1 # [ 0.611227] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_381041worker1 # [ 0.611719] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1042worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_231043worker1 # [ 0.611910] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1044worker2 # [ 0.000000] alternatives: applying boot alternatives1045worker1 # [ 0.612163] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1046worker1 # [ 0.612440] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1047worker1 # [ 0.612458] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1048worker1 # [ 0.612488] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5cljzl6wrgl74g4i9mg12h52ihgkgzia-nixos-system-worker2-test/init regInfo=/nix/store/6hjywiy50qhqnjghshq6dwj0rsaxxvm3-closure-info/registration console=ttyAMA0,115200n8 console=tty01050worker1 # [ 0.612983] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1051worker1 # [ 0.613169] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1052worker1 # [ 0.613184] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1053worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6hjywiy50qhqnjghshq6dwj0rsaxxvm3-closure-info/registration", will be passed to user space.1054worker1 # [ 0.613214] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes1056worker1 # [ 0.613817] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1057worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)1058worker1 # [ 0.613829] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1059worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)1060worker1 # [ 0.613834] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1061worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB1062worker2 # [ 0.000000] software IO TLB: area num 1.1063worker1 # [ 0.613895] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1064worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB1065worker1 # [ 0.613943] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1066worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)1067worker2 # [ 0.000000] Fallback order for Node 0: 01068worker1 # [ 0.613991] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1069worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 3932161070worker1 # [ 0.614039] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1071worker2 # [ 0.000000] Policy zone: DMA1072worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off1073worker1 # [ 0.614085] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1074worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=11075worker1 # [ 0.614134] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1076worker2 # [ 0.000000] allocated 3145728 bytes of page_ext1077worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages1078worker1 # [ 0.614183] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1079worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups1080worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.1081worker1 # [ 0.614231] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1082worker2 # [ 0.000000] rcu: RCU event tracing is enabled.1083worker1 # [ 0.614278] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1084worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.1085worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.1086worker1 # [ 0.614358] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1087worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.1088worker1 # [ 0.614404] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1089worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.1090worker1 # [ 0.614426] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1091worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.1092worker1 # [ 0.614448] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1093worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=11094worker1 # [ 0.614471] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1095worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1096worker1 # [ 0.614496] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1097worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1098worker1 # [ 0.614518] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1099worker1 # [ 0.614545] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1100worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.1101worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 01102worker1 # [ 0.614567] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1103worker2 # [ 0.000000] GICv3: 256 SPIs implemented1104worker1 # [ 0.614589] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1105worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented1106worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq1107worker1 # [ 0.614612] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1108worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI1109worker1 # [ 0.614635] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1110worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=01111worker1 # [ 0.614658] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1112worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a00001113worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]1114worker1 # [ 0.614680] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1115worker1 # [ 0.614702] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1116worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)1117worker1 # [ 0.614723] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1118worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)1119worker1 # [ 0.614744] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1120worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a000001121worker1 # [ 0.614765] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1122worker1 # [ 0.614787] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1123worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a200001124worker1 # [ 0.614808] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1125worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.1126worker1 # [ 0.614831] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1127worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns1128worker1 # [ 0.614853] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1129worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).1130worker1 # [ 0.614881] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1131worker1 # [ 0.614892] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1132worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns1133worker1 # [ 0.614897] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1134worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns1135worker1 # [ 0.616113] pci 0000:00:0a.0: enabling device (0000 -> 0002)1136worker2 # [ 0.000053] arm-pv: using stolen time PV1137worker2 # [ 0.001001] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)1138worker2 # [ 0.001187] Console: colour dummy device 80x251139worker2 # [ 0.001194] printk: legacy console [tty0] enabled1140worker2 # [ 0.001427] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)1141worker2 # [ 0.001435] pid_max: default: 32768 minimum: 3011142worker2 # [ 0.001538] LSM: initializing lsm=capability,landlock,yama,bpf,ima1143worker2 # [ 0.002208] landlock: Up and running.1144worker2 # [ 0.002214] Yama: becoming mindful.1145worker2 # [ 0.003038] LSM support for eBPF active1146worker2 # [ 0.003209] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1147worker2 # [ 0.003248] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1148worker2 # [ 0.004623] cacheinfo: Unable to detect cache hierarchy for CPU 01149worker2 # [ 0.005378] rcu: Hierarchical SRCU implementation.1150worker2 # [ 0.005383] rcu: Max phase no-delay instances is 1000.1151worker2 # [ 0.007381] fsl-mc MSI: its@8080000 domain created1152worker2 # [ 0.007503] EFI services will not be available.1153worker2 # [ 0.007603] smp: Bringing up secondary CPUs ...1154worker2 # [ 0.007613] smp: Brought up 1 node, 1 CPU1155worker2 # [ 0.007616] SMP: Total of 1 processors activated.1156worker2 # [ 0.007621] CPU: All CPU(s) started at EL11157worker2 # [ 0.007652] CPU features: detected: Branch Target Identification1158worker2 # [ 0.007660] CPU features: detected: ARMv8.4 Translation Table Level1159worker2 # [ 0.007663] CPU features: detected: Instruction cache invalidation not required for I/D coherence1160worker2 # [ 0.007667] CPU features: detected: Data cache clean to the PoU not required for I/D coherence1161worker2 # [ 0.007671] CPU features: detected: Common not Private translations1162worker2 # [ 0.007674] CPU features: detected: CRC32 instructions1163worker2 # [ 0.007677] CPU features: detected: Data cache clean to Point of Deep Persistence1164worker2 # [ 0.007681] CPU features: detected: Data cache clean to Point of Persistence1165worker2 # [ 0.007684] CPU features: detected: Data independent timing control (DIT)1166worker2 # [ 0.007687] CPU features: detected: E0PD1167worker2 # [ 0.007690] CPU features: detected: Enhanced Counter Virtualization1168worker2 # [ 0.007693] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)1169worker2 # [ 0.007696] CPU features: detected: Enhanced Virtualization Traps1170worker2 # [ 0.007699] CPU features: detected: Fine Grained Traps1171worker2 # [ 0.007702] CPU features: detected: Generic authentication (architected QARMA5 algorithm)1172worker1 # [ 0.683149] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1173worker2 # [ 0.007707] CPU features: detected: RCpc load-acquire (LDAPR)1174worker1 # [ 0.685690] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1175worker2 # [ 0.007710] CPU features: detected: LSE atomic instructions1176worker2 # [ 0.007713] CPU features: detected: Privileged Access Never1177worker2 # [ 0.007716] CPU features: detected: PMUv31178worker2 # [ 0.007719] CPU features: detected: RAS Extension Support1179worker2 # [ 0.007722] CPU features: detected: RASv1p1 Extension Support1180worker2 # [ 0.007724] CPU features: detected: Random Number Generator1181worker2 # [ 0.007727] CPU features: detected: Speculation barrier (SB)1182worker2 # [ 0.007730] CPU features: detected: Stage-2 Force Write-Back1183worker2 # [ 0.007740] CPU features: detected: TLB range maintenance instructions1184worker2 # [ 0.007745] CPU features: detected: Speculative Store Bypassing Safe (SSBS)1185worker2 # [ 0.007802] alternatives: applying system-wide alternatives1186worker2 # [ 0.012477] CPU features: detected: BBM Level 2 without TLB conflict abort1187worker2 # [ 0.012778] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)1188worker2 # [ 0.015639] devtmpfs: initialized1189worker1 # [ 0.695811] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1190worker2 # [ 0.017808] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)1191worker1 # [ 0.698124] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1192worker2 # [ 0.017832] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).1193worker2 # [ 0.018204] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL1194worker2 # [ 0.018209] 0 pages in range for non-PLT usage1195worker2 # [ 0.018210] 508288 pages in range for PLT usage1196worker2 # [ 0.018391] pinctrl core: initialized pinctrl subsystem1197worker2 # [ 0.019293] DMI not present or invalid.1198worker2 # [ 0.022505] NET: Registered PF_NETLINK/PF_ROUTE protocol family1199worker2 # [ 0.025143] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1200worker2 # [ 0.025388] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1201worker2 # [ 0.025614] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1202worker2 # [ 0.025653] audit: initializing netlink subsys (disabled)1203worker2 # [ 0.026308] thermal_sys: Registered thermal governor 'fair_share'1204worker2 # [ 0.026310] thermal_sys: Registered thermal governor 'bang_bang'1205worker2 # [ 0.026315] thermal_sys: Registered thermal governor 'step_wise'1206worker2 # [ 0.026320] thermal_sys: Registered thermal governor 'user_space'1207worker2 # [ 0.026323] thermal_sys: Registered thermal governor 'power_allocator'1208worker1 # [ 0.709554] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1209worker2 # [ 0.026359] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=11210worker2 # [ 0.026369] cpuidle: using governor ladder1211worker2 # [ 0.026375] cpuidle: using governor menu1212worker2 # [ 0.026587] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.1213worker2 # [ 0.026610] ASID allocator initialised with 65536 entries1214worker2 # [ 0.028012] Serial: AMBA PL011 UART driver1215worker2 # [ 0.033830] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev11216worker2 # [ 0.033967] printk: console [ttyAMA0] enabled1217worker1 # [ 0.718654] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1218worker2 # [ 0.160396] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1219worker2 # [ 0.160420] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page1220worker1 # [ 0.724553] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1221worker2 # [ 0.160425] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages1222worker1 # [ 0.727014] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1223worker2 # [ 0.160429] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page1224worker2 # [ 0.160434] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1225worker2 # [ 0.160438] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page1226worker2 # [ 0.160442] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages1227worker2 # [ 0.160446] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page1228worker2 # [ 0.168314] fbcon: Taking over console1229worker2 # [ 0.168334] ACPI: Interpreter disabled.1230worker1 # [ 0.738087] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1231worker2 # [ 0.176020] iommu: Default domain type: Translated1232worker2 # [ 0.176045] iommu: DMA domain TLB invalidation policy: strict mode1233worker1 # [ 0.744308] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1234worker2 # [ 0.183024] SCSI subsystem initialized1235worker2 # [ 0.190800] usbcore: registered new interface driver usbfs1236worker2 # [ 0.190829] usbcore: registered new interface driver hub1237worker2 # [ 0.190860] usbcore: registered new device driver usb1238worker2 # [ 0.191214] pps_core: LinuxPPS API ver. 1 registered1239worker1 # [ 0.755519] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1240worker2 # [ 0.191219] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>1241worker2 # [ 0.191230] PTP clock support registered1242worker2 # [ 0.191282] EDAC MC: Ver: 3.0.01243worker2 # [ 0.196080] scmi_core: SCMI protocol bus registered1244worker2 # [ 0.197410] FPGA manager framework1245worker2 # [ 0.198438] vgaarb: loaded1246worker2 # [ 0.199140] clocksource: Switched to clocksource arch_sys_counter1247worker1 # [ 0.768372] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1248worker2 # [ 0.206477] VFS: Disk quotas dquot_6.6.01249worker2 # [ 0.206533] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1250worker2 # [ 0.208976] netfs: FS-Cache loaded1251worker2 # [ 0.209125] pnp: PnP ACPI: disabled1252worker1 # [ 0.771410] msm_serial: driver initialized1253worker1 # [ 0.771580] SuperH (H)SCI(F) driver initialized1254worker1 # [ 0.771637] STM32 USART driver initialized1255worker2 # [ 0.213372] NET: Registered PF_INET protocol family1256worker2 # [ 0.213630] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1257client # [ 1.188728] Freeing initrd memory: 26160K1258client # [ 1.195564] tun: Universal TUN/TAP device driver, 1.61259client # [ 1.199900] thunder_xcv, ver 1.01260client # [ 1.199955] thunder_bgx, ver 1.01261client # [ 1.199982] nicpf, ver 1.01262client # [ 1.200595] e1000: Intel(R) PRO/1000 Network Driver1263client # [ 1.200602] e1000: Copyright (c) 1999-2006 Intel Corporation.1264worker1 # [ 0.824228] loop: module loaded1265client # [ 1.200631] e1000e: Intel(R) PRO/1000 Network Driver1266worker1 # [ 0.824457] virtio_blk virtio5: 1/0/0 default/read/poll queues1267client # [ 1.200639] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1268worker1 # [ 0.825275] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1269client # [ 1.200671] igb: Intel(R) Gigabit Ethernet Network Driver1270client # [ 1.200677] igb: Copyright (c) 2007-2014 Intel Corporation.1271client # [ 1.200719] igbvf: Intel(R) Gigabit Virtual Function Network Driver1272worker2 # [ 0.255391] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1273client # [ 1.200728] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1274client # [ 1.200874] sky2: driver version 1.301275worker2 # [ 0.255455] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1276client # [ 1.203357] ehci-pci 0000:00:0a.0: EHCI Host Controller1277worker2 # [ 0.255491] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1278client # [ 1.203389] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11279worker2 # [ 0.255590] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1280client # [ 1.204191] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001281worker2 # [ 0.255749] TCP: Hash tables configured (established 16384 bind 16384)1282lb # [ 1.199323] Freeing initrd memory: 26140K1283worker2 # [ 0.256215] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1284worker2 # [ 0.256486] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1285client # [ 1.213990] usbcore: registered new interface driver usb-storage1286worker2 # [ 0.256538] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1287client # [ 1.214096] usbcore: registered new interface driver usbserial_generic1288worker2 # [ 0.256723] NET: Registered PF_UNIX/PF_LOCAL protocol family1289client # [ 1.214114] usbserial: USB Serial support registered for generic1290worker2 # [ 0.257169] NET: Registered PF_XDP protocol family1291client # [ 1.214644] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001292worker2 # [ 0.257198] PCI: CLS 0 bytes, default 641293client # [ 1.215095] hub 1-0:1.0: USB hub found1294worker2 # [ 0.257893] Trying to unpack rootfs image as initramfs...1295client # [ 1.215117] hub 1-0:1.0: 6 ports detected1296lb # [ 1.206225] tun: Universal TUN/TAP device driver, 1.61297client # [ 1.219775] hv_vmbus: registering driver hyperv_keyboard1298client # [ 1.221391] rtc-pl031 9010000.pl031: registered as rtc01299client # [ 1.221419] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T12:53:36 UTC (1789995216)1300client # [ 1.221748] i2c_dev: i2c /dev entries driver1301worker1 # [ 0.835904] megasas: 07.734.00.00-rc11302worker1 # [ 0.836954] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1303worker1 # [ 0.840676] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001304worker1 # [ 0.840714] Intel/Sharp Extended Query Table at 0x00311305lb # [ 1.210346] thunder_xcv, ver 1.01306worker1 # [ 0.842372] Using buffer write method1307lb # [ 1.210395] thunder_bgx, ver 1.01308lb # [ 1.210434] nicpf, ver 1.01309worker2 # [ 0.279188] kvm [1]: HYP mode not available1310worker1 # [ 0.842472] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1311lb # [ 1.211061] e1000: Intel(R) PRO/1000 Network Driver1312lb # [ 1.211068] e1000: Copyright (c) 1999-2006 Intel Corporation.1313lb # [ 1.211096] e1000e: Intel(R) PRO/1000 Network Driver1314client # [ 1.227403] sdhci: Secure Digital Host Controller Interface driver1315client # [ 1.227420] sdhci: Copyright(c) Pierre Ossman1316lb # [ 1.211104] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1317client # [ 1.227743] Synopsys Designware Multimedia Card Interface Driver1318lb # [ 1.211141] igb: Intel(R) Gigabit Ethernet Network Driver1319client # [ 1.228171] sdhci-pltfm: SDHCI platform and OF driver helper1320lb # [ 1.211147] igb: Copyright (c) 2007-2014 Intel Corporation.1321client # [ 1.229945] hid: raw HID events driver (C) Jiri Kosina1322lb # [ 1.211178] igbvf: Intel(R) Gigabit Virtual Function Network Driver1323client # [ 1.230219] usbcore: registered new interface driver usbhid1324lb # [ 1.211186] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1325client # [ 1.230227] usbhid: USB HID core driver1326lb # [ 1.211335] sky2: driver version 1.301327lb # [ 1.213124] usbcore: registered new interface driver usb-storage1328lb # [ 1.213382] usbcore: registered new interface driver usbserial_generic1329lb # [ 1.213402] usbserial: USB Serial support registered for generic1330lb # [ 1.214640] ehci-pci 0000:00:0a.0: EHCI Host Controller1331lb # [ 1.214671] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11332client # [ 1.236007] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1333lb # [ 1.214887] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001334client # [ 1.237747] drop_monitor: Initializing network drop monitor service1335client # [ 1.237975] NET: Registered PF_INET6 protocol family1336lb # [ 1.226659] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001337worker1 # [ 0.857020] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001338lb # [ 1.227064] hub 1-0:1.0: USB hub found1339client # [ 1.241417] Segment Routing with IPv61340worker1 # [ 0.857079] Intel/Sharp Extended Query Table at 0x00311341lb # [ 1.227089] hub 1-0:1.0: 6 ports detected1342client # [ 1.241445] In-situ OAM (IOAM) with IPv61343worker1 # [ 0.858601] Using buffer write method1344client # [ 1.241480] NET: Registered PF_PACKET protocol family1345worker1 # [ 0.858655] Concatenating MTD devices:1346worker1 # [ 0.858661] (0): "0.flash"1347lb # [ 1.229426] hv_vmbus: registering driver hyperv_keyboard1348worker1 # [ 0.858665] (1): "0.flash"1349worker1 # [ 0.858669] into device "0.flash"1350lb # [ 1.231082] rtc-pl031 9010000.pl031: registered as rtc01351client # [ 1.243181] 9pnet: Installing 9P2000 support1352lb # [ 1.231115] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T12:53:36 UTC (1789995216)1353client # [ 1.245445] Key type dns_resolver registered1354lb # [ 1.231465] i2c_dev: i2c /dev entries driver1355lb # [ 1.236681] sdhci: Secure Digital Host Controller Interface driver1356lb # [ 1.236695] sdhci: Copyright(c) Pierre Ossman1357lb # [ 1.237001] Synopsys Designware Multimedia Card Interface Driver1358lb # [ 1.237394] sdhci-pltfm: SDHCI platform and OF driver helper1359client # [ 1.252426] registered taskstats version 11360client # [ 1.252601] Loading compiled-in X.509 certificates1361lb # [ 1.242124] hid: raw HID events driver (C) Jiri Kosina1362lb # [ 1.242434] usbcore: registered new interface driver usbhid1363lb # [ 1.242441] usbhid: USB HID core driver1364lb # [ 1.245539] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1365lb # [ 1.248441] drop_monitor: Initializing network drop monitor service1366client # [ 1.261764] Demotion targets for Node 0: null1367client # [ 1.261913] Key type .fscrypt registered1368lb # [ 1.248620] NET: Registered PF_INET6 protocol family1369client # [ 1.261919] Key type fscrypt-provisioning registered1370client # [ 1.262034] ima: No TPM chip found, activating TPM-bypass!1371lb # [ 1.250921] Segment Routing with IPv61372client # [ 1.262055] ima: Allocated hash algorithm: sha11373lb # [ 1.250955] In-situ OAM (IOAM) with IPv61374client # [ 1.262087] ima: No architecture policies found1375lb # [ 1.251056] NET: Registered PF_PACKET protocol family1376client # [ 1.266490] input: gpio-keys as /devices/platform/gpio-keys/input/input01377lb # [ 1.252755] 9pnet: Installing 9P2000 support1378lb # [ 1.255905] Key type dns_resolver registered1379lb # [ 1.273655] registered taskstats version 11380lb # [ 1.273836] Loading compiled-in X.509 certificates1381client # [ 1.303036] clk: Disabling unused clocks1382client # [ 1.303399] PM: genpd: Disabling unused power domains1383lb # [ 1.301481] Demotion targets for Node 0: null1384lb # [ 1.301674] Key type .fscrypt registered1385lb # [ 1.301683] Key type fscrypt-provisioning registered1386lb # [ 1.301825] ima: No TPM chip found, activating TPM-bypass!1387lb # [ 1.301848] ima: Allocated hash algorithm: sha11388lb # [ 1.301881] ima: No architecture policies found1389lb # [ 1.307446] input: gpio-keys as /devices/platform/gpio-keys/input/input01390lb # [ 1.330333] clk: Disabling unused clocks1391lb # [ 1.330374] PM: genpd: Disabling unused power domains1392lb # [ 1.336082] Freeing unused kernel memory: 4736K1393lb # [ 1.336405] Run /init as init process1394lb # [ 1.357094] systemd[1]: Successfully made /usr/ read-only.1395client # [ 1.315899] Freeing unused kernel memory: 4736K1396client # [ 1.316449] Run /init as init process1397worker2 # [ 0.436738] Initialise system trusted keyrings1398worker2 # [ 0.437572] workingset: timestamp_bits=42 max_order=19 bucket_order=01399worker2 # [ 0.438874] squashfs: version 4.0 (2009/01/31) Phillip Lougher1400worker2 # [ 0.438935] 9p: Installing v9fs 9p2000 file system support1401worker2 # [ 0.450883] Key type asymmetric registered1402worker2 # [ 0.450904] Asymmetric key parser 'x509' registered1403worker2 # [ 0.451090] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1404worker2 # [ 0.454768] io scheduler mq-deadline registered1405worker2 # [ 0.454781] io scheduler kyber registered1406client # [ 1.403147] systemd[1]: Successfully made /usr/ read-only.1407worker2 # [ 0.495352] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1408worker2 # [ 0.496944] ledtrig-cpu: registered to indicate activity on CPUs1409worker2 # [ 0.497403] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1410worker2 # [ 0.497421] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001411worker2 # [ 0.497432] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001412worker2 # [ 0.497441] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001413worker2 # [ 0.497462] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1414worker2 # [ 0.497490] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1415worker2 # [ 0.497571] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001416worker2 # [ 0.497581] pci_bus 0000:00: root bus resource [bus 00-ff]1417worker2 # [ 0.497599] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1418worker2 # [ 0.497605] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1419worker2 # [ 0.497610] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1420worker2 # [ 0.497703] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1421worker2 # [ 0.498138] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1422worker2 # [ 0.498335] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1423worker2 # [ 0.498352] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1424worker2 # [ 0.498380] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1425worker2 # [ 0.498396] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1426worker2 # [ 0.498841] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1427worker2 # [ 0.499022] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1428worker2 # [ 0.499038] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1429worker2 # [ 0.499067] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1430lb # [ 1.474526] usb 1-1: new high-speed USB device number 2 using ehci-pci1431client # [ 1.491278] usb 1-1: new high-speed USB device number 2 using ehci-pci1432worker2 # [ 0.538594] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1433worker2 # [ 0.538822] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1434worker2 # [ 0.538841] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1435worker2 # [ 0.538870] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1436worker2 # [ 0.539352] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1437worker2 # [ 0.539530] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1438worker2 # [ 0.539546] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1439worker2 # [ 0.539579] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1440worker2 # [ 0.540026] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1441worker2 # [ 0.540214] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1442worker2 # [ 0.540230] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1443worker2 # [ 0.540266] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1444worker2 # [ 0.540708] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1445worker2 # [ 0.540889] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1446worker2 # [ 0.540915] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1447worker2 # [ 0.540947] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1448worker2 # [ 0.541386] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1449worker2 # [ 0.541565] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1450worker2 # [ 0.541581] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1451worker2 # [ 0.541610] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1452worker2 # [ 0.541626] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1453worker2 # [ 0.542107] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1454worker2 # [ 0.542289] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1455worker2 # [ 0.542319] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1456worker2 # [ 0.542761] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1457worker2 # [ 0.542944] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1458worker2 # [ 0.542973] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1459worker2 # [ 0.543381] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1460worker2 # [ 0.543559] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1461worker2 # [ 0.543800] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1462worker2 # [ 0.544073] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1463worker2 # [ 0.544090] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1464worker2 # [ 0.544119] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1465worker2 # [ 0.544579] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1466worker2 # [ 0.544759] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1467worker2 # [ 0.544775] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1468worker2 # [ 0.544804] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1469worker2 # [ 0.545405] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1470worker2 # [ 0.545417] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1471worker2 # [ 0.545423] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1472worker2 # [ 0.545468] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1473worker2 # [ 0.545514] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1474worker2 # [ 0.545558] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1475worker2 # [ 0.545604] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1476worker2 # [ 0.545648] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1477worker2 # [ 0.545694] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1478worker2 # [ 0.545739] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1479worker2 # [ 0.545786] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1480worker2 # [ 0.545839] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1481worker2 # [ 0.545907] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1482worker2 # [ 0.545953] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1483worker2 # [ 0.545974] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1484worker2 # [ 0.545995] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1485worker2 # [ 0.546017] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1486worker2 # [ 0.546041] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1487worker2 # [ 0.546063] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1488worker2 # [ 0.546086] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1489worker2 # [ 0.546115] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1490worker2 # [ 0.546147] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1491worker2 # [ 0.546182] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1492worker2 # [ 0.546204] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1493worker2 # [ 0.546226] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1494worker2 # [ 0.546247] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1495worker2 # [ 0.546269] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1496worker2 # [ 0.546291] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1497worker2 # [ 0.546312] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1498worker2 # [ 0.546333] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1499worker2 # [ 0.546354] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1500worker2 # [ 0.546376] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1501worker2 # [ 0.546398] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1502worker2 # [ 0.546419] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1503worker2 # [ 0.546448] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1504worker2 # [ 0.546458] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1505worker2 # [ 0.546462] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1506worker2 # [ 0.547627] pci 0000:00:0a.0: enabling device (0000 -> 0002)1507worker2 # [ 0.632013] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1508worker2 # [ 0.635077] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1509worker2 # [ 0.645629] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1510worker2 # [ 0.657702] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1511worker2 # [ 0.660775] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1512worker2 # [ 0.662874] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1513worker2 # [ 0.682162] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1514worker2 # [ 0.700802] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1515client # [ 1.647383] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11516worker2 # [ 0.704888] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1517lb # [ 1.636608] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11518worker1 # [ 1.272449] Freeing initrd memory: 26148K1519worker1 # [ 1.283098] tun: Universal TUN/TAP device driver, 1.61520worker2 # [ 0.714654] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1521worker2 # [ 0.729433] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1522worker2 # [ 0.747218] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1523worker2 # [ 0.749989] msm_serial: driver initialized1524worker2 # [ 0.750166] SuperH (H)SCI(F) driver initialized1525worker2 # [ 0.750226] STM32 USART driver initialized1526worker1 # [ 1.287432] thunder_xcv, ver 1.01527worker1 # [ 1.287485] thunder_bgx, ver 1.01528worker1 # [ 1.287511] nicpf, ver 1.01529worker1 # [ 1.288130] e1000: Intel(R) PRO/1000 Network Driver1530worker1 # [ 1.288142] e1000: Copyright (c) 1999-2006 Intel Corporation.1531worker1 # [ 1.288173] e1000e: Intel(R) PRO/1000 Network Driver1532worker1 # [ 1.288180] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1533worker1 # [ 1.288212] igb: Intel(R) Gigabit Ethernet Network Driver1534worker1 # [ 1.288217] igb: Copyright (c) 2007-2014 Intel Corporation.1535worker1 # [ 1.288242] igbvf: Intel(R) Gigabit Virtual Function Network Driver1536worker1 # [ 1.288249] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1537worker1 # [ 1.288438] sky2: driver version 1.301538lb # [ 1.692026] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1539worker1 # [ 1.290100] usbcore: registered new interface driver usb-storage1540lb # [ 1.698590] systemd[1]: Detected virtualization qemu.1541worker1 # [ 1.290196] usbcore: registered new interface driver usbserial_generic1542lb # [ 1.699613] systemd[1]: Detected architecture arm64.1543worker1 # [ 1.290214] usbserial: USB Serial support registered for generic1544lb # [ 1.700561] systemd[1]: Running in initrd.1545worker1 # [ 1.290823] hv_vmbus: registering driver hyperv_keyboard1546worker1 # [ 1.334543] ehci-pci 0000:00:0a.0: EHCI Host Controller1547lb # [ 1.701942] systemd[1]: Initializing machine ID from random generator.1548worker1 # [ 1.334585] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11549lb # [ 1.703525] systemd[1]: Hostname set to <lb>.1550worker1 # [ 1.334777] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001551worker1 # [ 1.338698] rtc-pl031 9010000.pl031: registered as rtc01552worker1 # [ 1.338740] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T12:53:36 UTC (1789995216)1553client # [ 1.719205] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01554worker1 # [ 1.340956] i2c_dev: i2c /dev entries driver1555worker1 # [ 1.343104] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001556worker1 # [ 1.343458] hub 1-0:1.0: USB hub found1557worker1 # [ 1.343489] hub 1-0:1.0: 6 ports detected1558worker2 # [ 0.782751] loop: module loaded1559worker2 # [ 0.782969] virtio_blk virtio5: 1/0/0 default/read/poll queues1560lb # [ 1.718504] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01561worker1 # [ 1.347446] sdhci: Secure Digital Host Controller Interface driver1562worker1 # [ 1.347460] sdhci: Copyright(c) Pierre Ossman1563worker1 # [ 1.347882] Synopsys Designware Multimedia Card Interface Driver1564worker1 # [ 1.358721] sdhci-pltfm: SDHCI platform and OF driver helper1565worker2 # [ 0.792091] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1566worker1 # [ 1.362724] hid: raw HID events driver (C) Jiri Kosina1567worker1 # [ 1.363936] usbcore: registered new interface driver usbhid1568worker1 # [ 1.363947] usbhid: USB HID core driver1569worker1 # [ 1.366926] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1570client # [ 1.739303] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1571worker1 # [ 1.370487] drop_monitor: Initializing network drop monitor service1572worker1 # [ 1.370730] NET: Registered PF_INET6 protocol family1573client # [ 1.751425] systemd[1]: Detected virtualization qemu.1574worker2 # [ 0.799923] megasas: 07.734.00.00-rc11575worker1 # [ 1.373825] Segment Routing with IPv61576worker2 # [ 0.800725] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1577client # [ 1.753459] systemd[1]: Detected architecture arm64.1578worker1 # [ 1.373846] In-situ OAM (IOAM) with IPv61579worker1 # [ 1.373901] NET: Registered PF_PACKET protocol family1580worker2 # [ 0.806200] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001581worker2 # [ 0.806272] Intel/Sharp Extended Query Table at 0x00311582client # [ 1.755479] systemd[1]: Running in initrd.1583worker1 # [ 1.375872] 9pnet: Installing 9P2000 support1584client # [ 1.758126] systemd[1]: Initializing machine ID from random generator.1585worker1 # [ 1.378428] Key type dns_resolver registered1586worker2 # [ 0.812295] Using buffer write method1587client # [ 1.761089] systemd[1]: Hostname set to <client>.1588worker2 # [ 0.812379] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1589worker2 # [ 0.814319] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001590worker2 # [ 0.814353] Intel/Sharp Extended Query Table at 0x00311591worker1 # [ 1.386620] registered taskstats version 11592worker1 # [ 1.386797] Loading compiled-in X.509 certificates1593worker2 # [ 0.824665] Using buffer write method1594worker2 # [ 0.824709] Concatenating MTD devices:1595worker2 # [ 0.824713] (0): "0.flash"1596worker2 # [ 0.824717] (1): "0.flash"1597worker2 # [ 0.824721] into device "0.flash"1598worker1 # [ 1.396736] Demotion targets for Node 0: null1599worker1 # [ 1.396868] Key type .fscrypt registered1600worker1 # [ 1.396874] Key type fscrypt-provisioning registered1601worker1 # [ 1.396988] ima: No TPM chip found, activating TPM-bypass!1602worker1 # [ 1.397009] ima: Allocated hash algorithm: sha11603worker1 # [ 1.397039] ima: No architecture policies found1604worker1 # [ 1.401595] input: gpio-keys as /devices/platform/gpio-keys/input/input01605worker1 # [ 1.423985] clk: Disabling unused clocks1606worker1 # [ 1.424437] PM: genpd: Disabling unused power domains1607worker1 # [ 1.431508] Freeing unused kernel memory: 4736K1608worker1 # [ 1.431782] Run /init as init process1609worker1 # [ 1.450249] systemd[1]: Successfully made /usr/ read-only.1610client # [ 1.842652] usb 1-2: new high-speed USB device number 3 using ehci-pci1611lb # [ 1.846052] usb 1-2: new high-speed USB device number 3 using ehci-pci1612worker1 # [ 1.591134] usb 1-1: new high-speed USB device number 2 using ehci-pci1613lb # [ 2.031937] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21614lb # [ 2.034328] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01615worker2 # [ 1.190881] Freeing initrd memory: 26148K1616worker2 # [ 1.199031] tun: Universal TUN/TAP device driver, 1.61617worker2 # [ 1.203493] thunder_xcv, ver 1.01618worker2 # [ 1.203543] thunder_bgx, ver 1.01619worker2 # [ 1.203569] nicpf, ver 1.01620worker2 # [ 1.204556] e1000: Intel(R) PRO/1000 Network Driver1621worker2 # [ 1.204569] e1000: Copyright (c) 1999-2006 Intel Corporation.1622worker2 # [ 1.204641] e1000e: Intel(R) PRO/1000 Network Driver1623worker2 # [ 1.204709] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1624worker2 # [ 1.204798] igb: Intel(R) Gigabit Ethernet Network Driver1625worker2 # [ 1.204808] igb: Copyright (c) 2007-2014 Intel Corporation.1626worker2 # [ 1.204839] igbvf: Intel(R) Gigabit Virtual Function Network Driver1627worker2 # [ 1.204911] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1628worker2 # [ 1.205118] sky2: driver version 1.301629worker2 # [ 1.207047] usbcore: registered new interface driver usb-storage1630worker2 # [ 1.208011] ehci-pci 0000:00:0a.0: EHCI Host Controller1631worker2 # [ 1.208046] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11632worker2 # [ 1.208288] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001633worker2 # [ 1.218756] usbcore: registered new interface driver usbserial_generic1634worker2 # [ 1.218795] usbserial: USB Serial support registered for generic1635worker2 # [ 1.219197] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001636worker2 # [ 1.219617] hub 1-0:1.0: USB hub found1637worker2 # [ 1.219653] hub 1-0:1.0: 6 ports detected1638worker2 # [ 1.223775] hv_vmbus: registering driver hyperv_keyboard1639worker2 # [ 1.225452] rtc-pl031 9010000.pl031: registered as rtc01640worker2 # [ 1.225485] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T12:53:37 UTC (1789995217)1641worker2 # [ 1.225833] i2c_dev: i2c /dev entries driver1642worker2 # [ 1.231340] sdhci: Secure Digital Host Controller Interface driver1643worker2 # [ 1.231352] sdhci: Copyright(c) Pierre Ossman1644worker2 # [ 1.231771] Synopsys Designware Multimedia Card Interface Driver1645worker2 # [ 1.232200] sdhci-pltfm: SDHCI platform and OF driver helper1646worker2 # [ 1.234298] hid: raw HID events driver (C) Jiri Kosina1647worker2 # [ 1.234572] usbcore: registered new interface driver usbhid1648worker2 # [ 1.234578] usbhid: USB HID core driver1649worker1 # [ 1.792483] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1650worker1 # [ 1.806108] systemd[1]: Detected virtualization qemu.1651worker2 # [ 1.240500] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1652worker2 # [ 1.242512] drop_monitor: Initializing network drop monitor service1653worker1 # [ 1.808386] systemd[1]: Detected architecture arm64.1654worker2 # [ 1.242695] NET: Registered PF_INET6 protocol family1655worker2 # [ 1.246170] Segment Routing with IPv61656worker2 # [ 1.246194] In-situ OAM (IOAM) with IPv61657worker2 # [ 1.246228] NET: Registered PF_PACKET protocol family1658worker2 # [ 1.247918] 9pnet: Installing 9P2000 support1659worker2 # [ 1.250286] Key type dns_resolver registered1660worker1 # [ 1.810858] systemd[1]: Running in initrd.1661worker2 # [ 1.259941] registered taskstats version 11662worker2 # [ 1.260140] Loading compiled-in X.509 certificates1663worker1 # [ 1.832487] systemd[1]: Initializing machine ID from random generator.1664worker2 # [ 1.270316] Demotion targets for Node 0: null1665worker1 # [ 1.835662] systemd[1]: Hostname set to <worker1>.1666worker2 # [ 1.270698] Key type .fscrypt registered1667worker2 # [ 1.270709] Key type fscrypt-provisioning registered1668worker2 # [ 1.270870] ima: No TPM chip found, activating TPM-bypass!1669worker2 # [ 1.270900] ima: Allocated hash algorithm: sha11670worker2 # [ 1.274325] ima: No architecture policies found1671worker2 # [ 1.276329] input: gpio-keys as /devices/platform/gpio-keys/input/input01672worker1 # [ 1.843515] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11673worker2 # [ 1.299726] clk: Disabling unused clocks1674worker2 # [ 1.299766] PM: genpd: Disabling unused power domains1675worker2 # [ 1.305786] Freeing unused kernel memory: 4736K1676worker2 # [ 1.306465] Run /init as init process1677worker2 # [ 1.326532] systemd[1]: Successfully made /usr/ read-only.1678worker2 # [ 1.467672] usb 1-1: new high-speed USB device number 2 using ehci-pci1679worker1 # [ 2.076694] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680client # [ 2.465864] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21681client # [ 2.474755] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01682worker1 # [ 2.205540] usb 1-2: new high-speed USB device number 3 using ehci-pci1683worker2 # [ 1.642225] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684worker2 # [ 1.666736] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1685worker2 # [ 1.680039] systemd[1]: Detected virtualization qemu.1686worker2 # [ 1.682743] systemd[1]: Detected architecture arm64.1687worker2 # [ 1.693301] systemd[1]: Running in initrd.1688worker2 # [ 1.696757] systemd[1]: Initializing machine ID from random generator.1689worker2 # [ 1.700091] systemd[1]: Hostname set to <worker2>.1690worker2 # [ 1.738817] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01691lb # [ 2.687034] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker1 # [ 2.383679] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker1 # [ 2.388897] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694worker1 # [ 2.413151] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695client # [ 2.783885] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696worker2 # [ 1.879265] usb 1-2: new high-speed USB device number 3 using ehci-pci1697lb # [ 2.822135] systemd[1]: Queued start job for default target Initrd Default Target.1698lb # [ 2.828252] systemd[1]: Created slice Slice /system/modprobe.1699lb # [ 2.829282] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1700lb # [ 2.830352] systemd[1]: Expecting device /dev/disk/by-label/nixos...1701lb # [ 2.831172] systemd[1]: Reached target Path Units.1702lb # [ 2.831794] systemd[1]: Reached target Slice Units.1703lb # [ 2.832431] systemd[1]: Reached target Swaps.1704lb # [ 2.833016] systemd[1]: Reached target Timer Units.1705lb # [ 2.834175] systemd[1]: Listening on D-Bus System Message Bus Socket.1706lb # [ 2.835158] systemd[1]: Listening on Journal Socket (/dev/log).1707lb # [ 2.836031] systemd[1]: Listening on Journal Sockets.1708lb # [ 2.836801] systemd[1]: Listening on udev Control Socket.1709lb # [ 2.837569] systemd[1]: Listening on udev Kernel Socket.1710lb # [ 2.838313] systemd[1]: Reached target Socket Units.1711lb # [ 2.840548] systemd[1]: Starting Create List of Static Device Nodes...1712client # [ 2.857778] systemd[1]: Queued start job for default target Initrd Default Target.1713client # [ 2.868826] systemd[1]: Created slice Slice /system/modprobe.1714worker1 # [ 2.488679] systemd[1]: Queued start job for default target Initrd Default Target.1715lb # [ 2.853548] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1716lb # [ 2.854496] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1717client # [ 2.869868] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1718client # [ 2.870987] systemd[1]: Expecting device /dev/disk/by-label/nixos...1719client # [ 2.871030] systemd[1]: Reached target Path Units.1720client # [ 2.871047] systemd[1]: Reached target Slice Units.1721client # [ 2.871065] systemd[1]: Reached target Swaps.1722client # [ 2.871083] systemd[1]: Reached target Timer Units.1723lb # [ 2.862161] systemd[1]: Mounting Kernel Configuration File System...1724client # [ 2.871288] systemd[1]: Listening on D-Bus System Message Bus Socket.1725client # [ 2.871419] systemd[1]: Listening on Journal Socket (/dev/log).1726client # [ 2.871535] systemd[1]: Listening on Journal Sockets.1727client # [ 2.871644] systemd[1]: Listening on udev Control Socket.1728worker1 # [ 2.498033] systemd[1]: Created slice Slice /system/modprobe.1729client # [ 2.871747] systemd[1]: Listening on udev Kernel Socket.1730client # [ 2.871766] systemd[1]: Reached target Socket Units.1731worker1 # [ 2.499197] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1732worker1 # [ 2.500254] systemd[1]: Expecting device /dev/disk/by-label/nixos...1733client # [ 2.880781] systemd[1]: Starting Create List of Static Device Nodes...1734worker1 # [ 2.501144] systemd[1]: Reached target Path Units.1735worker1 # [ 2.501830] systemd[1]: Reached target Slice Units.1736worker1 # [ 2.502504] systemd[1]: Reached target Swaps.1737worker1 # [ 2.503124] systemd[1]: Reached target Timer Units.1738lb # [ 2.870382] systemd[1]: Starting Journal Service...1739worker1 # [ 2.504054] systemd[1]: Listening on D-Bus System Message Bus Socket.1740worker1 # [ 2.505118] systemd[1]: Listening on Journal Socket (/dev/log).1741worker1 # [ 2.506104] systemd[1]: Listening on Journal Sockets.1742worker1 # [ 2.506956] systemd[1]: Listening on udev Control Socket.1743client # [ 2.887492] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1744worker1 # [ 2.507935] systemd[1]: Listening on udev Kernel Socket.1745client # [ 2.888460] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1746worker1 # [ 2.508704] systemd[1]: Reached target Socket Units.1747worker1 # [ 2.511163] systemd[1]: Starting Create List of Static Device Nodes...1748client # [ 2.898960] systemd[1]: Mounting Kernel Configuration File System...1749worker1 # [ 2.519382] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1750worker1 # [ 2.520479] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1751lb # [ 2.894434] systemd[1]: Starting Load Kernel Modules...1752lb # [ 2.895538] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1753lb # [ 2.902166] systemd[1]: Starting Coldplug All udev Devices...1754client # [ 2.919249] systemd[1]: Starting Journal Service...1755worker1 # [ 2.540753] systemd[1]: Mounting Kernel Configuration File System...1756lb # [ 2.915565] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1757client # [ 2.938935] systemd[1]: Starting Load Kernel Modules...1758client # [ 2.939833] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1759worker1 # [ 2.563308] systemd[1]: Starting Journal Service...1760lb # [ 2.932385] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1761client # [ 2.946878] systemd[1]: Starting Coldplug All udev Devices...1762worker1 # [ 2.571284] systemd[1]: Starting Load Kernel Modules...1763worker1 # [ 2.572205] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1764lb # [ 2.941019] systemd-journald[73]: Collecting audit messages is disabled.1765lb # [ 2.946348] systemd[1]: Finished Create List of Static Device Nodes.1766lb # [ 2.946795] systemd[1]: Mounted Kernel Configuration File System.1767worker1 # [ 2.587307] systemd[1]: Starting Coldplug All udev Devices...1768lb # [ 2.954559] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1769client # [ 2.966868] systemd[1]: Finished Create List of Static Device Nodes.1770client # [ 2.968174] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1771client # [ 2.968508] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1772client # [ 2.968841] systemd[1]: Mounted Kernel Configuration File System.1773worker2 # [ 2.031334] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21774client # [ 2.983126] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1775worker2 # [ 2.037379] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01776worker1 # [ 2.611490] systemd[1]: Finished Create List of Static Device Nodes.1777worker1 # [ 2.613817] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1778worker1 # [ 2.622244] systemd-journald[73]: Collecting audit messages is disabled.1779worker1 # [ 2.627617] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1780worker1 # [ 2.628068] systemd[1]: Mounted Kernel Configuration File System.1781worker1 # [ 2.635562] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1782client # [ 3.022720] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1783client # [ 3.026657] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1784worker2 # [ 2.087091] systemd[1]: bpf-restrict-fs: LSM BPF program attached1785lb # [ 3.022781] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1786lb # [ 3.025705] systemd[1]: Starting Create Static Device Nodes in /dev...1787client # [ 3.041962] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01788client # [ 3.042247] [drm] features: -virgl +edid -resource_blob -host_visible1789client # [ 3.042257] [drm] features: -context_init1790worker1 # [ 2.672650] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1791lb # [ 3.035108] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1792lb # [ 3.038074] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1793lb # [ 3.057586] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01794client # [ 3.071302] [drm] number of scanouts: 11795client # [ 3.071344] [drm] number of cap sets: 01796lb # [ 3.057851] [drm] features: -virgl +edid -resource_blob -host_visible1797worker1 # [ 2.691249] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1798lb # [ 3.057862] [drm] features: -context_init1799lb # [ 3.062844] [drm] number of scanouts: 11800lb # [ 3.062877] [drm] number of cap sets: 01801client # [ 3.075324] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1802client # [ 3.075355] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01803worker1 # [ 2.703460] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01804worker1 # [ 2.703720] [drm] features: -virgl +edid -resource_blob -host_visible1805worker1 # [ 2.703734] [drm] features: -context_init1806worker1 # [ 2.704591] [drm] number of scanouts: 11807worker1 # [ 2.704614] [drm] number of cap sets: 01808client # [ 3.091299] systemd-journald[73]: Collecting audit messages is disabled.1809lb # [ 3.070549] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1810lb # [ 3.070582] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01811client # [ 3.096348] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1812worker1 # [ 2.719362] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1813worker1 # [ 2.722535] systemd[1]: Starting Create Static Device Nodes in /dev...1814client # [ 3.103063] systemd[1]: Starting Create Static Device Nodes in /dev...1815worker1 # [ 2.727622] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1816worker1 # [ 2.727657] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01817lb # [ 3.102747] Console: switching to colour frame buffer device 160x501818lb # [ 3.103372] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1819client # [ 3.114920] Console: switching to colour frame buffer device 160x501820client # [ 3.121903] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1821worker2 # [ 2.182887] systemd[1]: Queued start job for default target Initrd Default Target.1822lb # [ 3.119902] systemd[1]: Finished Create Static Device Nodes in /dev.1823lb # [ 3.120287] systemd[1]: Reached target Preparation for Local File Systems.1824lb # [ 3.120332] systemd[1]: Reached target Local File Systems.1825worker2 # [ 2.193589] systemd[1]: Created slice Slice /system/modprobe.1826worker2 # [ 2.194665] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1827lb # [ 3.126713] systemd[1]: Starting Rule-based Manager for Device Events and Files...1828worker2 # [ 2.195889] systemd[1]: Expecting device /dev/disk/by-label/nixos...1829worker2 # [ 2.195926] systemd[1]: Reached target Path Units.1830worker2 # [ 2.195946] systemd[1]: Reached target Slice Units.1831worker2 # [ 2.195964] systemd[1]: Reached target Swaps.1832worker2 # [ 2.195982] systemd[1]: Reached target Timer Units.1833worker2 # [ 2.196181] systemd[1]: Listening on D-Bus System Message Bus Socket.1834worker2 # [ 2.196485] systemd[1]: Listening on Journal Socket (/dev/log).1835worker1 # [ 2.759557] Console: switching to colour frame buffer device 160x501836worker2 # [ 2.196682] systemd[1]: Listening on Journal Sockets.1837client # [ 3.147885] systemd[1]: Finished Load Kernel Modules.1838worker2 # [ 2.196804] systemd[1]: Listening on udev Control Socket.1839worker2 # [ 2.196915] systemd[1]: Listening on udev Kernel Socket.1840worker2 # [ 2.196934] systemd[1]: Reached target Socket Units.1841worker2 # [ 2.206046] systemd[1]: Starting Create List of Static Device Nodes...1842worker2 # [ 2.213553] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1843client # [ 3.155317] systemd[1]: Starting Apply Kernel Variables...1844worker2 # [ 2.214515] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1845lb # [ 3.150918] systemd[1]: Finished Load Kernel Modules.1846lb # [ 3.155411] systemd[1]: Starting Apply Kernel Variables...1847worker2 # [ 2.227537] systemd[1]: Mounting Kernel Configuration File System...1848worker1 # [ 2.796008] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1849worker1 # [ 2.805001] systemd[1]: Finished Load Kernel Modules.1850worker1 # [ 2.815548] systemd[1]: Starting Apply Kernel Variables...1851client # [ 3.199529] systemd[1]: Finished Create Static Device Nodes in /dev.1852client # [ 3.199935] systemd[1]: Reached target Preparation for Local File Systems.1853client # [ 3.199982] systemd[1]: Reached target Local File Systems.1854lb # [ 3.186198] systemd[1]: Started Journal Service.1855lb # [ 3.169035] systemd-modules-load[74]: Inserted module 'dm_mod'1856lb # [ 3.174354] systemd-modules-load[74]: Module 'virtio_balloon' is built in1857client # [ 3.211082] systemd[1]: Starting Rule-based Manager for Device Events and Files...1858worker1 # [ 2.831301] systemd[1]: Finished Create Static Device Nodes in /dev.1859worker1 # [ 2.831650] systemd[1]: Reached target Preparation for Local File Systems.1860worker1 # [ 2.831689] systemd[1]: Reached target Local File Systems.1861worker1 # [ 2.869339] systemd[1]: Starting Rule-based Manager for Device Events and Files...1862lb # [ 3.220373] systemd-modules-load[74]: Module 'virtio_console' is built in1863worker2 # [ 2.311488] systemd[1]: Starting Journal Service...1864lb # [ 3.224595] systemd-modules-load[74]: Inserted module 'virtio_gpu'1865worker2 # [ 2.314067] systemd[1]: Starting Load Kernel Modules...1866worker2 # [ 2.314976] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1867lb # [ 3.231675] systemd-modules-load[74]: Module 'virtio_rng' is built in1868worker2 # [ 2.323471] systemd[1]: Starting Coldplug All udev Devices...1869client # [ 3.271166] systemd[1]: Finished Apply Kernel Variables.1870lb # [ 3.237430] systemd[1]: Starting Create System Files and Directories...1871worker2 # [ 2.331485] systemd[1]: Finished Create List of Static Device Nodes.1872worker2 # [ 2.333431] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1873lb # [ 3.245811] systemd[1]: Finished Apply Kernel Variables.1874worker1 # [ 2.905764] systemd[1]: Started Journal Service.1875worker1 # [ 2.878855] systemd-modules-load[74]: Inserted module 'dm_mod'1876worker1 # [ 2.888631] systemd-modules-load[74]: Module 'virtio_balloon' is built in1877worker1 # [ 2.889812] systemd-modules-load[74]: Module 'virtio_console' is built in1878worker1 # [ 2.890971] systemd-modules-load[74]: Inserted module 'virtio_gpu'1879worker1 # [ 2.891974] systemd-modules-load[74]: Module 'virtio_rng' is built in1880worker2 # [ 2.355723] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1881worker2 # [ 2.356449] systemd[1]: Mounted Kernel Configuration File System.1882client # [ 3.306134] systemd[1]: Started Journal Service.1883client # [ 3.290205] systemd-modules-load[74]: Inserted module 'dm_mod'1884worker1 # [ 2.905811] systemd[1]: Starting Create System Files and Directories...1885client # [ 3.297650] systemd-modules-load[74]: Module 'virtio_balloon' is built in1886worker2 # [ 2.371791] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1887worker1 # [ 2.906944] systemd[1]: Finished Apply Kernel Variables.1888client # [ 3.304496] systemd-modules-load[74]: Module 'virtio_console' is built in1889lb # [ 3.290409] systemd[1]: Finished Create System Files and Directories.1890client # [ 3.308560] systemd-modules-load[74]: Inserted module 'virtio_gpu'1891worker2 # [ 2.386355] systemd-journald[73]: Collecting audit messages is disabled.1892lb # [ 3.295238] systemd-udevd[79]: Using default interface naming scheme 'v261'.1893client # [ 3.316502] systemd-modules-load[74]: Module 'virtio_rng' is built in1894client # [ 3.318621] systemd[1]: Starting Create System Files and Directories...1895worker1 # [ 2.934008] systemd-udevd[81]: Using default interface naming scheme 'v261'.1896client # [ 3.329835] systemd-udevd[80]: Using default interface naming scheme 'v261'.1897worker1 # [ 2.953412] systemd[1]: Finished Create System Files and Directories.1898worker2 # [ 2.420375] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1899worker2 # [ 2.422200] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1900lb # [ 3.333326] systemd[1]: Started Rule-based Manager for Device Events and Files.1901worker2 # [ 2.431722] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01902worker2 # [ 2.431984] [drm] features: -virgl +edid -resource_blob -host_visible1903worker2 # [ 2.431996] [drm] features: -context_init1904worker2 # [ 2.432801] [drm] number of scanouts: 11905worker2 # [ 2.432820] [drm] number of cap sets: 01906worker2 # [ 2.439998] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1907worker1 # [ 2.978146] systemd[1]: Started Rule-based Manager for Device Events and Files.1908worker2 # [ 2.446259] systemd[1]: Starting Create Static Device Nodes in /dev...1909client # [ 3.369809] systemd[1]: Finished Create System Files and Directories.1910client # [ 3.381696] systemd[1]: Started Rule-based Manager for Device Events and Files.1911worker2 # [ 2.463072] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1912worker2 # [ 2.463107] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01913lb # [ 3.404636] systemd[1]: Starting Virtual Console Setup...1914worker2 # [ 2.503082] Console: switching to colour frame buffer device 160x501915worker1 # [ 3.050456] systemd[1]: Starting Virtual Console Setup...1916client # [ 3.453361] systemd[1]: Starting Virtual Console Setup...1917worker2 # [ 2.532767] systemd[1]: Finished Create Static Device Nodes in /dev.1918worker2 # [ 2.533030] systemd[1]: Reached target Preparation for Local File Systems.1919worker2 # [ 2.533056] systemd[1]: Reached target Local File Systems.1920worker2 # [ 2.537915] systemd[1]: Starting Rule-based Manager for Device Events and Files...1921worker2 # [ 2.539342] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1922lb # [ 3.464689] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1923lb # [ 3.469236] systemd[1]: Finished Virtual Console Setup.1924worker2 # [ 2.563410] systemd[1]: Finished Load Kernel Modules.1925worker2 # [ 2.571418] systemd[1]: Starting Apply Kernel Variables...1926worker1 # [ 3.108636] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1927worker1 # [ 3.113067] systemd[1]: Finished Virtual Console Setup.1928client # [ 3.516759] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1929client # [ 3.521228] systemd[1]: Finished Virtual Console Setup.1930worker2 # [ 2.655924] systemd[1]: Finished Apply Kernel Variables.1931worker2 # [ 2.648868] systemd-modules-load[74]: Inserted module 'dm_mod'1932worker2 # [ 2.669204] systemd[1]: Started Journal Service.1933worker2 # [ 2.656378] systemd-modules-load[74]: Module 'virtio_balloon' is built in1934worker2 # [ 2.657860] systemd-modules-load[74]: Module 'virtio_console' is built in1935worker2 # [ 2.659201] systemd-modules-load[74]: Inserted module 'virtio_gpu'1936worker2 # [ 2.676519] systemd-modules-load[74]: Module 'virtio_rng' is built in1937worker2 # [ 2.677624] systemd[1]: Starting Create System Files and Directories...1938worker2 # [ 2.679043] systemd-udevd[79]: Using default interface naming scheme 'v261'.1939worker2 # [ 2.730414] systemd[1]: Finished Create System Files and Directories.1940worker2 # [ 2.740543] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker2 # [ 2.830900] systemd[1]: Starting Virtual Console Setup...1942worker2 # [ 2.898268] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943worker2 # [ 2.910398] systemd[1]: Finished Virtual Console Setup.1944lb # [ 4.133799] systemd[1]: Finished Coldplug All udev Devices.1945lb # [ 4.134723] systemd[1]: Reached target System Initialization.1946lb # [ 4.135539] systemd[1]: Reached target Basic System.1947worker1 # [ 3.801839] systemd[1]: Finished Coldplug All udev Devices.1948worker1 # [ 3.802867] systemd[1]: Reached target System Initialization.1949worker1 # [ 3.804578] systemd[1]: Reached target Basic System.1950client # [ 4.195105] systemd[1]: Finished Coldplug All udev Devices.1951client # [ 4.196148] systemd[1]: Reached target System Initialization.1952client # [ 4.196967] systemd[1]: Reached target Basic System.1953worker1 # [ 3.951601] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1954lb # [ 4.341281] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1955lb # [ 4.347060] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1956client # [ 4.364094] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957client # [ 4.368717] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1958client # [ 4.385988] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 4.374847] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1960worker1 # [ 4.001766] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1961worker1 # [ 4.005996] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1962lb # [ 4.440661] systemd[1]: Found device /dev/disk/by-label/nixos.1963lb # [ 4.443261] systemd[1]: Reached target Initrd Root Device.1964client # [ 4.457361] systemd[1]: Found device /dev/disk/by-label/nixos.1965lb # [ 4.448389] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1966client # [ 4.462474] systemd[1]: Reached target Initrd Root Device.1967worker1 # [ 4.074199] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker1 # [ 4.077238] systemd[1]: Reached target Initrd Root Device.1969client # [ 4.467624] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1970worker1 # [ 4.082991] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1971worker2 # [ 3.542662] systemd[1]: Finished Coldplug All udev Devices.1972worker2 # [ 3.548869] systemd[1]: Reached target System Initialization.1973worker2 # [ 3.549873] systemd[1]: Reached target Basic System.1974lb # [ 4.506348] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1975client # [ 4.528907] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1976lb # [ 4.519270] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1977lb # [ 4.523336] systemd[1]: Mounting /sysroot...1978client # [ 4.536213] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1979worker1 # [ 4.147410] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1980client # [ 4.540975] systemd[1]: Mounting /sysroot...1981worker1 # [ 4.155217] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1982worker1 # [ 4.164991] systemd[1]: Mounting /sysroot...1983client # [ 4.606353] EXT4-fs (vda): mounted filesystem b7150ad9-3f38-4d9a-bad3-d91924cea87a r/w with ordered data mode. Quota mode: none.1984lb # [ 4.593408] EXT4-fs (vda): mounted filesystem 456f51ef-4fe2-4291-a68d-213e15070be5 r/w with ordered data mode. Quota mode: none.1985lb # [ 4.578327] systemd[1]: Mounted /sysroot.1986client # [ 4.593421] systemd[1]: Mounted /sysroot.1987lb # [ 4.581007] systemd[1]: Reached target Initrd Root File System.1988client # [ 4.594718] systemd[1]: Reached target Initrd Root File System.1989client # [ 4.599835] systemd[1]: Starting Mountpoints Configured in the Real Root...1990worker1 # [ 4.247258] EXT4-fs (vda): mounted filesystem 294d8c26-b609-45e4-ba9d-cdbfb2f7ce97 r/w with ordered data mode. Quota mode: none.1991lb # [ 4.607074] systemd[1]: Starting Mountpoints Configured in the Real Root...1992worker1 # [ 4.248887] systemd[1]: Mounted /sysroot.1993worker1 # [ 4.252149] systemd[1]: Reached target Initrd Root File System.1994worker1 # [ 4.288881] systemd[1]: Starting Mountpoints Configured in the Real Root...1995client # [ 4.756446] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1996lb # [ 4.745645] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.1997client # [ 4.760247] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1998worker1 # [ 4.373578] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1999client # [ 4.766137] systemd[1]: Reloading...2000worker1 # [ 4.379689] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...
loading 1,609 hidden lines…
3610worker2 # [ 23.297979] nix-grpc-daemon[779]: ts=2026-09-21T12:53:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3611worker2 # [ 23.303976] systemd[1]: Started Nix worker-protocol over gRPC.3612worker1 # [ 23.857935] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513613worker1 # [ 23.861583] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker2 # [ 23.336103] nix-grpc-daemon[779]: E0000 00:00:1789995239.628718 820 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3615worker2 # [ 23.355958] nix-grpc-daemon[779]: E0000 00:00:1789995239.649281 820 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3616worker2 # [ 23.360064] nix-grpc-daemon[779]: E0000 00:00:1789995239.652669 820 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3617worker2 # [ 23.377781] nix-grpc-daemon[779]: E0000 00:00:1789995239.670349 820 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3618worker2 # [ 23.393449] nix-grpc-daemon[779]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513619worker2 # [ 23.395322] nix-grpc-daemon[779]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3620worker1 # [ 24.063047] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513621worker1 # [ 24.073677] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3622worker2 # [ 23.597130] nix-grpc-daemon[779]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 23.602426] nix-grpc-daemon[779]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker1 # [ 24.269420] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513625worker1 # [ 24.274695] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker2 # [ 23.803600] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513627worker2 # [ 23.809049] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 24.475580] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513629worker1 # [ 24.481525] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker2 # [ 24.010003] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 24.015239] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker1 # [ 24.684367] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 24.688416] nix-grpc-daemon[773]: ts=2026-09-21T12:53:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker2 # [ 24.216384] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 24.221447] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 24.885064] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 24.890504] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 24.425251] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 24.428848] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 25.091538] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 25.097175] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 24.625581] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 24.631067] nix-grpc-daemon[779]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 25.297995] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 25.303353] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 24.832376] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 24.837563] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 25.504358] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 25.509471] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 25.038744] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 25.044161] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 25.712119] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 25.717215] nix-grpc-daemon[773]: ts=2026-09-21T12:54:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 25.245428] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 25.250917] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 25.917762] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 25.923075] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 25.454054] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 25.458264] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 26.124163] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 26.129428] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 25.655151] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 25.660938] nix-grpc-daemon[779]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 26.330336] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 26.335650] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 25.862144] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 25.867519] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 26.536557] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 26.541771] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 26.068772] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 26.074198] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 26.744989] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 26.749359] nix-grpc-daemon[773]: ts=2026-09-21T12:54:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 26.275578] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 26.281259] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 26.945548] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 26.951076] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 26.484163] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 26.488277] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 27.151964] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 27.157327] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 26.685231] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 26.690610] nix-grpc-daemon[779]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 27.358355] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 27.365959] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 26.891817] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 26.897357] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 27.565242] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 27.570685] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 27.098448] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 27.103639] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 27.772882] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 27.778178] nix-grpc-daemon[773]: ts=2026-09-21T12:54:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 27.305730] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 27.311044] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 27.979099] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 27.984685] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 27.514376] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 27.518735] nix-grpc-daemon[779]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 28.185437] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 28.190761] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 27.714943] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 27.718325] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 28.391654] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 28.397151] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 27.919658] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 27.925134] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 28.598125] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 28.603536] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 28.126310] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 28.131530] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 28.807818] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 28.811992] nix-grpc-daemon[773]: ts=2026-09-21T12:54:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 28.332817] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 28.337983] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 29.007624] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 29.013027] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 28.541101] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 28.545229] nix-grpc-daemon[779]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 29.213789] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 29.219069] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 28.741368] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 28.743978] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 29.419869] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 29.425284] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 28.945260] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 28.950398] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 29.626734] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 29.632892] nix-grpc-daemon[773]: ts=2026-09-21T12:54:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 29.151649] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 29.157033] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 29.835688] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 29.840103] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 29.358011] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 29.363197] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 30.036127] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 30.041647] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 29.566803] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 29.571038] nix-grpc-daemon[779]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 30.242648] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 30.248338] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 29.767427] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 29.772957] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 30.449171] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 30.454423] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 29.974043] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 29.979270] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 30.655270] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 30.659645] nix-grpc-daemon[773]: ts=2026-09-21T12:54:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 30.180504] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 30.185541] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 30.862420] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 30.866747] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 30.386332] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 30.390894] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 31.063096] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 31.068476] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 30.593491] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 30.599040] nix-grpc-daemon[779]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 31.269576] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 31.275072] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 30.800389] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 30.805919] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 31.475891] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 31.481284] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 31.007323] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 31.018238] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 31.682196] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 31.687805] nix-grpc-daemon[773]: ts=2026-09-21T12:54:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 31.216549] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 31.222089] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 31.890835] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 31.895127] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 31.423530] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 31.429261] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 32.091500] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 32.097316] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 31.635866] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 31.640504] nix-grpc-daemon[779]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 32.300427] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 32.309396] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 31.846434] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 31.859615] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 32.521035] nix-grpc-daemon[773]: ts=2026-09-21T12:54:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3785worker2 # [ 32.069628] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513786worker2 # [ 32.082742] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3787worker2 # [ 32.298099] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513788worker2 # [ 32.327942] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3789worker1 # [ 32.945511] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 32.950874] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791worker2 # [ 32.529272] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513792worker2 # [ 32.534788] nix-grpc-daemon[779]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3793worker1 # [ 33.151466] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513794worker1 # [ 33.157126] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795worker2 # [ 32.738970] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513796worker2 # [ 32.743960] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3797worker1 # [ 33.358102] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513798worker1 # [ 33.363609] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3799lb # [ 33.844535] rustfs-bucket-start[1016]: mb s3://farm3800lb # [ 33.853873] systemd[1]: Finished rustfs-bucket.service.3801lb # [ 33.865553] systemd[1]: Starting niks3 server...3802worker2 # [ 32.945494] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 32.951330] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker1 # [ 33.564706] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 33.570492] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker2 # [ 33.152743] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 33.158194] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808lb # [ 34.125649] postgres[1032]: [1032] ERROR: relation "goose_db_version" does not exist at character 363809lb # [ 34.126960] postgres[1032]: [1032] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3810worker1 # [ 33.771402] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_connected addr=lb:500513811worker1 # [ 33.777864] nix-grpc-daemon[773]: ts=2026-09-21T12:54:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812lb # [ 34.175591] niks3-server[1026]: 2026/09/21 12:54:09 OK 20241026095416_initial_model.sql (33.94ms)3813lb # [ 34.180557] niks3-server[1026]: 2026/09/21 12:54:09 OK 20251210153512_drop_unused_gin_index.sql (5.28ms)3814lb # [ 34.186888] niks3-server[1026]: 2026/09/21 12:54:09 OK 20251218171726_add_pins.sql (6.27ms)3815lb # [ 34.193694] niks3-server[1026]: 2026/09/21 12:54:09 OK 20260628120000_add_object_size_and_stats.sql (6.77ms)3816lb # [ 34.201104] niks3-server[1026]: 2026/09/21 12:54:09 OK 20260905000000_add_claims.sql (7.38ms)3817lb # [ 34.208401] niks3-server[1026]: 2026/09/21 12:54:09 OK 20260920000000_drop_claims.sql (6.85ms)3818lb # [ 34.210725] niks3-server[1026]: 2026/09/21 12:54:09 goose: successfully migrated database to version: 202609200000003819lb # [ 34.218079] niks3-server[1026]: 2026/09/21 12:54:09 OK 1_commit_pending_closure.sql (9.93ms)3820lb # [ 34.223978] niks3-server[1026]: 2026/09/21 12:54:09 OK 2_object_stats_trigger.sql (5.85ms)3821lb # [ 34.226212] niks3-server[1026]: 2026/09/21 12:54:09 goose: up to current file version: 23822lb # [ 34.230768] niks3-server[1026]: 2026/09/21 12:54:09 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3823lb # [ 34.277238] niks3-server[1026]: 2026/09/21 12:54:09 INFO Read proxy enabled — serving cache objects from S33824lb # [ 34.281607] niks3-server[1026]: 2026/09/21 12:54:09 INFO Using socket-activated listener address=0.0.0.0:57513825lb # [ 34.285543] niks3-server[1026]: 2026/09/21 12:54:09 INFO systemd watchdog enabled interval=15s3826lb # [ 34.286985] niks3-server[1026]: 2026/09/21 12:54:09 INFO Starting HTTP server address=0.0.0.0:57513827lb # [ 34.290133] systemd[1]: Started niks3 server.3828lb # [ 34.290844] systemd[1]: Reached target Multi-User System.3829lb # [ 34.291584] systemd[1]: Startup finished in 1.321s (kernel) + 5.723s (initrd) + 27.239s (userspace) = 34.284s.3830worker2 # [ 33.359435] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513831worker2 # [ 33.364933] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3832lb # [ 34.298885] niks3-server[1026]: 2026/09/21 12:54:09 INFO lead: acquired remote=192.168.1.4:404923833worker2 # [ 33.372991] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_take_over3834worker1 # [ 33.979702] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513835worker1 # [ 33.984533] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3836worker2 # [ 33.566395] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513837worker2 # [ 33.571866] nix-grpc-daemon[779]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3838worker1 # [ 34.179934] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513839worker1 # [ 34.185019] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3840worker2 # [ 33.773423] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_connected addr=lb:500513841worker2 # [ 33.776305] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842worker1 # [ 34.386123] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513843worker1 # [ 34.390657] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3844worker2 # [ 33.977990] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_connected addr=lb:500513845worker2 # [ 33.983015] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3846worker1 # [ 34.591877] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513847worker1 # [ 34.597646] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3848worker2 # [ 34.184422] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_connected addr=lb:500513849worker2 # [ 34.189689] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850worker1 # [ 34.798423] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_connected addr=lb:500513851worker1 # [ 34.804094] nix-grpc-daemon[773]: ts=2026-09-21T12:54:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3852worker2 # [ 34.390487] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=scheduler_connected addr=lb:500513853worker2 # [ 34.410044] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=oidc_jwks provider=mock keys=13854worker2 # [ 34.412432] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:397003855worker2 # [ 34.415522] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03856lb: (finished: waiting for unit niks3.service, in 17.39 seconds)3857lb: waiting for TCP port 5751 on localhost3858worker1 # [ 35.006228] nix-grpc-daemon[773]: ts=2026-09-21T12:54:10Z level=info event=scheduler_connected addr=lb:500513859worker2 # [ 34.456419] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:397003860worker2 # [ 34.459353] nix-grpc-daemon[779]: ts=2026-09-21T12:54:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03861lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3862lb: (finished: waiting for TCP port 5751 on localhost, in 0.11 seconds)3863lb: waiting for unit envoy.service3864lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3865lb: waiting for TCP port 50051 on localhost3866lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3867lb # [ 35.572282] envoy[915]: conn peer=[::1]:37050 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33868lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3869worker2 # [ 34.839018] systemd[1]: Starting scheduler-token.service...3870worker2 # [ 34.935033] systemd[1]: scheduler-token.service: Deactivated successfully.3871worker2 # [ 34.939214] systemd[1]: Finished scheduler-token.service.3872worker2 # [ 34.941822] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 98ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3873lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3874lb: must succeed: curl -sf localhost:9901/clusters3875lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3876lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3877subtest: one node schedules and both builders hold a WorkerSession on it3878lb: must succeed: curl -sf localhost:9901/clusters3879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3880worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3881worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3882lb: must succeed: curl -sf localhost:9901/clusters3883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3884lb: must succeed: curl -sf localhost:9901/clusters3885lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3886lb: must succeed: curl -sf localhost:9901/clusters3887lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3888lb: must succeed: curl -sf localhost:9901/clusters3889lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3890worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3891worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3892worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3893worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3894lb: must succeed: curl -sf localhost:9901/clusters3895lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3896lb: must succeed: curl -sf localhost:9901/clusters3897lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3898worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3899worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3900worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3901worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3902worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3903worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3904worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3905worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3906(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.86 seconds)3907subtest: balancer auth: client cert, bearer token, nothing, unknown CN3908worker1 # [ 36.861301] nix-grpc-daemon[773]: ts=2026-09-21T12:54:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544183909worker1 # [ 36.882765] systemd[1]: Started Nix Daemon.3910worker1 # [ 39.387891] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)3911worker1 # [ 39.415499] nix-grpc-daemon[773]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)3912worker1 # [ 39.428361] nix-grpc-daemon[773]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)3913worker1 # [ 39.575958] nix-grpc-daemon[773]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 3/5)3914worker1 # [ 39.639287] nix-grpc-daemon[773]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5)3915worker1 # [ 40.290383] nix-grpc-daemon[773]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3916worker1 # [ 40.290945] nix-grpc-daemon[773]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3917worker1 # [ 40.324281] nix-grpc-daemon[773]: ts=2026-09-21T12:54:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:544183918worker1 # [ 40.339522] nix-grpc-daemon[773]: ts=2026-09-21T12:54:15Z level=debug event=path_info_miss path=wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test error="error: path '/nix/store/wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test' is required, but there is no substituter that can build it"3919worker1 # [ 40.352335] nix-grpc-daemon[773]: ts=2026-09-21T12:54:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:544183920worker1 # [ 40.363004] nix-grpc-daemon[773]: ts=2026-09-21T12:54:15Z level=debug event=path_info_miss path=wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test error="error: path '/nix/store/wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test' is required, but there is no substituter that can build it"3921client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3922client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3923client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3924client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3925worker1 # [ 40.819291] nix-grpc-daemon[773]: ts=2026-09-21T12:54:16Z level=info event=oidc_jwks provider=mock keys=13926worker1 # [ 40.821497] nix-grpc-daemon[773]: ts=2026-09-21T12:54:16Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:544183927worker2 # [ 40.295372] nix-grpc-daemon[779]: ts=2026-09-21T12:54:16Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:418063928worker2 # [ 40.312894] systemd[1]: Started Nix Daemon.3929worker2 # [ 40.601591] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3930worker2 # [ 40.628091] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)3931worker2 # [ 40.689195] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)3932worker2 # [ 40.857427] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 3/5)3933worker2 # [ 40.922223] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 4/5)3934worker2 # [ 40.969274] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3935worker2 # [ 40.978110] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3936worker2 # [ 40.993076] nix-grpc-daemon[779]: ts=2026-09-21T12:54:17Z level=debug event=path_info_miss path=wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test error="error: path '/nix/store/wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test' is required, but there is no substituter that can build it"3937worker2 # [ 41.010370] nix-grpc-daemon[779]: ts=2026-09-21T12:54:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:418063938worker2 # [ 41.018148] nix-grpc-daemon[779]: ts=2026-09-21T12:54:17Z level=debug event=path_info_miss path=wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test error="error: path '/nix/store/wzyai2lcdcb8sgsxr1wpx21k8i95k7mn-nixos-system-client-test' is required, but there is no substituter that can build it"3939client: must succeed: rm -r /var/lib/nix-grpc-store3940client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)3941worker1 # [ 41.923235] nix-grpc-daemon[773]: ts=2026-09-21T12:54:17Z level=info event=denied method=QueryMissing cn=- role=none3942worker2 # [ 41.631630] nix-grpc-daemon[779]: ts=2026-09-21T12:54:17Z level=info event=denied method=QueryMissing cn=stranger role=none3943(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.75 seconds)3944subtest: DAG build is scheduled across workers and lands in the cache3945client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23946worker2 # [ 41.890120] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418063947client # don't know how to build these paths:3948client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3949worker1 # [ 42.456555] nix-grpc-daemon[773]: ts=2026-09-21T12:54:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544183950worker2 # [ 41.939281] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03951worker2 # [ 41.947029] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13952worker2 # [ 41.986726] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418063953worker2 # [ 42.009143] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418063954client # copying 1 paths to 'lb:50051'...3955worker2 # [ 42.026112] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418063956client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3957worker2 # [ 42.056955] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3958worker2 # [ 42.095141] nix-grpc-daemon[779]: ts=2026-09-21T12:54:18Z level=info event=niks3_push_started pid=8793959worker2 # [ 42.411692] nix-grpc-daemon[879]: time=2026-09-21T12:54:18.703Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""3960worker2 # [ 42.606454] nix-daemon[864]: accepted connection from pid 892, user nix-grpc-daemon (trusted)3961worker2 # [ 42.626790] nix-daemon[864]: reaped child process 895, status = succeeded3962worker2 # [ 42.802814] nix-daemon[864]: accepted connection from pid 897, user nix-grpc-daemon (trusted)3963worker2 # [ 42.823457] nix-daemon[864]: reaped child process 900, status = succeeded3964lb # [ 43.763169] envoy[915]: conn peer=[2001:db8:1::1]:57910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=35603965lb # [ 43.766157] envoy[915]: conn peer=[2001:db8:1::1]:57906 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2613 tx=406 ms=7753966lb # [ 43.769432] envoy[915]: conn peer=[2001:db8:1::1]:57910 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=123967lb # [ 43.772203] envoy[915]: conn peer=[2001:db8:1::1]:57914 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=133968lb # [ 43.778441] niks3-server[1026]: 2026/09/21 12:54:19 INFO Received uploads request method=POST path=/api/pending_closures3969worker2 # [ 42.861992] nix-grpc-daemon[879]: time=2026-09-21T12:54:19.155Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3970worker2 # [ 42.865891] nix-grpc-daemon[879]: time=2026-09-21T12:54:19.157Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3971lb # [ 43.824485] niks3-server[1026]: 2026/09/21 12:54:19 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3972lb # [ 43.833144] niks3-server[1026]: 2026/09/21 12:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3973lb # [ 43.839952] niks3-server[1026]: 2026/09/21 12:54:19 INFO Signed narinfos id=1 count=13974worker2 # [ 42.909032] nix-grpc-daemon[879]: time=2026-09-21T12:54:19.201Z level=INFO msg="Uploading 1 narinfos"3975lb # [ 43.855051] niks3-server[1026]: 2026/09/21 12:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3976lb # [ 43.868243] niks3-server[1026]: 2026/09/21 12:54:19 INFO Completed upload id=13977worker2 # [ 42.942100] nix-grpc-daemon[879]: time=2026-09-21T12:54:19.235Z level=INFO msg="Upload complete. (521ms)"3978lb # [ 43.879521] niks3-server[1026]: 2026/09/21 12:54:19 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3979worker2 # [ 42.954773] nix-grpc-daemon[779]: ts=2026-09-21T12:54:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=5043980worker2 # [ 42.969687] nix-grpc-daemon[879]: time=2026-09-21T12:54:19.246Z level=WARN msg="Failed to register uploaded object" key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo error="server returned 404: object is not pending upload\n"3981worker2 # [ 42.979672] nix-grpc-daemon[779]: ts=2026-09-21T12:54:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418063982worker2 # [ 42.985280] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3983worker2 # [ 42.987025] nix-daemon[864]: reaped child process 877, status = succeeded3984client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3985worker2 # [ 42.994760] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3986client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3987client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds3988client # farm-a> LOG-farm-a-t13989worker2 # [ 43.822541] nix-daemon[864]: reaped child process 904, status = succeeded3990worker2 # [ 43.989658] nix-daemon[864]: accepted connection from pid 920, user nix-grpc-daemon (trusted)3991worker2 # [ 44.007800] nix-daemon[864]: reaped child process 923, status = succeeded3992lb # [ 44.950104] niks3-server[1026]: 2026/09/21 12:54:20 INFO Received uploads request method=POST path=/api/pending_closures3993worker2 # [ 44.037308] nix-grpc-daemon[879]: time=2026-09-21T12:54:20.329Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3994worker2 # [ 44.039190] nix-grpc-daemon[879]: time=2026-09-21T12:54:20.329Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3995lb # [ 45.018307] niks3-server[1026]: 2026/09/21 12:54:20 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3996worker2 # [ 44.101718] nix-grpc-daemon[879]: time=2026-09-21T12:54:20.393Z level=INFO msg="Uploading 1 narinfos"3997lb # [ 45.033261] niks3-server[1026]: 2026/09/21 12:54:20 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3998lb # [ 45.041182] niks3-server[1026]: 2026/09/21 12:54:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3999lb # [ 45.046060] niks3-server[1026]: 2026/09/21 12:54:20 INFO Signed narinfos id=2 count=14000lb # [ 45.054479] niks3-server[1026]: 2026/09/21 12:54:20 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4001lb # [ 45.059584] niks3-server[1026]: 2026/09/21 12:54:20 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4002lb # [ 45.062752] niks3-server[1026]: 2026/09/21 12:54:20 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4003worker2 # [ 44.136290] nix-grpc-daemon[879]: time=2026-09-21T12:54:20.429Z level=INFO msg="Upload complete. (330ms)"4004lb # [ 45.066253] niks3-server[1026]: 2026/09/21 12:54:20 INFO Completed upload id=24005worker2 # [ 44.146974] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14006worker2 # [ 44.157630] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04007worker2 # [ 44.160516] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=14008worker2 # [ 44.162739] nix-daemon[864]: reaped child process 902, status = succeeded4009worker2 # [ 44.202995] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064010worker2 # [ 44.219545] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064011worker2 # [ 44.234992] nix-grpc-daemon[879]: time=2026-09-21T12:54:20.528Z level=INFO msg="All 1 paths already cached"4012client # copying 1 paths to 'lb:50051'...4013worker2 # [ 44.245875] nix-grpc-daemon[779]: ts=2026-09-21T12:54:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064014client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4015worker2 # [ 44.272416] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4016worker2 # [ 44.502767] nix-daemon[864]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4017worker2 # [ 44.522386] nix-daemon[864]: reaped child process 933, status = succeeded4018worker2 # [ 44.701347] nix-daemon[864]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4019worker2 # [ 44.720798] nix-daemon[864]: reaped child process 938, status = succeeded4020lb # [ 45.656660] niks3-server[1026]: 2026/09/21 12:54:20 INFO Received uploads request method=POST path=/api/pending_closures4021worker2 # [ 44.738254] nix-grpc-daemon[879]: time=2026-09-21T12:54:21.031Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4022worker2 # [ 44.739746] nix-grpc-daemon[879]: time=2026-09-21T12:54:21.031Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4023lb # [ 45.690155] niks3-server[1026]: 2026/09/21 12:54:20 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4024worker2 # [ 44.772440] nix-grpc-daemon[879]: time=2026-09-21T12:54:21.065Z level=INFO msg="Uploading 1 narinfos"4025lb # [ 45.708238] niks3-server[1026]: 2026/09/21 12:54:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4026lb # [ 45.718382] niks3-server[1026]: 2026/09/21 12:54:20 INFO Signed narinfos id=3 count=14027lb # [ 45.732366] niks3-server[1026]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4028lb # [ 45.736437] niks3-server[1026]: 2026/09/21 12:54:21 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4029lb # [ 45.743549] niks3-server[1026]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4030lb # [ 45.748154] niks3-server[1026]: 2026/09/21 12:54:21 INFO Completed upload id=34031worker2 # [ 44.818277] nix-grpc-daemon[879]: time=2026-09-21T12:54:21.111Z level=INFO msg="Upload complete. (518ms)"4032worker2 # [ 44.824281] nix-grpc-daemon[779]: ts=2026-09-21T12:54:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=6244033worker2 # [ 44.832647] nix-grpc-daemon[779]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064034worker2 # [ 44.845336] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4035worker2 # [ 44.847608] nix-daemon[864]: reaped child process 927, status = succeeded4036worker2 # [ 44.857225] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4037client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4038client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4039client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4040client # farm-b> LOG-farm-b-t14041worker2 # [ 46.085777] nix-daemon[864]: reaped child process 942, status = succeeded4042worker2 # [ 46.265049] nix-daemon[864]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4043worker2 # [ 46.285516] nix-daemon[864]: reaped child process 961, status = succeeded4044lb # [ 47.230287] niks3-server[1026]: 2026/09/21 12:54:22 INFO Received uploads request method=POST path=/api/pending_closures4045worker2 # [ 46.316974] nix-grpc-daemon[879]: time=2026-09-21T12:54:22.609Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4046worker2 # [ 46.318488] nix-grpc-daemon[879]: time=2026-09-21T12:54:22.609Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4047lb # [ 47.260926] niks3-server[1026]: 2026/09/21 12:54:22 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4048lb # [ 47.267422] niks3-server[1026]: 2026/09/21 12:54:22 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4049worker2 # [ 46.342151] nix-grpc-daemon[879]: time=2026-09-21T12:54:22.635Z level=INFO msg="Uploading 1 narinfos"4050lb # [ 47.273677] niks3-server[1026]: 2026/09/21 12:54:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4051lb # [ 47.278100] niks3-server[1026]: 2026/09/21 12:54:22 INFO Signed narinfos id=4 count=14052lb # [ 47.280898] niks3-server[1026]: 2026/09/21 12:54:22 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4053lb # [ 47.286840] niks3-server[1026]: 2026/09/21 12:54:22 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4054lb # [ 47.289704] niks3-server[1026]: 2026/09/21 12:54:22 INFO Completed upload id=44055worker2 # [ 46.360104] nix-grpc-daemon[879]: time=2026-09-21T12:54:22.653Z level=INFO msg="Upload complete. (292ms)"4056lb # [ 47.291925] niks3-server[1026]: 2026/09/21 12:54:22 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4057worker2 # [ 46.369809] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14058worker2 # [ 46.383910] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04059worker2 # [ 46.386796] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14060worker2 # [ 46.391222] nix-daemon[864]: reaped child process 940, status = succeeded4061worker2 # [ 46.421740] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064062worker2 # [ 46.433069] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064063worker2 # [ 46.443251] nix-grpc-daemon[879]: time=2026-09-21T12:54:22.736Z level=INFO msg="All 2 paths already cached"4064client # copying 1 paths to 'lb:50051'...4065worker2 # [ 46.451263] nix-grpc-daemon[779]: ts=2026-09-21T12:54:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064066client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4067worker2 # [ 46.474140] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4068worker2 # [ 46.677493] nix-daemon[864]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4069worker2 # [ 46.699357] nix-daemon[864]: reaped child process 970, status = succeeded4070worker2 # [ 46.880855] nix-daemon[864]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4071worker2 # [ 46.901662] nix-daemon[864]: reaped child process 975, status = succeeded4072lb # [ 47.842583] niks3-server[1026]: 2026/09/21 12:54:23 INFO Received uploads request method=POST path=/api/pending_closures4073worker2 # [ 46.923872] nix-grpc-daemon[879]: time=2026-09-21T12:54:23.217Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4074worker2 # [ 46.925544] nix-grpc-daemon[879]: time=2026-09-21T12:54:23.217Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4075lb # [ 47.874091] niks3-server[1026]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4076worker2 # [ 46.962379] nix-grpc-daemon[879]: time=2026-09-21T12:54:23.255Z level=INFO msg="Uploading 1 narinfos"4077lb # [ 47.884538] niks3-server[1026]: 2026/09/21 12:54:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4078lb # [ 47.897664] niks3-server[1026]: 2026/09/21 12:54:23 INFO Signed narinfos id=5 count=14079lb # [ 47.903500] niks3-server[1026]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4080lb # [ 47.910299] niks3-server[1026]: 2026/09/21 12:54:23 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4081worker2 # [ 46.987610] nix-grpc-daemon[879]: time=2026-09-21T12:54:23.281Z level=INFO msg="Upload complete. (498ms)"4082lb # [ 47.917333] niks3-server[1026]: 2026/09/21 12:54:23 INFO Completed upload id=54083lb # [ 47.920463] niks3-server[1026]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4084worker2 # [ 46.992300] nix-grpc-daemon[779]: ts=2026-09-21T12:54:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=7524085worker2 # [ 47.000625] nix-grpc-daemon[779]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064086worker2 # [ 47.004368] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4087worker2 # [ 47.009276] nix-daemon[864]: reaped child process 965, status = succeeded4088client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4089worker2 # [ 47.020135] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4090client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4091client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4092client # farm-top> LOG-farm-top-t14093worker2 # [ 48.468660] nix-daemon[864]: reaped child process 979, status = succeeded4094worker2 # [ 48.652976] nix-daemon[864]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4095worker2 # [ 48.671452] nix-daemon[864]: reaped child process 999, status = succeeded4096lb # [ 49.614005] niks3-server[1026]: 2026/09/21 12:54:24 INFO Received uploads request method=POST path=/api/pending_closures4097worker2 # [ 48.696536] nix-grpc-daemon[879]: time=2026-09-21T12:54:24.988Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4098worker2 # [ 48.698437] nix-grpc-daemon[879]: time=2026-09-21T12:54:24.988Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4099lb # [ 49.648775] niks3-server[1026]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4100lb # [ 49.654447] niks3-server[1026]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4101lb # [ 49.661681] niks3-server[1026]: 2026/09/21 12:54:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4102lb # [ 49.665996] niks3-server[1026]: 2026/09/21 12:54:24 INFO Signed narinfos id=6 count=14103worker2 # [ 48.736494] nix-grpc-daemon[879]: time=2026-09-21T12:54:25.029Z level=INFO msg="Uploading 1 narinfos"4104lb # [ 49.668728] niks3-server[1026]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4105lb # [ 49.676744] niks3-server[1026]: 2026/09/21 12:54:24 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4106lb # [ 49.682744] niks3-server[1026]: 2026/09/21 12:54:24 INFO Completed upload id=64107worker2 # [ 48.753101] nix-grpc-daemon[879]: time=2026-09-21T12:54:25.046Z level=INFO msg="Upload complete. (319ms)"4108lb # [ 49.684971] niks3-server[1026]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4109worker2 # [ 48.765139] nix-grpc-daemon[779]: ts=2026-09-21T12:54:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14110worker2 # [ 48.782175] nix-daemon[864]: reaped child process 977, status = succeeded4111client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.17 seconds)4112client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4113client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4114client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4115client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4116client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4117client # copying 3 paths...4118client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4119client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4120client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4121client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4122(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.83 seconds)4123subtest: repeat is answered Cached by the scheduler without building4124worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4125worker1 # finding garbage collector roots...4126worker1 # deleting specified paths...4127worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4128worker1 # deleting unused links...4129worker1 # note: hard linking is currently saving 0.0 KiB4130worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4131worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4132worker2 # finding garbage collector roots...4133worker2 # deleting specified paths...4134worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4135worker2 # deleting unused links...4136worker2 # note: hard linking is currently saving 0.0 KiB4137worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.26 seconds)4138client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24139client # don't know how to build these paths:4140client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4141worker1 # [ 50.881694] nix-grpc-daemon[773]: ts=2026-09-21T12:54:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544184142worker1 # [ 50.889946] nix-grpc-daemon[773]: ts=2026-09-21T12:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184143worker2 # [ 50.365532] nix-grpc-daemon[779]: ts=2026-09-21T12:54:26Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14144client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.36 seconds)4145client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4146client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4147lb: must succeed: curl -sf localhost:9901/clusters4148lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4149lb: must succeed: curl -sf localhost:9901/clusters4150lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4151worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4152worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4153(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.46 seconds)4154subtest: inputs already in the cache are not sent to the scheduler4155lb: must succeed: curl -sf localhost:9901/clusters4156lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4157lb: must succeed: curl -sf localhost:9901/clusters4158lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4159worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4160worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4161client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24162client # don't know how to build these paths:4163client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4164worker2 # [ 51.413635] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418064165worker2 # [ 51.422622] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064166worker2 # [ 51.430802] nix-grpc-daemon[879]: time=2026-09-21T12:54:27.722Z level=INFO msg="All 2 paths already cached"4167worker2 # [ 51.445206] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04168worker2 # [ 51.452979] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14169worker2 # [ 51.491112] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064170worker2 # [ 51.509245] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064171worker2 # [ 51.524855] nix-grpc-daemon[879]: time=2026-09-21T12:54:27.818Z level=INFO msg="All 2 paths already cached"4172client # copying 1 paths to 'lb:50051'...4173client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4174worker2 # [ 51.537569] nix-grpc-daemon[779]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064175worker2 # [ 51.555265] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4176worker2 # [ 51.757275] nix-daemon[864]: accepted connection from pid 1028, user nix-grpc-daemon (trusted)4177worker2 # [ 51.776644] nix-daemon[864]: reaped child process 1031, status = succeeded4178worker2 # [ 51.947647] nix-daemon[864]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4179worker2 # [ 51.968497] nix-daemon[864]: reaped child process 1036, status = succeeded4180lb # [ 52.905468] niks3-server[1026]: 2026/09/21 12:54:28 INFO Received uploads request method=POST path=/api/pending_closures4181worker2 # [ 51.986449] nix-grpc-daemon[879]: time=2026-09-21T12:54:28.279Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4182worker2 # [ 51.987980] nix-grpc-daemon[879]: time=2026-09-21T12:54:28.279Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4183lb # [ 52.936161] niks3-server[1026]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4184lb # [ 52.950820] niks3-server[1026]: 2026/09/21 12:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4185worker2 # [ 52.029257] nix-grpc-daemon[879]: time=2026-09-21T12:54:28.322Z level=INFO msg="Uploading 1 narinfos"4186lb # [ 52.964060] niks3-server[1026]: 2026/09/21 12:54:28 INFO Signed narinfos id=7 count=14187lb # [ 52.967317] niks3-server[1026]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4188lb # [ 52.980550] niks3-server[1026]: 2026/09/21 12:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4189worker2 # [ 52.056818] nix-grpc-daemon[879]: time=2026-09-21T12:54:28.350Z level=INFO msg="Upload complete. (482ms)"4190lb # [ 52.987167] niks3-server[1026]: 2026/09/21 12:54:28 INFO Completed upload id=74191lb # [ 52.990763] niks3-server[1026]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4192worker2 # [ 52.064113] nix-grpc-daemon[779]: ts=2026-09-21T12:54:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=7524193worker2 # [ 52.067795] nix-grpc-daemon[779]: ts=2026-09-21T12:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064194worker2 # [ 52.077057] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4195worker2 # [ 52.080718] nix-daemon[864]: reaped child process 1026, status = succeeded4196client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4197worker2 # [ 52.091110] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4198client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4199lb # [ 53.761199] envoy[915]: conn peer=[2001:db8:1::1]:57918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=4080 ms=69024200lb # [ 53.764376] envoy[915]: conn peer=[2001:db8:1::1]:33498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=694201client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4202client # farm-top> LOG-farm-top-t1b4203worker2 # [ 53.534911] nix-daemon[864]: reaped child process 1040, status = succeeded4204worker2 # [ 53.724434] nix-daemon[864]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4205worker2 # [ 53.743414] nix-daemon[864]: reaped child process 1059, status = succeeded4206lb # [ 54.687462] niks3-server[1026]: 2026/09/21 12:54:29 INFO Received uploads request method=POST path=/api/pending_closures4207worker2 # [ 53.768621] nix-grpc-daemon[879]: time=2026-09-21T12:54:30.061Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4208worker2 # [ 53.774692] nix-grpc-daemon[879]: time=2026-09-21T12:54:30.064Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4209lb # [ 54.726655] niks3-server[1026]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4210lb # [ 54.741185] niks3-server[1026]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4211lb # [ 54.754155] niks3-server[1026]: 2026/09/21 12:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4212worker2 # [ 53.829477] nix-grpc-daemon[879]: time=2026-09-21T12:54:30.122Z level=INFO msg="Uploading 1 narinfos"4213lb # [ 54.761546] niks3-server[1026]: 2026/09/21 12:54:30 INFO Signed narinfos id=8 count=14214lb # [ 54.766886] niks3-server[1026]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4215lb # [ 54.775471] niks3-server[1026]: 2026/09/21 12:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4216lb # [ 54.784626] niks3-server[1026]: 2026/09/21 12:54:30 INFO Completed upload id=84217lb # [ 54.787359] niks3-server[1026]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4218worker2 # [ 53.853393] nix-grpc-daemon[879]: time=2026-09-21T12:54:30.146Z level=INFO msg="Upload complete. (352ms)"4219worker2 # [ 53.869573] nix-grpc-daemon[779]: ts=2026-09-21T12:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14220worker2 # [ 53.886570] nix-daemon[864]: reaped child process 1038, status = succeeded4221client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.76 seconds)4222client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4223client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4224lb: must succeed: curl -sf localhost:9901/clusters4225lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4226lb: must succeed: curl -sf localhost:9901/clusters4227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4228worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4229worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4230(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.39 seconds)4231subtest: two clients wanting the same drv build it once4232client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4233client # [ 55.361935] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4234client # Running as unit: dedup1.service; invocation ID: 2daec71e29db44a1871931ff3427fb244235client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4236client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4237worker2 # [ 54.682596] nix-grpc-daemon[779]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418064238client # [ 55.633146] nix[879]: don't know how to build these paths:4239client # [ 55.634817] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4240worker2 # [ 54.703063] nix-grpc-daemon[779]: ts=2026-09-21T12:54:30Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04241worker2 # [ 54.709400] nix-grpc-daemon[779]: ts=2026-09-21T12:54:30Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14242worker1 # [ 55.269166] nix-grpc-daemon[773]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184243worker1 # [ 55.287375] nix-grpc-daemon[773]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184244client # [ 55.691145] nix[879]: copying 1 paths to 'lb:50051'...4245client # [ 55.692945] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4246worker1 # [ 55.302945] nix-grpc-daemon[773]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544184247worker1 # [ 55.319066] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4248worker1 # [ 55.349372] nix-grpc-daemon[773]: ts=2026-09-21T12:54:30Z level=info event=niks3_push_started pid=8664249worker1 # [ 55.611278] nix-grpc-daemon[866]: time=2026-09-21T12:54:30.801Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4250worker1 # [ 55.796942] nix-daemon[837]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4251worker1 # [ 55.818736] nix-daemon[837]: reaped child process 882, status = succeeded4252worker1 # [ 56.001048] nix-daemon[837]: accepted connection from pid 884, user nix-grpc-daemon (trusted)4253worker1 # [ 56.020264] nix-daemon[837]: reaped child process 887, status = succeeded4254lb # [ 56.407611] niks3-server[1026]: 2026/09/21 12:54:31 INFO Received uploads request method=POST path=/api/pending_closures4255worker1 # [ 56.038978] nix-grpc-daemon[866]: time=2026-09-21T12:54:31.231Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4256worker1 # [ 56.042788] nix-grpc-daemon[866]: time=2026-09-21T12:54:31.232Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4257lb # [ 56.438095] niks3-server[1026]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4258worker1 # [ 56.069671] nix-grpc-daemon[866]: time=2026-09-21T12:54:31.261Z level=INFO msg="Uploading 1 narinfos"4259lb # [ 56.445096] niks3-server[1026]: 2026/09/21 12:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4260lb # [ 56.447279] niks3-server[1026]: 2026/09/21 12:54:31 INFO Signed narinfos id=9 count=14261lb # [ 56.456095] niks3-server[1026]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4262lb # [ 56.461383] niks3-server[1026]: 2026/09/21 12:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4263lb # [ 56.465945] niks3-server[1026]: 2026/09/21 12:54:31 INFO Completed upload id=94264worker1 # [ 56.089543] nix-grpc-daemon[866]: time=2026-09-21T12:54:31.281Z level=INFO msg="Upload complete. (471ms)"4265lb # [ 56.467831] niks3-server[1026]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4266worker1 # [ 56.096105] nix-grpc-daemon[773]: ts=2026-09-21T12:54:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_in=4324267worker1 # [ 56.104236] nix-daemon[837]: reaped child process 864, status = succeeded4268client # [ 56.482832] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4269client # Running as unit: dedup2.service; invocation ID: 386adc0cde3648b4841f17f5e936b45f4270worker1 # [ 56.120622] nix-grpc-daemon[773]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184271worker1 # [ 56.128100] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4272client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4273client # [ 56.525994] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4274worker1 # [ 56.140850] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4275worker1 # [ 56.457208] nix-grpc-daemon[773]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544184276client # [ 56.861005] nix[895]: this derivation will be built:4277worker2 # [ 55.923852] nix-grpc-daemon[779]: ts=2026-09-21T12:54:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064278worker2 # [ 55.944219] nix-grpc-daemon[779]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4279client # [ 56.918624] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4280worker2 # [ 55.993289] nix-grpc-daemon[779]: ts=2026-09-21T12:54:32Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04281worker1 # [ 56.558059] nix-grpc-daemon[773]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184282client # [ 56.954876] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4283client: must succeed: sleep 24284client: (finished: must succeed: sleep 2, in 2.09 seconds)4285worker1 # [ 60.973075] nix-daemon[837]: reaped child process 893, status = succeeded4286client: waiting for success: ! systemctl is-active dedup1 dedup24287worker1 # [ 61.143229] nix-daemon[837]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4288worker1 # [ 61.160765] nix-daemon[837]: reaped child process 928, status = succeeded4289lb # [ 61.553730] niks3-server[1026]: 2026/09/21 12:54:36 INFO Received uploads request method=POST path=/api/pending_closures4290worker1 # [ 61.190542] nix-grpc-daemon[866]: time=2026-09-21T12:54:36.381Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4291worker1 # [ 61.192670] nix-grpc-daemon[866]: time=2026-09-21T12:54:36.381Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4292lb # [ 61.599037] niks3-server[1026]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4293lb # [ 61.611263] niks3-server[1026]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4294lb # [ 61.626191] niks3-server[1026]: 2026/09/21 12:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4295worker1 # [ 61.255936] nix-grpc-daemon[866]: time=2026-09-21T12:54:36.446Z level=INFO msg="Uploading 1 narinfos"4296lb # [ 61.632736] niks3-server[1026]: 2026/09/21 12:54:36 INFO Signed narinfos id=10 count=14297lb # [ 61.636097] niks3-server[1026]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4298lb # [ 61.650967] niks3-server[1026]: 2026/09/21 12:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4299lb # [ 61.658141] niks3-server[1026]: 2026/09/21 12:54:36 INFO Completed upload id=104300worker1 # [ 61.281703] nix-grpc-daemon[866]: time=2026-09-21T12:54:36.473Z level=INFO msg="Upload complete. (324ms)"4301lb # [ 61.661002] niks3-server[1026]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4302worker1 # [ 61.293064] nix-grpc-daemon[773]: ts=2026-09-21T12:54:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14303worker1 # [ 61.309777] nix-grpc-daemon[773]: ts=2026-09-21T12:54:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14304worker1 # [ 61.316500] nix-daemon[837]: reaped child process 889, status = succeeded4305client # [ 61.723940] systemd[1]: dedup2.service: Deactivated successfully.4306client # [ 61.727935] systemd[1]: dedup2.service: Consumed 223ms CPU time over 5.244s wall clock time, 33.2M memory peak, 6.5K incoming IP traffic, 5.8K outgoing IP traffic.4307client # [ 61.759737] systemd[1]: dedup1.service: Deactivated successfully.4308client # [ 61.761520] systemd[1]: dedup1.service: Consumed 172ms CPU time over 6.400s wall clock time, 39.3M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4309client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4310client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4311client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4312worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4313worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4314worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4315worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4316worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4317worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4318worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4319worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4320(finished: subtest: two clients wanting the same drv build it once, in 7.61 seconds)4321subtest: build hook: nix-daemon with builders = grpc://lb4322client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4323client # [ 63.125802] systemd[1]: Started Nix Daemon.4324client # [ 63.317961] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4325worker1 # [ 63.153288] nix-grpc-daemon[773]: ts=2026-09-21T12:54:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:544184326worker1 # [ 63.158071] nix-grpc-daemon[773]: ts=2026-09-21T12:54:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184327worker2 # [ 62.621799] nix-grpc-daemon[779]: ts=2026-09-21T12:54:38Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04328worker2 # [ 62.628468] nix-grpc-daemon[779]: ts=2026-09-21T12:54:38Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14329worker2 # [ 62.676101] nix-grpc-daemon[779]: ts=2026-09-21T12:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064330worker2 # [ 62.695396] nix-grpc-daemon[779]: ts=2026-09-21T12:54:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064331worker2 # [ 62.712693] nix-grpc-daemon[779]: ts=2026-09-21T12:54:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064332worker2 # [ 62.735937] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4333lb # [ 63.764228] envoy[915]: conn peer=[2001:db8:1::1]:33504 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1826 ms=24664334lb # [ 63.769178] envoy[915]: conn peer=[2001:db8:1::1]:33522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1487 ms=48584335lb # [ 63.774446] envoy[915]: conn peer=[2001:db8:1::1]:33520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1451 ms=61174336worker2 # [ 62.958087] nix-daemon[864]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4337worker2 # [ 62.978108] nix-daemon[864]: reaped child process 1103, status = succeeded4338worker2 # [ 63.155471] nix-daemon[864]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4339worker2 # [ 63.173397] nix-daemon[864]: reaped child process 1108, status = succeeded4340lb # [ 64.118733] niks3-server[1026]: 2026/09/21 12:54:39 INFO Received uploads request method=POST path=/api/pending_closures4341worker2 # [ 63.196941] nix-grpc-daemon[879]: time=2026-09-21T12:54:39.489Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4342worker2 # [ 63.198388] nix-grpc-daemon[879]: time=2026-09-21T12:54:39.489Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4343lb # [ 64.148327] niks3-server[1026]: 2026/09/21 12:54:39 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4344worker2 # [ 63.233959] nix-grpc-daemon[879]: time=2026-09-21T12:54:39.527Z level=INFO msg="Uploading 1 narinfos"4345lb # [ 64.167065] niks3-server[1026]: 2026/09/21 12:54:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4346lb # [ 64.176071] niks3-server[1026]: 2026/09/21 12:54:39 INFO Signed narinfos id=11 count=14347lb # [ 64.185761] niks3-server[1026]: 2026/09/21 12:54:39 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4348lb # [ 64.190262] niks3-server[1026]: 2026/09/21 12:54:39 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4349worker2 # [ 63.254761] nix-grpc-daemon[879]: time=2026-09-21T12:54:39.548Z level=INFO msg="Upload complete. (499ms)"4350lb # [ 64.195114] niks3-server[1026]: 2026/09/21 12:54:39 INFO Completed upload id=114351worker2 # [ 63.264418] nix-grpc-daemon[779]: ts=2026-09-21T12:54:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=5124352lb # [ 64.197343] niks3-server[1026]: 2026/09/21 12:54:39 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4353worker2 # [ 63.275451] nix-grpc-daemon[779]: ts=2026-09-21T12:54:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064354worker2 # [ 63.282477] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4355worker2 # [ 63.287441] nix-daemon[864]: reaped child process 1098, status = succeeded4356worker2 # [ 63.300155] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4357worker2 # [ 64.355920] nix-daemon[864]: reaped child process 1112, status = succeeded4358worker2 # [ 64.527563] nix-daemon[864]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4359worker2 # [ 64.547227] nix-daemon[864]: reaped child process 1131, status = succeeded4360lb # [ 65.490076] niks3-server[1026]: 2026/09/21 12:54:40 INFO Received uploads request method=POST path=/api/pending_closures4361worker2 # [ 64.568465] nix-grpc-daemon[879]: time=2026-09-21T12:54:40.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4362worker2 # [ 64.570040] nix-grpc-daemon[879]: time=2026-09-21T12:54:40.860Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4363lb # [ 65.523757] niks3-server[1026]: 2026/09/21 12:54:40 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4364lb # [ 65.533207] niks3-server[1026]: 2026/09/21 12:54:40 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4365lb # [ 65.539360] niks3-server[1026]: 2026/09/21 12:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4366worker2 # [ 64.613295] nix-grpc-daemon[879]: time=2026-09-21T12:54:40.906Z level=INFO msg="Uploading 1 narinfos"4367lb # [ 65.547176] niks3-server[1026]: 2026/09/21 12:54:40 INFO Signed narinfos id=12 count=14368lb # [ 65.551375] niks3-server[1026]: 2026/09/21 12:54:40 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4369lb # [ 65.557459] niks3-server[1026]: 2026/09/21 12:54:40 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4370lb # [ 65.562599] niks3-server[1026]: 2026/09/21 12:54:40 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4371lb # [ 65.566772] niks3-server[1026]: 2026/09/21 12:54:40 INFO Completed upload id=124372worker2 # [ 64.637268] nix-grpc-daemon[879]: time=2026-09-21T12:54:40.930Z level=INFO msg="Upload complete. (322ms)"4373worker2 # [ 64.649128] nix-grpc-daemon[779]: ts=2026-09-21T12:54:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14374worker2 # [ 64.665214] nix-grpc-daemon[779]: ts=2026-09-21T12:54:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064375worker2 # [ 64.669433] nix-daemon[864]: reaped child process 1110, status = succeeded4376worker1 # [ 65.236158] nix-grpc-daemon[773]: ts=2026-09-21T12:54:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:544184377worker1 # [ 65.246768] nix-grpc-daemon[773]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4378worker1 # [ 65.269177] nix-grpc-daemon[773]: ts=2026-09-21T12:54:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_out=1284379worker2 # [ 65.021789] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418064380worker1 # [ 65.580156] nix-grpc-daemon[773]: ts=2026-09-21T12:54:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184381worker1 # [ 65.593256] nix-grpc-daemon[866]: time=2026-09-21T12:54:40.784Z level=INFO msg="All 1 paths already cached"4382worker2 # [ 65.056164] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04383worker2 # [ 65.062403] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14384worker2 # [ 65.072182] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064385worker2 # [ 65.083654] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064386worker2 # [ 65.098044] nix-grpc-daemon[879]: time=2026-09-21T12:54:41.391Z level=INFO msg="All 1 paths already cached"4387worker2 # [ 65.106935] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064388worker2 # [ 65.121887] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4389worker2 # [ 65.336185] nix-daemon[864]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4390worker2 # [ 65.356000] nix-daemon[864]: reaped child process 1140, status = succeeded4391worker2 # [ 65.533875] nix-daemon[864]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4392worker2 # [ 65.551601] nix-daemon[864]: reaped child process 1145, status = succeeded4393lb # [ 66.494231] niks3-server[1026]: 2026/09/21 12:54:41 INFO Received uploads request method=POST path=/api/pending_closures4394worker2 # [ 65.572419] nix-grpc-daemon[879]: time=2026-09-21T12:54:41.865Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4395worker2 # [ 65.573947] nix-grpc-daemon[879]: time=2026-09-21T12:54:41.865Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4396lb # [ 66.520457] niks3-server[1026]: 2026/09/21 12:54:41 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4397lb # [ 66.529521] niks3-server[1026]: 2026/09/21 12:54:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4398worker2 # [ 65.604280] nix-grpc-daemon[879]: time=2026-09-21T12:54:41.897Z level=INFO msg="Uploading 1 narinfos"4399lb # [ 66.537420] niks3-server[1026]: 2026/09/21 12:54:41 INFO Signed narinfos id=13 count=14400lb # [ 66.539238] niks3-server[1026]: 2026/09/21 12:54:41 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4401lb # [ 66.547406] niks3-server[1026]: 2026/09/21 12:54:41 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4402lb # [ 66.552756] niks3-server[1026]: 2026/09/21 12:54:41 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4403lb # [ 66.557042] niks3-server[1026]: 2026/09/21 12:54:41 INFO Completed upload id=134404worker2 # [ 65.627422] nix-grpc-daemon[879]: time=2026-09-21T12:54:41.920Z level=INFO msg="Upload complete. (487ms)"4405worker2 # [ 65.635952] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=6244406worker2 # [ 65.639577] nix-grpc-daemon[779]: ts=2026-09-21T12:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064407worker2 # [ 65.652150] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4408worker2 # [ 65.655948] nix-daemon[864]: reaped child process 1135, status = succeeded4409worker2 # [ 65.666049] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4410worker2 # [ 66.515808] nix-daemon[864]: reaped child process 1149, status = succeeded4411worker2 # [ 66.692692] nix-daemon[864]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4412worker2 # [ 66.712718] nix-daemon[864]: reaped child process 1168, status = succeeded4413lb # [ 67.653773] niks3-server[1026]: 2026/09/21 12:54:42 INFO Received uploads request method=POST path=/api/pending_closures4414worker2 # [ 66.731945] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.022Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4415worker2 # [ 66.733631] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.022Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4416lb # [ 67.672159] niks3-server[1026]: 2026/09/21 12:54:42 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4417lb # [ 67.681250] niks3-server[1026]: 2026/09/21 12:54:42 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4418lb # [ 67.687142] niks3-server[1026]: 2026/09/21 12:54:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4419worker2 # [ 66.760441] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.053Z level=INFO msg="Uploading 1 narinfos"4420lb # [ 67.691436] niks3-server[1026]: 2026/09/21 12:54:42 INFO Signed narinfos id=14 count=14421lb # [ 67.695205] niks3-server[1026]: 2026/09/21 12:54:42 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4422lb # [ 67.702085] niks3-server[1026]: 2026/09/21 12:54:42 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4423lb # [ 67.705847] niks3-server[1026]: 2026/09/21 12:54:42 INFO Completed upload id=144424worker2 # [ 66.776209] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.069Z level=INFO msg="Upload complete. (299ms)"4425lb # [ 67.707547] niks3-server[1026]: 2026/09/21 12:54:42 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4426worker2 # [ 66.784646] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14427worker2 # [ 66.794422] nix-daemon[864]: reaped child process 1147, status = succeeded4428worker2 # [ 66.829880] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064429worker1 # [ 67.411548] nix-grpc-daemon[773]: ts=2026-09-21T12:54:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:544184430worker1 # [ 67.426485] nix-grpc-daemon[773]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4431worker1 # [ 67.442346] nix-grpc-daemon[773]: ts=2026-09-21T12:54:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_out=1764432worker2 # [ 67.210752] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418064433worker2 # [ 67.218914] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064434worker2 # [ 67.226453] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.518Z level=INFO msg="All 2 paths already cached"4435worker2 # [ 67.240152] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04436worker2 # [ 67.248608] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14437worker2 # [ 67.257852] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064438worker2 # [ 67.270658] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064439worker2 # [ 67.284364] nix-grpc-daemon[879]: time=2026-09-21T12:54:43.577Z level=INFO msg="All 2 paths already cached"4440worker2 # [ 67.294251] nix-grpc-daemon[779]: ts=2026-09-21T12:54:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064441worker2 # [ 67.308735] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4442worker2 # [ 67.516665] nix-daemon[864]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)4443worker2 # [ 67.537546] nix-daemon[864]: reaped child process 1177, status = succeeded4444worker2 # [ 67.717650] nix-daemon[864]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4445worker2 # [ 67.735564] nix-daemon[864]: reaped child process 1182, status = succeeded4446lb # [ 68.676455] niks3-server[1026]: 2026/09/21 12:54:43 INFO Received uploads request method=POST path=/api/pending_closures4447worker2 # [ 67.754266] nix-grpc-daemon[879]: time=2026-09-21T12:54:44.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4448worker2 # [ 67.756215] nix-grpc-daemon[879]: time=2026-09-21T12:54:44.049Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4449lb # [ 68.699961] niks3-server[1026]: 2026/09/21 12:54:43 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4450lb # [ 68.711704] niks3-server[1026]: 2026/09/21 12:54:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4451worker2 # [ 67.785977] nix-grpc-daemon[879]: time=2026-09-21T12:54:44.079Z level=INFO msg="Uploading 1 narinfos"4452lb # [ 68.720453] niks3-server[1026]: 2026/09/21 12:54:44 INFO Signed narinfos id=15 count=14453lb # [ 68.722655] niks3-server[1026]: 2026/09/21 12:54:44 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4454lb # [ 68.731782] niks3-server[1026]: 2026/09/21 12:54:44 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4455lb # [ 68.738477] niks3-server[1026]: 2026/09/21 12:54:44 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4456lb # [ 68.742227] niks3-server[1026]: 2026/09/21 12:54:44 INFO Completed upload id=154457worker2 # [ 67.812294] nix-grpc-daemon[879]: time=2026-09-21T12:54:44.105Z level=INFO msg="Upload complete. (486ms)"4458worker2 # [ 67.820915] nix-grpc-daemon[779]: ts=2026-09-21T12:54:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_in=7524459worker2 # [ 67.831133] nix-grpc-daemon[779]: ts=2026-09-21T12:54:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064460worker2 # [ 67.836551] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4461worker2 # [ 67.841112] nix-daemon[864]: reaped child process 1172, status = succeeded4462worker2 # [ 67.853109] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4463worker2 # [ 69.279138] nix-daemon[864]: reaped child process 1186, status = succeeded4464worker2 # [ 69.438135] nix-daemon[864]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)4465worker2 # [ 69.456306] nix-daemon[864]: reaped child process 1205, status = succeeded4466lb # [ 70.397464] niks3-server[1026]: 2026/09/21 12:54:45 INFO Received uploads request method=POST path=/api/pending_closures4467worker2 # [ 69.479931] nix-grpc-daemon[879]: time=2026-09-21T12:54:45.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4468worker2 # [ 69.482575] nix-grpc-daemon[879]: time=2026-09-21T12:54:45.771Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4469lb # [ 70.431925] niks3-server[1026]: 2026/09/21 12:54:45 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4470lb # [ 70.450032] niks3-server[1026]: 2026/09/21 12:54:45 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4471lb # [ 70.464463] niks3-server[1026]: 2026/09/21 12:54:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4472worker2 # [ 69.542519] nix-grpc-daemon[879]: time=2026-09-21T12:54:45.835Z level=INFO msg="Uploading 1 narinfos"4473lb # [ 70.477030] niks3-server[1026]: 2026/09/21 12:54:45 INFO Signed narinfos id=16 count=14474lb # [ 70.483227] niks3-server[1026]: 2026/09/21 12:54:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4475lb # [ 70.491899] niks3-server[1026]: 2026/09/21 12:54:45 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4476lb # [ 70.500883] niks3-server[1026]: 2026/09/21 12:54:45 INFO Completed upload id=164477worker2 # [ 69.570523] nix-grpc-daemon[879]: time=2026-09-21T12:54:45.864Z level=INFO msg="Upload complete. (308ms)"4478lb # [ 70.504627] niks3-server[1026]: 2026/09/21 12:54:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4479worker2 # [ 69.583295] nix-grpc-daemon[779]: ts=2026-09-21T12:54:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14480worker2 # [ 69.603804] nix-grpc-daemon[779]: ts=2026-09-21T12:54:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064481worker2 # [ 69.608660] nix-daemon[864]: reaped child process 1184, status = succeeded4482worker2 # [ 69.619912] nix-grpc-daemon[779]: ts=2026-09-21T12:54:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418064483worker2 # [ 69.626911] nix-grpc-daemon[779]: ts=2026-09-21T12:54:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_out=2324484client # [ 70.683404] nix-daemon[937]: reaped child process 939, status = succeeded4485client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.81 seconds)4486client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4487client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4488client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4489client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4490client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4491client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4492(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.93 seconds)4493subtest: an input only one worker has still reaches the builder4494client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4495client # this derivation will be built:4496client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4497client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4498client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4499client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4500client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4501worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4502worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)4503worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4504worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4505client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24506client # [ 71.961264] nix-daemon[937]: accepted connection from pid 1020, user root (trusted)4507client # this derivation will be built:4508client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4509worker2 # [ 71.230018] nix-grpc-daemon[779]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418064510client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4511worker2 # [ 71.233829] nix-grpc-daemon[779]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064512client # copying 1 paths...4513client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4514worker1 # [ 71.803147] nix-grpc-daemon[773]: ts=2026-09-21T12:54:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544184515worker1 # [ 71.818968] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4516worker1 # [ 72.037456] nix-daemon[837]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4517worker1 # [ 72.055850] nix-daemon[837]: reaped child process 951, status = succeeded4518lb # [ 72.445015] niks3-server[1026]: 2026/09/21 12:54:47 INFO Received uploads request method=POST path=/api/pending_closures4519worker1 # [ 72.078475] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.270Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4520worker1 # [ 72.080081] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.270Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4521lb # [ 72.476443] niks3-server[1026]: 2026/09/21 12:54:47 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4522worker1 # [ 72.113667] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.305Z level=INFO msg="Uploading 1 narinfos"4523lb # [ 72.491161] niks3-server[1026]: 2026/09/21 12:54:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4524lb # [ 72.503527] niks3-server[1026]: 2026/09/21 12:54:47 INFO Signed narinfos id=17 count=14525lb # [ 72.508390] niks3-server[1026]: 2026/09/21 12:54:47 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4526lb # [ 72.517428] niks3-server[1026]: 2026/09/21 12:54:47 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4527lb # [ 72.522585] niks3-server[1026]: 2026/09/21 12:54:47 INFO Completed upload id=174528worker1 # [ 72.146694] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.338Z level=INFO msg="Upload complete. (312ms)"4529lb # [ 72.525355] niks3-server[1026]: 2026/09/21 12:54:47 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4530worker1 # [ 72.156420] nix-grpc-daemon[773]: ts=2026-09-21T12:54:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_in=1204531worker1 # [ 72.171701] nix-daemon[837]: reaped child process 946, status = succeeded4532worker2 # [ 71.618773] nix-grpc-daemon[779]: ts=2026-09-21T12:54:47Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04533worker2 # [ 71.625358] nix-grpc-daemon[779]: ts=2026-09-21T12:54:47Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14534worker1 # [ 72.221127] nix-grpc-daemon[773]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184535worker1 # [ 72.238316] nix-grpc-daemon[773]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184536client # copying 2 paths to 'lb:50051'...4537worker1 # [ 72.253434] nix-grpc-daemon[773]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544184538client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4539client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4540worker1 # [ 72.265144] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4541worker1 # [ 72.447403] nix-daemon[837]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4542worker1 # [ 72.466462] nix-daemon[837]: reaped child process 958, status = succeeded4543worker1 # [ 72.648728] nix-daemon[837]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4544worker1 # [ 72.666406] nix-daemon[837]: reaped child process 963, status = succeeded4545lb # [ 73.056226] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received uploads request method=POST path=/api/pending_closures4546lb # [ 73.065883] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received uploads request method=POST path=/api/pending_closures4547worker1 # [ 72.699571] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.891Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4548worker1 # [ 72.701301] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.891Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4549worker1 # [ 72.703073] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.891Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4550lb # [ 73.099372] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4551lb # [ 73.118064] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4552lb # [ 73.134062] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4553lb # [ 73.141544] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4554worker1 # [ 72.774635] nix-grpc-daemon[866]: time=2026-09-21T12:54:47.966Z level=INFO msg="Uploading 3 narinfos"4555lb # [ 73.153178] niks3-server[1026]: 2026/09/21 12:54:48 INFO Signed narinfos id=18 count=14556lb # [ 73.155872] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4557lb # [ 73.165853] niks3-server[1026]: 2026/09/21 12:54:48 INFO Signed narinfos id=19 count=24558lb # [ 73.170831] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4559lb # [ 73.185348] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4560lb # [ 73.191855] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4561lb # [ 73.197495] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4562lb # [ 73.201562] niks3-server[1026]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4563lb # [ 73.204857] niks3-server[1026]: 2026/09/21 12:54:48 INFO Completed upload id=184564lb # [ 73.207762] niks3-server[1026]: 2026/09/21 12:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4565lb # [ 73.211270] niks3-server[1026]: 2026/09/21 12:54:48 INFO Completed upload id=194566worker1 # [ 72.835326] nix-grpc-daemon[866]: time=2026-09-21T12:54:48.027Z level=INFO msg="Upload complete. (559ms)"4567worker1 # [ 72.842795] nix-grpc-daemon[773]: ts=2026-09-21T12:54:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=2 nar_bytes_in=9844568worker1 # [ 72.852548] nix-grpc-daemon[773]: ts=2026-09-21T12:54:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184569worker1 # [ 72.863824] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4570worker1 # [ 72.870194] nix-daemon[837]: reaped child process 953, status = succeeded4571client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4572worker1 # [ 72.881641] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4573client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4574client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4575lb # [ 73.763446] envoy[915]: conn peer=[2001:db8:1::1]:53810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3431 tx=1758 ms=21764576lb # [ 73.768779] envoy[915]: conn peer=[2001:db8:1::1]:53820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1044577lb # [ 73.773490] envoy[915]: conn peer=[2001:db8:1::1]:53834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=1930 ms=19244578lb # [ 73.778587] envoy[915]: conn peer=[2001:db8:1::1]:53840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=964579lb # [ 73.783303] envoy[915]: conn peer=[2001:db8:1::1]:53856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4356 tx=2185 ms=24754580lb # [ 73.789754] envoy[915]: conn peer=[2001:db8:1::1]:53860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=684581worker1 # [ 73.733327] nix-daemon[837]: reaped child process 967, status = succeeded4582worker1 # [ 73.914375] nix-daemon[837]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4583worker1 # [ 73.933029] nix-daemon[837]: reaped child process 986, status = succeeded4584lb # [ 74.323203] niks3-server[1026]: 2026/09/21 12:54:49 INFO Received uploads request method=POST path=/api/pending_closures4585worker1 # [ 73.956463] nix-grpc-daemon[866]: time=2026-09-21T12:54:49.148Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4586lb # [ 74.356634] niks3-server[1026]: 2026/09/21 12:54:49 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4587worker1 # [ 73.994793] nix-grpc-daemon[866]: time=2026-09-21T12:54:49.186Z level=INFO msg="Uploading 1 narinfos"4588lb # [ 74.370535] niks3-server[1026]: 2026/09/21 12:54:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4589lb # [ 74.378287] niks3-server[1026]: 2026/09/21 12:54:49 INFO Signed narinfos id=20 count=14590lb # [ 74.384611] niks3-server[1026]: 2026/09/21 12:54:49 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4591lb # [ 74.395358] niks3-server[1026]: 2026/09/21 12:54:49 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4592lb # [ 74.404757] niks3-server[1026]: 2026/09/21 12:54:49 INFO Completed upload id=204593worker1 # [ 74.028868] nix-grpc-daemon[866]: time=2026-09-21T12:54:49.221Z level=INFO msg="Upload complete. (334ms)"4594lb # [ 74.408320] niks3-server[1026]: 2026/09/21 12:54:49 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4595client # copying 1 paths...4596worker2 # [ 73.493534] nix-grpc-daemon[779]: ts=2026-09-21T12:54:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064597worker1 # [ 74.040866] nix-grpc-daemon[773]: ts=2026-09-21T12:54:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14598worker1 # [ 74.054828] nix-daemon[837]: reaped child process 965, status = succeeded4599worker2 # [ 73.507439] nix-grpc-daemon[779]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4600client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4601worker1 # [ 74.108290] nix-grpc-daemon[773]: ts=2026-09-21T12:54:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:544184602worker1 # [ 74.112224] nix-grpc-daemon[773]: ts=2026-09-21T12:54:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_out=1204603client # [ 74.592669] nix-daemon[937]: reaped child process 1023, status = succeeded4604client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.87 seconds)4605client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24606client # [ 74.811881] nix-daemon[937]: accepted connection from pid 1042, user root (trusted)4607client # this derivation will be built:4608client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4609worker2 # [ 74.090418] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418064610client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4611worker2 # [ 74.098830] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064612worker2 # [ 74.119609] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04613worker2 # [ 74.129275] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14614worker2 # [ 74.138074] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064615worker2 # [ 74.147344] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064616client # copying 1 paths to 'lb:50051'...4617client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4618worker2 # [ 74.165616] nix-grpc-daemon[779]: ts=2026-09-21T12:54:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418064619worker2 # [ 74.181106] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4620worker2 # [ 74.201935] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)4621worker2 # [ 74.290873] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 2/5)4622worker2 # [ 74.476702] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 3/5)4623worker2 # [ 74.654422] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 4/5)4624worker2 # [ 74.708113] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4625worker2 # [ 74.711321] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4626worker2 # [ 74.715202] nix-grpc-daemon[779]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4627worker2 # [ 74.914296] nix-daemon[864]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)4628worker2 # [ 74.933589] nix-daemon[864]: reaped child process 1225, status = succeeded4629worker2 # [ 75.119886] nix-daemon[864]: accepted connection from pid 1227, user nix-grpc-daemon (trusted)4630worker2 # [ 75.143305] nix-daemon[864]: reaped child process 1230, status = succeeded4631lb # [ 76.088620] niks3-server[1026]: 2026/09/21 12:54:51 INFO Received uploads request method=POST path=/api/pending_closures4632worker2 # [ 75.171102] nix-grpc-daemon[879]: time=2026-09-21T12:54:51.463Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4633worker2 # [ 75.172908] nix-grpc-daemon[879]: time=2026-09-21T12:54:51.463Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4634lb # [ 76.119424] niks3-server[1026]: 2026/09/21 12:54:51 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4635lb # [ 76.136900] niks3-server[1026]: 2026/09/21 12:54:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4636worker2 # [ 75.212176] nix-grpc-daemon[879]: time=2026-09-21T12:54:51.505Z level=INFO msg="Uploading 1 narinfos"4637lb # [ 76.147607] niks3-server[1026]: 2026/09/21 12:54:51 INFO Signed narinfos id=21 count=14638lb # [ 76.153815] niks3-server[1026]: 2026/09/21 12:54:51 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4639lb # [ 76.164512] niks3-server[1026]: 2026/09/21 12:54:51 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4640worker2 # [ 75.240235] nix-grpc-daemon[879]: time=2026-09-21T12:54:51.533Z level=INFO msg="Upload complete. (509ms)"4641lb # [ 76.171091] niks3-server[1026]: 2026/09/21 12:54:51 INFO Completed upload id=214642lb # [ 76.175276] niks3-server[1026]: 2026/09/21 12:54:51 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4643worker2 # [ 75.248148] nix-grpc-daemon[779]: ts=2026-09-21T12:54:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 paths=1 nar_bytes_in=5684644worker2 # [ 75.259932] nix-grpc-daemon[779]: ts=2026-09-21T12:54:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418064645worker2 # [ 75.272814] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4646worker2 # [ 75.278548] nix-daemon[864]: reaped child process 1215, status = succeeded4647worker2 # [ 75.286251] nix-grpc-daemon[779]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4648client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4649worker2 # [ 75.302762] nix-daemon[864]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4650client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4651client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4652worker2 # [ 76.337863] nix-daemon[864]: reaped child process 1235, status = succeeded4653worker2 # [ 76.508758] nix-daemon[864]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)4654worker2 # [ 76.530100] nix-daemon[864]: reaped child process 1254, status = succeeded4655lb # [ 77.474524] niks3-server[1026]: 2026/09/21 12:54:52 INFO Received uploads request method=POST path=/api/pending_closures4656worker2 # [ 76.556874] nix-grpc-daemon[879]: time=2026-09-21T12:54:52.849Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4657lb # [ 77.505497] niks3-server[1026]: 2026/09/21 12:54:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4658worker2 # [ 76.584500] nix-grpc-daemon[879]: time=2026-09-21T12:54:52.877Z level=INFO msg="Uploading 1 narinfos"4659lb # [ 77.519523] niks3-server[1026]: 2026/09/21 12:54:52 INFO Signed narinfos id=22 count=14660lb # [ 77.527752] niks3-server[1026]: 2026/09/21 12:54:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4661worker2 # [ 76.609612] nix-grpc-daemon[879]: time=2026-09-21T12:54:52.903Z level=INFO msg="Upload complete. (292ms)"4662lb # [ 77.535224] niks3-server[1026]: 2026/09/21 12:54:52 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4663lb # [ 77.545882] niks3-server[1026]: 2026/09/21 12:54:52 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4664lb # [ 77.549641] niks3-server[1026]: 2026/09/21 12:54:52 INFO Completed upload id=224665lb # [ 77.551505] niks3-server[1026]: 2026/09/21 12:54:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4666worker2 # [ 76.620900] nix-grpc-daemon[779]: ts=2026-09-21T12:54:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14667worker2 # [ 76.633437] nix-daemon[864]: reaped child process 1232, status = succeeded4668client # copying 1 paths...4669worker2 # [ 76.662631] nix-grpc-daemon[779]: ts=2026-09-21T12:54:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064670client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4671worker2 # [ 76.698995] nix-grpc-daemon[779]: ts=2026-09-21T12:54:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418064672worker2 # [ 76.704526] nix-grpc-daemon[779]: ts=2026-09-21T12:54:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_out=1204673client # [ 77.781062] nix-daemon[937]: reaped child process 1045, status = succeeded4674client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.19 seconds)4675client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24676client # [ 77.995446] nix-daemon[937]: accepted connection from pid 1064, user root (trusted)4677client # this derivation will be built:4678client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4679worker1 # [ 77.816769] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:544184680client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4681worker2 # [ 77.266364] nix-grpc-daemon[779]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418064682worker2 # [ 77.277192] nix-grpc-daemon[879]: time=2026-09-21T12:54:53.570Z level=INFO msg="All 1 paths already cached"4683worker2 # [ 77.298114] nix-grpc-daemon[779]: ts=2026-09-21T12:54:53Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04684worker2 # [ 77.305068] nix-grpc-daemon[779]: ts=2026-09-21T12:54:53Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14685worker1 # [ 77.896922] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184686worker1 # [ 77.915469] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544184687client # copying 1 paths to 'lb:50051'...4688worker1 # [ 77.933178] nix-grpc-daemon[866]: time=2026-09-21T12:54:53.123Z level=INFO msg="All 1 paths already cached"4689client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4690worker1 # [ 77.939949] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544184691worker1 # [ 77.955748] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4692worker1 # [ 78.166318] nix-daemon[837]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4693worker1 # [ 78.186125] nix-daemon[837]: reaped child process 993, status = succeeded4694worker1 # [ 78.355685] nix-daemon[837]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4695worker1 # [ 78.376967] nix-daemon[837]: reaped child process 998, status = succeeded4696lb # [ 78.761735] niks3-server[1026]: 2026/09/21 12:54:54 INFO Received uploads request method=POST path=/api/pending_closures4697worker1 # [ 78.395715] nix-grpc-daemon[866]: time=2026-09-21T12:54:53.587Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4698worker1 # [ 78.397395] nix-grpc-daemon[866]: time=2026-09-21T12:54:53.587Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4699lb # [ 78.791859] niks3-server[1026]: 2026/09/21 12:54:54 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4700lb # [ 78.808889] niks3-server[1026]: 2026/09/21 12:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4701worker1 # [ 78.438681] nix-grpc-daemon[866]: time=2026-09-21T12:54:53.630Z level=INFO msg="Uploading 1 narinfos"4702lb # [ 78.822045] niks3-server[1026]: 2026/09/21 12:54:54 INFO Signed narinfos id=23 count=14703lb # [ 78.828866] niks3-server[1026]: 2026/09/21 12:54:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4704lb # [ 78.839662] niks3-server[1026]: 2026/09/21 12:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4705lb # [ 78.845017] niks3-server[1026]: 2026/09/21 12:54:54 INFO Completed upload id=234706worker1 # [ 78.469136] nix-grpc-daemon[866]: time=2026-09-21T12:54:53.661Z level=INFO msg="Upload complete. (495ms)"4707lb # [ 78.847945] niks3-server[1026]: 2026/09/21 12:54:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4708worker1 # [ 78.476355] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=0 paths=1 nar_bytes_in=5684709worker1 # [ 78.491877] nix-grpc-daemon[773]: ts=2026-09-21T12:54:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544184710worker1 # [ 78.499958] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4711worker1 # [ 78.505687] nix-daemon[837]: reaped child process 988, status = succeeded4712client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4713worker1 # [ 78.520226] nix-daemon[837]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4714client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4715client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4716worker1 # [ 79.945715] nix-daemon[837]: reaped child process 1002, status = succeeded4717worker1 # [ 80.109304] nix-daemon[837]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)4718worker1 # [ 80.129198] nix-daemon[837]: reaped child process 1021, status = succeeded4719lb # [ 80.522153] niks3-server[1026]: 2026/09/21 12:54:55 INFO Received uploads request method=POST path=/api/pending_closures4720worker1 # [ 80.157432] nix-grpc-daemon[866]: time=2026-09-21T12:54:55.347Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4721lb # [ 80.554925] niks3-server[1026]: 2026/09/21 12:54:55 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4722worker1 # [ 80.189682] nix-grpc-daemon[866]: time=2026-09-21T12:54:55.381Z level=INFO msg="Uploading 1 narinfos"4723lb # [ 80.566746] niks3-server[1026]: 2026/09/21 12:54:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4724lb # [ 80.578073] niks3-server[1026]: 2026/09/21 12:54:55 INFO Signed narinfos id=24 count=14725worker1 # [ 80.211832] nix-grpc-daemon[866]: time=2026-09-21T12:54:55.403Z level=INFO msg="Upload complete. (279ms)"4726lb # [ 80.583853] niks3-server[1026]: 2026/09/21 12:54:55 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4727lb # [ 80.595715] niks3-server[1026]: 2026/09/21 12:54:55 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4728client # copying 1 paths...4729worker1 # [ 80.220842] nix-grpc-daemon[773]: ts=2026-09-21T12:54:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54418 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14730lb # [ 80.600396] niks3-server[1026]: 2026/09/21 12:54:55 INFO Completed upload id=244731lb # [ 80.603398] niks3-server[1026]: 2026/09/21 12:54:55 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4732worker1 # [ 80.232701] nix-daemon[837]: reaped child process 1000, status = succeeded4733worker2 # [ 79.675761] nix-grpc-daemon[779]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418064734worker2 # [ 79.689536] nix-grpc-daemon[779]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4735client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4736worker2 # [ 79.729879] nix-grpc-daemon[779]: ts=2026-09-21T12:54:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418064737worker2 # [ 79.735315] nix-grpc-daemon[779]: ts=2026-09-21T12:54:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41806 duration_s=0 paths=1 nar_bytes_out=1204738client # [ 80.807261] nix-daemon[937]: reaped child process 1067, status = succeeded4739client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.03 seconds)4740(finished: subtest: an input only one worker has still reaches the builder, in 10.00 seconds)4741subtest: upload whose reference the worker lacks is completed from the cache4742client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4743client # this derivation will be built:4744client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4745client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4746client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4747client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4748client # this derivation will be built:4749client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4750client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4751client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4752client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4753client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4754worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4755worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4756worker2 # [ 81.281168] systemd[1]: Stopping Nix worker-protocol over gRPC...4757worker2 # [ 81.344645] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=signal_exit4758worker2 # [ 81.573008] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=scheduler_yield4759worker1 # [ 82.130728] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_restarting addr=lb:500514760worker2 # [ 81.578773] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397004761worker2 # [ 81.583587] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=worker_gone addr=192.168.1.3:500514762lb # [ 82.507149] niks3-server[1026]: 2026/09/21 12:54:57 INFO lead: released remote=192.168.1.4:404924763worker2 # [ 81.589588] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=scheduler_restarting addr=lb:500514764worker2 # [ 81.596395] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397004765worker2 # [ 81.600913] nix-grpc-daemon[779]: ts=2026-09-21T12:54:57Z level=info event=worker_gone addr=192.168.1.4:500514766worker1 # [ 82.331324] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_connected addr=lb:500514767worker1 # [ 82.336870] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4768worker2 # [ 81.790925] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514769worker2 # [ 81.796493] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4770worker1 # [ 82.537709] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_connected addr=lb:500514771worker1 # [ 82.543193] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4772worker2 # [ 81.997713] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514773worker2 # [ 82.003382] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4774worker1 # [ 82.743779] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_connected addr=lb:500514775worker1 # [ 82.746822] nix-grpc-daemon[773]: ts=2026-09-21T12:54:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4776worker2 # [ 82.204674] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514777worker2 # [ 82.209825] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4778worker1 # [ 82.947667] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514779worker1 # [ 82.952084] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4780worker2 # [ 82.412315] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514781worker2 # [ 82.417384] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4782worker2 # [ 82.576149] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=scheduler_restarting4783worker1 # [ 83.155107] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514784worker1 # [ 83.159692] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4785worker2 # [ 82.612268] nix-daemon[864]: reaped child process 866, status = succeeded4786worker2 # [ 82.643748] nix-grpc-daemon[779]: ts=2026-09-21T12:54:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4787worker1 # [ 83.355890] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514788worker1 # [ 83.361653] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4789lb # [ 83.761214] envoy[915]: conn peer=[2001:db8:1::1]:46332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=254790lb # [ 83.764152] envoy[915]: conn peer=[2001:db8:1::1]:46328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4271 tx=1962 ms=23464791lb # [ 83.766937] envoy[915]: conn peer=[2001:db8:1::1]:46334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=1902 ms=26694792lb # [ 83.769671] envoy[915]: conn peer=[2001:db8:1::1]:46344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=664793lb # [ 83.772201] envoy[915]: conn peer=[2001:db8:1::1]:46360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3783 tx=2029 ms=25274794lb # [ 83.774698] envoy[915]: conn peer=[2001:db8:1::1]:46362 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=624795lb # [ 83.777058] envoy[915]: conn peer=[2001:db8:1::4]:40436 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=46055 tx=2806 ms=592384796worker1 # [ 83.562584] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514797worker1 # [ 83.568321] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4798worker1 # [ 83.769170] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_connected addr=lb:500514799worker1 # [ 83.774724] nix-grpc-daemon[773]: ts=2026-09-21T12:54:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4800worker1 # [ 83.914188] nix-grpc-daemon[773]: ts=2026-09-21T12:54:59Z level=info event=scheduler_take_over4801lb # [ 84.291369] niks3-server[1026]: 2026/09/21 12:54:59 INFO lead: acquired remote=192.168.1.3:606444802worker1 # [ 83.975228] nix-grpc-daemon[773]: ts=2026-09-21T12:54:59Z level=info event=scheduler_connected addr=lb:500514803worker1 # [ 83.990067] nix-grpc-daemon[773]: ts=2026-09-21T12:54:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:411364804worker1 # [ 83.994147] nix-grpc-daemon[773]: ts=2026-09-21T12:54:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04805worker2 # [ 83.454735] nix-grpc-daemon[779]: E0000 00:00:1789995299.746936 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4806worker2 # [ 83.495016] nix-grpc-daemon[779]: E0000 00:00:1789995299.788377 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4807worker2 # [ 84.021085] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4808worker2 # [ 84.030172] systemd[1]: Stopped Nix worker-protocol over gRPC.4809worker2 # [ 84.035595] systemd[1]: nix-grpc-daemon.service: Consumed 3.788s CPU time over 1min 1.020s wall clock time, 65.2M memory peak, 93K incoming IP traffic, 231.5K outgoing IP traffic.4810worker2 # [ 84.049605] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4811worker2 # [ 84.052687] systemd[1]: Closed Nix worker-protocol over gRPC.4812worker2 # [ 84.055244] systemd[1]: nix-grpc-daemon.socket: Consumed 787us CPU time over 1min 11.018s wall clock time, 256K memory peak, 94.4K incoming IP traffic, 76.3K outgoing IP traffic.4813lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4814lb: must succeed: curl -sf localhost:9901/clusters4815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4816lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4817client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24818worker1 # [ 84.961043] nix-grpc-daemon[773]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544184819worker1 # [ 84.971536] nix-grpc-daemon[773]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:544184820client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4821client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4822worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4823worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4824worker2 # [ 84.562404] systemd[1]: Listening on Nix worker-protocol over gRPC.4825worker1 # [ 85.202182] systemd[1]: Stopping Nix worker-protocol over gRPC...4826worker2 # [ 85.010092] systemd[1]: Starting scheduler-token.service...4827worker2 # [ 85.116852] systemd[1]: scheduler-token.service: Deactivated successfully.4828worker2 # [ 85.124222] systemd[1]: Finished scheduler-token.service.4829worker2 # [ 85.125956] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 110ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4830worker2 # [ 85.133958] systemd[1]: Starting Nix worker-protocol over gRPC...4831worker1 # [ 85.815764] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=signal_exit4832worker2 # [ 85.325786] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4833worker2 # [ 85.328325] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4834worker2 # [ 85.332389] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14835worker2 # [ 85.339715] nix-grpc-daemon[1281]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4836worker2 # [ 85.350793] nix-grpc-daemon[1281]: E0000 00:00:1789995301.632642 1281 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4837worker2 # [ 85.355957] nix-grpc-daemon[1281]: E0000 00:00:1789995301.636471 1281 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4838worker2 # [ 85.362184] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4839worker2 # [ 85.364367] systemd[1]: Started Nix worker-protocol over gRPC.4840worker2 # [ 85.370205] nix-grpc-daemon[1281]: E0000 00:00:1789995301.662663 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4841worker2 # [ 85.396149] nix-grpc-daemon[1281]: E0000 00:00:1789995301.680143 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4842worker2 # [ 85.398419] nix-grpc-daemon[1281]: E0000 00:00:1789995301.680320 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4843worker2 # [ 85.407210] nix-grpc-daemon[1281]: E0000 00:00:1789995301.699877 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4844worker2 # [ 85.418384] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=scheduler_connected addr=lb:500514845worker2 # [ 85.420055] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4846worker1 # [ 86.064288] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_yield4847worker1 # [ 86.070182] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_restarting addr=lb:500514848lb # [ 86.444875] niks3-server[1026]: 2026/09/21 12:55:01 INFO lead: released remote=192.168.1.3:606444849worker1 # [ 86.076587] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:411364850worker1 # [ 86.082209] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=worker_gone addr=192.168.1.3:500514851worker2 # [ 85.621724] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=scheduler_connected addr=lb:500514852worker2 # [ 85.627956] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4853worker1 # [ 86.270445] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_connected addr=lb:500514854worker1 # [ 86.273659] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4855worker2 # [ 85.829310] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514856worker2 # [ 85.834616] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4857worker1 # [ 86.474581] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_connected addr=lb:500514858worker1 # [ 86.480146] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4859worker2 # [ 86.035447] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514860worker2 # [ 86.040943] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4861worker1 # [ 86.680766] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_connected addr=lb:500514862worker1 # [ 86.686018] nix-grpc-daemon[773]: ts=2026-09-21T12:55:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4863worker2 # [ 86.242307] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514864worker2 # [ 86.247717] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker1 # [ 86.886970] nix-grpc-daemon[773]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514866worker1 # [ 86.892285] nix-grpc-daemon[773]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4867worker2 # [ 86.450893] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514868worker2 # [ 86.455284] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4869worker1 # [ 87.072010] nix-grpc-daemon[773]: ts=2026-09-21T12:55:02Z level=info event=scheduler_restarting4870worker1 # [ 87.120817] nix-daemon[837]: reaped child process 840, status = succeeded4871worker1 # [ 87.123173] nix-grpc-daemon[773]: ts=2026-09-21T12:55:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4872worker2 # [ 86.651603] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_connected addr=lb:500514873worker2 # [ 86.657078] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4874worker2 # [ 86.858234] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500514875worker2 # [ 86.863361] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4876worker2 # [ 87.064605] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500514877worker2 # [ 87.069692] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4878worker2 # [ 87.270900] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500514879worker2 # [ 87.276145] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4880worker1 # [ 87.924825] nix-grpc-daemon[773]: E0000 00:00:1789995303.116728 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4881worker1 # [ 87.974891] nix-grpc-daemon[773]: E0000 00:00:1789995303.166947 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4882worker2 # [ 87.478996] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500514883worker2 # [ 87.483207] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4884worker2 # [ 87.679879] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500514885worker2 # [ 87.685140] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4886worker2 # [ 87.886038] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_connected addr=lb:500514887worker2 # [ 87.891254] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4888worker1 # [ 88.493111] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4889worker1 # [ 88.499369] systemd[1]: Stopped Nix worker-protocol over gRPC.4890worker1 # [ 88.505891] systemd[1]: nix-grpc-daemon.service: Consumed 2.021s CPU time over 1min 5.090s wall clock time, 52.4M memory peak, 55.2K incoming IP traffic, 97.4K outgoing IP traffic.4891worker1 # [ 88.519932] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4892worker1 # [ 88.523511] systemd[1]: Closed Nix worker-protocol over gRPC.4893worker1 # [ 88.526114] systemd[1]: nix-grpc-daemon.socket: Consumed 726us CPU time over 1min 15.617s wall clock time, 256K memory peak, 65.9K incoming IP traffic, 58.5K outgoing IP traffic.4894lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4895lb: must succeed: curl -sf localhost:9901/clusters4896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4897lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4898client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24899worker2 # [ 88.092507] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_connected addr=lb:500514900worker2 # [ 88.097669] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4901worker2 # [ 88.298815] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_connected addr=lb:500514902worker2 # [ 88.304085] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4903worker2 # [ 88.328096] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606564904worker2 # [ 88.343896] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)4905worker2 # [ 88.374590] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)4906worker2 # [ 88.421451] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)4907worker2 # [ 88.438331] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 3/5)4908worker2 # [ 88.507236] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_connected addr=lb:500514909worker2 # [ 88.512157] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4910worker2 # [ 88.517199] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 418 ms (attempt 4/5)4911worker2 # [ 88.708326] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514912worker2 # [ 88.714010] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4913worker2 # [ 88.915445] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514914worker2 # [ 88.921331] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4915worker2 # [ 88.930538] nix-grpc-daemon[1281]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4916worker2 # [ 88.937298] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4917client # copying 2 paths...4918client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4919worker2 # [ 88.977167] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606564920client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4921worker2 # [ 88.997805] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)4922worker2 # [ 89.030583] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=niks3_push_started pid=13414923worker2 # [ 89.123300] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514924worker2 # [ 89.126076] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4925worker2 # [ 89.254187] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.546Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4926worker2 # [ 89.328703] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514927worker2 # [ 89.332314] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4928worker2 # [ 89.436738] nix-daemon[864]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)4929worker2 # [ 89.454925] nix-daemon[864]: reaped child process 1357, status = succeeded4930lb # [ 90.399493] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received uploads request method=POST path=/api/pending_closures4931lb # [ 90.410695] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received uploads request method=POST path=/api/pending_closures4932worker2 # [ 89.490144] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.783Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4933worker2 # [ 89.494977] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.785Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4934worker2 # [ 89.497160] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.785Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4935worker2 # [ 89.532178] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514936worker2 # [ 89.534583] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4937lb # [ 90.466183] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4938lb # [ 90.478662] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4939lb # [ 90.488286] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4940lb # [ 90.495314] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4941lb # [ 90.502185] niks3-server[1026]: 2026/09/21 12:55:05 INFO Signed narinfos id=25 count=14942worker2 # [ 89.574432] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.866Z level=INFO msg="Uploading 3 narinfos"4943lb # [ 90.506468] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4944lb # [ 90.513424] niks3-server[1026]: 2026/09/21 12:55:05 INFO Signed narinfos id=26 count=24945lb # [ 90.515659] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4946lb # [ 90.537805] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4947lb # [ 90.546771] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4948worker2 # [ 89.620160] nix-grpc-daemon[1341]: time=2026-09-21T12:55:05.913Z level=INFO msg="Upload complete. (362ms)"4949lb # [ 90.552990] niks3-server[1026]: 2026/09/21 12:55:05 INFO Completed upload id=254950lb # [ 90.557166] niks3-server[1026]: 2026/09/21 12:55:05 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4951lb # [ 90.561841] niks3-server[1026]: 2026/09/21 12:55:05 INFO Completed upload id=264952lb # [ 90.563811] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4953worker2 # [ 89.630173] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=0 paths=2 nar_bytes_in=3124954lb # [ 90.569678] niks3-server[1026]: 2026/09/21 12:55:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4955worker2 # [ 89.649191] nix-daemon[864]: reaped child process 1339, status = succeeded4956client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.61 seconds)4957worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4958worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4959worker2 # [ 89.735756] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:06Z level=info event=scheduler_connected addr=lb:500514960worker2 # [ 89.741179] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4961worker1 # [ 90.324987] systemd[1]: Listening on Nix worker-protocol over gRPC.4962worker1 # [ 90.330920] systemd[1]: Starting Nix worker-protocol over gRPC...4963worker2 # [ 89.941895] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:06Z level=info event=scheduler_connected addr=lb:500514964worker2 # [ 89.945174] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4965worker1 # [ 90.515726] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4966worker1 # [ 90.518458] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4967worker1 # [ 90.522828] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14968worker1 # [ 90.530482] nix-grpc-daemon[1044]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4969worker1 # [ 90.531952] nix-grpc-daemon[1044]: E0000 00:00:1789995305.722114 1044 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4970worker1 # [ 90.549910] nix-grpc-daemon[1044]: E0000 00:00:1789995305.728358 1044 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4971worker1 # [ 90.553650] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4972lb # [ 90.928181] niks3-server[1026]: 2026/09/21 12:55:06 INFO lead: acquired remote=192.168.1.3:571124973lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4974lb: must succeed: curl -sf localhost:9901/clusters4975worker1 # [ 90.580820] systemd[1]: Started Nix worker-protocol over gRPC.4976worker1 # [ 90.585009] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=scheduler_take_over4977worker1 # [ 90.586199] nix-grpc-daemon[1044]: E0000 00:00:1789995305.749041 1086 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4978worker1 # [ 90.595466] nix-grpc-daemon[1044]: E0000 00:00:1789995305.766266 1086 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4979worker1 # [ 90.598416] nix-grpc-daemon[1044]: E0000 00:00:1789995305.766430 1086 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4980worker1 # [ 90.601070] nix-grpc-daemon[1044]: E0000 00:00:1789995305.786873 1086 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4981worker1 # [ 90.613016] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=scheduler_connected addr=lb:500514982worker1 # [ 90.618712] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:446564983worker1 # [ 90.620626] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04984lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4985lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4986(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.24 seconds)4987subtest: scheduler down: the other node takes the lock and keeps it4988lb: must succeed: curl -sf localhost:9901/clusters4989worker2 # [ 90.146334] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:06Z level=info event=scheduler_connected addr=lb:500514990worker1 # [ 90.712141] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=oidc_jwks provider=mock keys=14991worker1 # [ 90.717492] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:446564992worker1 # [ 90.722637] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04993lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4994lb: must succeed: curl -sf localhost:9901/clusters4995lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4996lb: must succeed: curl -sf localhost:9901/clusters4997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4998lb: must succeed: curl -sf localhost:9901/clusters4999lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5000worker1 # [ 90.949905] systemd[1]: Stopping Nix worker-protocol over gRPC...5001worker1 # [ 91.540530] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=signal_exit5002worker1 # [ 91.750491] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=scheduler_yield5003worker2 # [ 91.202324] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:07Z level=info event=scheduler_restarting addr=lb:500515004lb # [ 92.132771] niks3-server[1026]: 2026/09/21 12:55:07 INFO lead: released remote=192.168.1.3:571125005worker1 # [ 91.755828] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446565006worker1 # [ 91.762062] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=worker_gone addr=192.168.1.4:500515007worker1 # [ 91.767962] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=scheduler_restarting addr=lb:500515008worker1 # [ 91.772009] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446565009worker1 # [ 91.776422] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:06Z level=info event=worker_gone addr=192.168.1.3:500515010worker2 # [ 91.402286] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515011worker1 # [ 91.962777] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515012worker2 # [ 91.409553] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5013worker1 # [ 91.969348] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5014worker2 # [ 91.611084] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515015worker1 # [ 92.170204] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 91.616636] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5017worker1 # [ 92.176789] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5018worker2 # [ 91.817627] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_connected addr=lb:500515019worker1 # [ 92.377910] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515020worker2 # [ 91.822907] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5021worker1 # [ 92.385126] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5022worker2 # [ 92.024040] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_connected addr=lb:500515023worker2 # [ 92.029253] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5024worker1 # [ 92.587939] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_connected addr=lb:500515025worker1 # [ 92.592147] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker1 # [ 92.756220] nix-grpc-daemon[1044]: ts=2026-09-21T12:55:07Z level=info event=scheduler_restarting5027worker2 # [ 92.232153] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_connected addr=lb:500515028worker2 # [ 92.236494] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker2 # [ 92.433169] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_connected addr=lb:500515030worker2 # [ 92.438508] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5031worker2 # [ 92.639715] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_connected addr=lb:500515032worker2 # [ 92.645234] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5033lb # [ 93.764475] envoy[915]: conn peer=[2001:db8:1::1]:56136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=325034lb # [ 93.764771] envoy[915]: conn peer=[2001:db8:1::3]:40752 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10007 tx=2765 ms=632485035lb # [ 93.764940] envoy[915]: conn peer=[2001:db8:1::1]:56148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=13195036worker2 # [ 92.846315] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_connected addr=lb:500515037lb # [ 93.765043] envoy[915]: conn peer=[2001:db8:1::3]:53812 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1028 tx=377 ms=21895038worker2 # [ 92.855889] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5039worker1 # [ 93.571416] nix-grpc-daemon[1044]: E0000 00:00:1789995308.763304 1089 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5040worker1 # [ 93.571829] nix-grpc-daemon[1044]: E0000 00:00:1789995308.763530 1089 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5041worker2 # [ 93.057586] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_connected addr=lb:500515042worker2 # [ 93.062911] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker2 # [ 93.266031] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 93.270233] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker2 # [ 93.466892] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 93.472391] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker2 # [ 93.673686] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_connected addr=lb:500515048worker2 # [ 93.679016] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5049worker2 # [ 93.880405] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_connected addr=lb:500515050worker2 # [ 93.885736] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5051worker1 # [ 94.567061] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5052worker1 # [ 94.573315] systemd[1]: Stopped Nix worker-protocol over gRPC.5053worker1 # [ 94.575800] systemd[1]: nix-grpc-daemon.service: Consumed 214ms CPU time over 4.232s wall clock time, 13.8M memory peak, 9.3K incoming IP traffic, 7.9K outgoing IP traffic.5054worker1 # [ 94.591794] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5055worker1 # [ 94.594216] systemd[1]: Closed Nix worker-protocol over gRPC.5056worker1 # [ 94.597100] systemd[1]: nix-grpc-daemon.socket: Consumed 892us CPU time over 4.276s wall clock time, 256K memory peak, 19.6K incoming IP traffic, 13.3K outgoing IP traffic.5057worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5058worker2 # [ 94.087239] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_connected addr=lb:500515059worker2 # [ 94.093759] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5060worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds)5061worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5062worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5063worker2 # [ 94.296713] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_connected addr=lb:500515064worker2 # [ 94.301022] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5065worker2 # [ 94.497878] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_connected addr=lb:500515066worker2 # [ 94.503415] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5067worker2 # [ 94.704830] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:10Z level=info event=scheduler_connected addr=lb:500515068worker2 # [ 94.710349] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5069worker2 # [ 94.911715] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_connected addr=lb:500515070worker2 # [ 94.917382] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5071worker2 # [ 95.118525] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_connected addr=lb:500515072worker2 # [ 95.124109] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5073worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5074worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5075worker2 # [ 95.327450] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_connected addr=lb:500515076worker2 # [ 95.331882] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5077worker2 # [ 95.366996] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_take_over5078lb # [ 96.297643] niks3-server[1026]: 2026/09/21 12:55:11 INFO lead: acquired remote=192.168.1.4:607825079worker2 # [ 95.528777] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=scheduler_connected addr=lb:500515080worker2 # [ 95.558639] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=oidc_jwks provider=mock keys=15081worker2 # [ 95.563354] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:531845082worker2 # [ 95.568926] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05083worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5084worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5085client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25086client # don't know how to build these paths:5087client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5088worker2 # [ 96.655984] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606565089worker2 # [ 96.664933] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606565090worker2 # [ 96.694539] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:12Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05091worker2 # [ 96.702276] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:12Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15092worker2 # [ 96.742601] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565093worker2 # [ 96.760963] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606565094client # copying 1 paths to 'lb:50051'...5095client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5096worker2 # [ 96.778052] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606565097worker2 # [ 96.794288] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5098worker2 # [ 97.004258] nix-daemon[864]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5099worker2 # [ 97.024922] nix-daemon[864]: reaped child process 1384, status = succeeded5100worker2 # [ 97.206507] nix-daemon[864]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5101worker2 # [ 97.224839] nix-daemon[864]: reaped child process 1389, status = succeeded5102lb # [ 98.167091] niks3-server[1026]: 2026/09/21 12:55:13 INFO Received uploads request method=POST path=/api/pending_closures5103worker2 # [ 97.247524] nix-grpc-daemon[1341]: time=2026-09-21T12:55:13.540Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5104worker2 # [ 97.249182] nix-grpc-daemon[1341]: time=2026-09-21T12:55:13.540Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5105lb # [ 98.200505] niks3-server[1026]: 2026/09/21 12:55:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5106lb # [ 98.217044] niks3-server[1026]: 2026/09/21 12:55:13 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5107lb # [ 98.229241] niks3-server[1026]: 2026/09/21 12:55:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5108lb # [ 98.239220] niks3-server[1026]: 2026/09/21 12:55:13 INFO Signed narinfos id=27 count=15109worker2 # [ 97.310541] nix-grpc-daemon[1341]: time=2026-09-21T12:55:13.603Z level=INFO msg="Uploading 1 narinfos"5110lb # [ 98.248527] niks3-server[1026]: 2026/09/21 12:55:13 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5111lb # [ 98.256116] niks3-server[1026]: 2026/09/21 12:55:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5112lb # [ 98.268536] niks3-server[1026]: 2026/09/21 12:55:13 INFO Completed upload id=275113worker2 # [ 97.338975] nix-grpc-daemon[1341]: time=2026-09-21T12:55:13.632Z level=INFO msg="Upload complete. (526ms)"5114worker2 # [ 97.349920] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=0 paths=1 nar_bytes_in=5205115worker2 # [ 97.365064] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565116worker2 # [ 97.377814] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5117worker2 # [ 97.386630] nix-daemon[864]: reaped child process 1379, status = succeeded5118client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5119client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5120worker2 # [ 97.402503] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5121client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5122client # farm-a> LOG-farm-a-on-standby5123worker2 # [ 98.223768] nix-daemon[864]: reaped child process 1393, status = succeeded5124worker2 # [ 98.392386] nix-daemon[864]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5125worker2 # [ 98.411787] nix-daemon[864]: reaped child process 1412, status = succeeded5126lb # [ 99.354716] niks3-server[1026]: 2026/09/21 12:55:14 INFO Received uploads request method=POST path=/api/pending_closures5127worker2 # [ 98.439977] nix-grpc-daemon[1341]: time=2026-09-21T12:55:14.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5128worker2 # [ 98.441525] nix-grpc-daemon[1341]: time=2026-09-21T12:55:14.731Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5129lb # [ 99.399356] niks3-server[1026]: 2026/09/21 12:55:14 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5130lb # [ 99.412888] niks3-server[1026]: 2026/09/21 12:55:14 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5131lb # [ 99.422951] niks3-server[1026]: 2026/09/21 12:55:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5132lb # [ 99.428614] niks3-server[1026]: 2026/09/21 12:55:14 INFO Signed narinfos id=28 count=15133worker2 # [ 98.499014] nix-grpc-daemon[1341]: time=2026-09-21T12:55:14.791Z level=INFO msg="Uploading 1 narinfos"5134lb # [ 99.432932] niks3-server[1026]: 2026/09/21 12:55:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5135lb # [ 99.444183] niks3-server[1026]: 2026/09/21 12:55:14 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5136lb # [ 99.449711] niks3-server[1026]: 2026/09/21 12:55:14 INFO Completed upload id=285137worker2 # [ 98.519431] nix-grpc-daemon[1341]: time=2026-09-21T12:55:14.812Z level=INFO msg="Upload complete. (306ms)"5138lb # [ 99.452537] niks3-server[1026]: 2026/09/21 12:55:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5139worker2 # [ 98.539531] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15140worker2 # [ 98.555131] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05141worker2 # [ 98.558300] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15142worker2 # [ 98.562020] nix-daemon[864]: reaped child process 1391, status = succeeded5143worker2 # [ 98.582603] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565144worker2 # [ 98.597695] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606565145worker2 # [ 98.612693] nix-grpc-daemon[1341]: time=2026-09-21T12:55:14.906Z level=INFO msg="All 1 paths already cached"5146client # copying 1 paths to 'lb:50051'...5147client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5148worker2 # [ 98.622719] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606565149worker2 # [ 98.634738] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5150worker2 # [ 98.837923] nix-daemon[864]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5151worker2 # [ 98.858254] nix-daemon[864]: reaped child process 1421, status = succeeded5152worker2 # [ 99.031283] nix-daemon[864]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5153worker2 # [ 99.050899] nix-daemon[864]: reaped child process 1426, status = succeeded5154lb # [ 99.993217] niks3-server[1026]: 2026/09/21 12:55:15 INFO Received uploads request method=POST path=/api/pending_closures5155worker2 # [ 99.075186] nix-grpc-daemon[1341]: time=2026-09-21T12:55:15.368Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5156worker2 # [ 99.076793] nix-grpc-daemon[1341]: time=2026-09-21T12:55:15.368Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5157lb # [ 100.023192] niks3-server[1026]: 2026/09/21 12:55:15 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5158lb # [ 100.037633] niks3-server[1026]: 2026/09/21 12:55:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5159worker2 # [ 99.113697] nix-grpc-daemon[1341]: time=2026-09-21T12:55:15.406Z level=INFO msg="Uploading 1 narinfos"5160lb # [ 100.050021] niks3-server[1026]: 2026/09/21 12:55:15 INFO Signed narinfos id=29 count=15161worker2 # [ 99.130325] nix-grpc-daemon[1341]: time=2026-09-21T12:55:15.423Z level=INFO msg="Upload complete. (474ms)"5162lb # [ 100.055415] niks3-server[1026]: 2026/09/21 12:55:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5163lb # [ 100.064988] niks3-server[1026]: 2026/09/21 12:55:15 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5164worker2 # [ 99.136226] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=0 paths=1 nar_bytes_in=6405165lb # [ 100.069557] niks3-server[1026]: 2026/09/21 12:55:15 INFO Completed upload id=295166lb # [ 100.071593] niks3-server[1026]: 2026/09/21 12:55:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5167worker2 # [ 99.149400] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565168worker2 # [ 99.155704] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5169worker2 # [ 99.160905] nix-daemon[864]: reaped child process 1416, status = succeeded5170client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5171client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5172worker2 # [ 99.173381] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5173client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5174client # farm-b> LOG-farm-b-on-standby5175worker2 # [ 100.617432] nix-daemon[864]: reaped child process 1430, status = succeeded5176worker2 # [ 100.788130] nix-daemon[864]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5177worker2 # [ 100.806631] nix-daemon[864]: reaped child process 1449, status = succeeded5178lb # [ 101.749861] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received uploads request method=POST path=/api/pending_closures5179worker2 # [ 100.830821] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5180worker2 # [ 100.832529] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.123Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5181lb # [ 101.785903] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5182lb # [ 101.799504] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5183lb # [ 101.809478] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5184worker2 # [ 100.885652] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.178Z level=INFO msg="Uploading 1 narinfos"5185lb # [ 101.819839] niks3-server[1026]: 2026/09/21 12:55:17 INFO Signed narinfos id=30 count=15186lb # [ 101.826127] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5187lb # [ 101.836637] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5188lb # [ 101.842682] niks3-server[1026]: 2026/09/21 12:55:17 INFO Completed upload id=305189worker2 # [ 100.913185] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.206Z level=INFO msg="Upload complete. (329ms)"5190lb # [ 101.845557] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5191worker2 # [ 100.930338] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15192worker2 # [ 100.947702] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05193worker2 # [ 100.951684] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15194worker2 # [ 100.955788] nix-daemon[864]: reaped child process 1428, status = succeeded5195worker2 # [ 100.974511] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565196worker2 # [ 100.989780] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606565197worker2 # [ 101.005170] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.298Z level=INFO msg="All 2 paths already cached"5198client # copying 1 paths to 'lb:50051'...5199client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5200worker2 # [ 101.015298] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606565201worker2 # [ 101.030582] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5202worker2 # [ 101.237310] nix-daemon[864]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5203worker2 # [ 101.254926] nix-daemon[864]: reaped child process 1458, status = succeeded5204worker2 # [ 101.428110] nix-daemon[864]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5205worker2 # [ 101.444265] nix-daemon[864]: reaped child process 1463, status = succeeded5206lb # [ 102.387694] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received uploads request method=POST path=/api/pending_closures5207worker2 # [ 101.467747] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.761Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5208worker2 # [ 101.469375] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.761Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5209lb # [ 102.413747] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5210worker2 # [ 101.496482] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.789Z level=INFO msg="Uploading 1 narinfos"5211lb # [ 102.428371] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5212lb # [ 102.437500] niks3-server[1026]: 2026/09/21 12:55:17 INFO Signed narinfos id=31 count=15213lb # [ 102.445000] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5214worker2 # [ 101.522871] nix-grpc-daemon[1341]: time=2026-09-21T12:55:17.816Z level=INFO msg="Upload complete. (475ms)"5215lb # [ 102.453073] niks3-server[1026]: 2026/09/21 12:55:17 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5216lb # [ 102.458541] niks3-server[1026]: 2026/09/21 12:55:17 INFO Completed upload id=315217lb # [ 102.461895] niks3-server[1026]: 2026/09/21 12:55:17 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5218worker2 # [ 101.532801] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=0 paths=1 nar_bytes_in=7685219worker2 # [ 101.540372] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565220worker2 # [ 101.545863] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5221worker2 # [ 101.550057] nix-daemon[864]: reaped child process 1453, status = succeeded5222client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5223worker2 # [ 101.561119] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5224client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5225client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5226client # farm-top> LOG-farm-top-on-standby5227worker2 # [ 102.993999] nix-daemon[864]: reaped child process 1467, status = succeeded5228worker2 # [ 103.164282] nix-daemon[864]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5229worker2 # [ 103.181340] nix-daemon[864]: reaped child process 1486, status = succeeded5230lb # [ 104.125907] niks3-server[1026]: 2026/09/21 12:55:19 INFO Received uploads request method=POST path=/api/pending_closures5231worker2 # [ 103.208494] nix-grpc-daemon[1341]: time=2026-09-21T12:55:19.500Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5232worker2 # [ 103.210292] nix-grpc-daemon[1341]: time=2026-09-21T12:55:19.500Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5233lb # [ 104.158094] niks3-server[1026]: 2026/09/21 12:55:19 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5234lb # [ 104.175488] niks3-server[1026]: 2026/09/21 12:55:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5235worker2 # [ 103.252085] nix-grpc-daemon[1341]: time=2026-09-21T12:55:19.545Z level=INFO msg="Uploading 1 narinfos"5236lb # [ 104.185860] niks3-server[1026]: 2026/09/21 12:55:19 INFO Signed narinfos id=32 count=15237lb # [ 104.191152] niks3-server[1026]: 2026/09/21 12:55:19 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5238lb # [ 104.196555] niks3-server[1026]: 2026/09/21 12:55:19 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5239lb # [ 104.203206] niks3-server[1026]: 2026/09/21 12:55:19 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5240lb # [ 104.207388] niks3-server[1026]: 2026/09/21 12:55:19 INFO Completed upload id=325241worker2 # [ 103.277870] nix-grpc-daemon[1341]: time=2026-09-21T12:55:19.571Z level=INFO msg="Upload complete. (307ms)"5242lb # [ 104.210173] niks3-server[1026]: 2026/09/21 12:55:19 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5243worker2 # [ 103.293169] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15244worker2 # [ 103.309162] nix-daemon[864]: reaped child process 1465, status = succeeded5245client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.97 seconds)5246client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5247client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5248worker1 # [ 104.221170] systemd[1]: Listening on Nix worker-protocol over gRPC.5249worker1 # [ 104.226998] systemd[1]: Starting Nix worker-protocol over gRPC...5250worker1 # [ 104.422403] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5251worker1 # [ 104.425012] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5252worker1 # [ 104.429132] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15253worker1 # [ 104.436066] nix-grpc-daemon[1111]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5254worker1 # [ 104.437505] nix-grpc-daemon[1111]: E0000 00:00:1789995319.627642 1111 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5255worker1 # [ 104.447808] nix-grpc-daemon[1111]: E0000 00:00:1789995319.633866 1111 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5256worker1 # [ 104.451531] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5257worker1 # [ 104.455717] systemd[1]: Started Nix worker-protocol over gRPC.5258worker1 # [ 104.460172] nix-grpc-daemon[1111]: E0000 00:00:1789995319.650041 1150 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5259lb: must succeed: curl -sf localhost:9901/clusters5260worker1 # [ 104.488126] nix-grpc-daemon[1111]: E0000 00:00:1789995319.668763 1150 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5261worker1 # [ 104.490479] nix-grpc-daemon[1111]: E0000 00:00:1789995319.668940 1150 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5262worker1 # [ 104.497202] nix-grpc-daemon[1111]: E0000 00:00:1789995319.688415 1150 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5263worker1 # [ 104.514916] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:19Z level=info event=scheduler_connected addr=lb:500515264worker2 # [ 103.963183] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:531845265worker2 # [ 103.963574] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05266lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5267worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5268worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5269lb: must succeed: curl -sf localhost:9901/clusters5270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5271lb: must succeed: curl -sf localhost:9901/clusters5272lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5273worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5274worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5275(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.18 seconds)5276subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5277client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5278client # [ 105.312284] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5279client # Running as unit: rs.service; invocation ID: e5427b6fb09849688d88b44cca09e5515280client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5281client # [ 105.560446] nix[1168]: don't know how to build these paths:5282client # [ 105.561449] nix[1168]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5283worker2 # [ 104.617482] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606565284worker2 # [ 104.633019] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05285worker2 # [ 104.640532] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15286worker2 # [ 104.678274] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565287worker2 # [ 104.693918] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606565288client # [ 105.651870] nix[1168]: copying 1 paths to 'lb:50051'...5289worker2 # [ 104.708610] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606565290client # [ 105.654308] nix[1168]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5291worker2 # [ 104.723976] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5292worker2 # [ 104.931285] nix-daemon[864]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5293worker2 # [ 104.949449] nix-daemon[864]: reaped child process 1502, status = succeeded5294worker2 # [ 105.125537] nix-daemon[864]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5295worker2 # [ 105.143574] nix-daemon[864]: reaped child process 1507, status = succeeded5296lb # [ 106.085475] niks3-server[1026]: 2026/09/21 12:55:21 INFO Received uploads request method=POST path=/api/pending_closures5297worker2 # [ 105.168414] nix-grpc-daemon[1341]: time=2026-09-21T12:55:21.461Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5298worker2 # [ 105.169911] nix-grpc-daemon[1341]: time=2026-09-21T12:55:21.461Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5299lb # [ 106.115529] niks3-server[1026]: 2026/09/21 12:55:21 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5300worker2 # [ 105.198450] nix-grpc-daemon[1341]: time=2026-09-21T12:55:21.491Z level=INFO msg="Uploading 1 narinfos"5301lb # [ 106.130643] niks3-server[1026]: 2026/09/21 12:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5302lb # [ 106.142234] niks3-server[1026]: 2026/09/21 12:55:21 INFO Signed narinfos id=33 count=15303lb # [ 106.146416] niks3-server[1026]: 2026/09/21 12:55:21 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5304worker2 # [ 105.222900] nix-grpc-daemon[1341]: time=2026-09-21T12:55:21.516Z level=INFO msg="Upload complete. (481ms)"5305lb # [ 106.154665] niks3-server[1026]: 2026/09/21 12:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5306lb # [ 106.159513] niks3-server[1026]: 2026/09/21 12:55:21 INFO Completed upload id=335307worker2 # [ 105.229699] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=0 paths=1 nar_bytes_in=4165308lb # [ 106.162687] niks3-server[1026]: 2026/09/21 12:55:21 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5309worker2 # [ 105.243285] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606565310worker2 # [ 105.249345] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5311worker2 # [ 105.254077] nix-daemon[864]: reaped child process 1497, status = succeeded5312client # [ 106.206884] nix[1168]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5313worker2 # [ 105.266996] nix-daemon[864]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5314lb: must succeed: curl -sf localhost:9901/clusters5315lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5316lb: must succeed: curl -sf localhost:9901/clusters5317lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5318lb: must succeed: curl -sf localhost:9901/clusters5319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5320lb: must succeed: curl -sf localhost:9901/clusters5321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5322worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5323worker1 # [ 107.881219] systemd-journald[274]: Received client request to rotate journal, rotating.5324worker1 # [ 107.901694] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/57c1ea9de2ff4248a3a2bb9e4bf3fc45.5325worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5326worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5327worker2 # [ 107.391830] systemd[1]: Stopping Nix worker-protocol over gRPC...5328worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5329client: must succeed: sleep 5.05330worker2 # [ 108.365499] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:24Z level=info event=draining5331client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5332worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5333lb # [ 113.768241] envoy[915]: conn peer=[2001:db8:1::1]:40602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9623 tx=4120 ms=66485334worker2 # [ 112.909601] nix-daemon[864]: reaped child process 1511, status = succeeded5335worker2 # [ 113.061993] nix-daemon[864]: accepted connection from pid 1552, user nix-grpc-daemon (trusted)5336worker2 # [ 113.083425] nix-daemon[864]: reaped child process 1555, status = succeeded5337lb # [ 114.026697] niks3-server[1026]: 2026/09/21 12:55:29 INFO Received uploads request method=POST path=/api/pending_closures5338worker2 # [ 113.108582] nix-grpc-daemon[1341]: time=2026-09-21T12:55:29.399Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5339worker2 # [ 113.110236] nix-grpc-daemon[1341]: time=2026-09-21T12:55:29.399Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5340lb # [ 114.068233] niks3-server[1026]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5341lb # [ 114.082108] niks3-server[1026]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5342lb # [ 114.091225] niks3-server[1026]: 2026/09/21 12:55:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5343worker2 # [ 113.167150] nix-grpc-daemon[1341]: time=2026-09-21T12:55:29.460Z level=INFO msg="Uploading 1 narinfos"5344lb # [ 114.098333] niks3-server[1026]: 2026/09/21 12:55:29 INFO Signed narinfos id=34 count=15345lb # [ 114.103857] niks3-server[1026]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5346lb # [ 114.115443] niks3-server[1026]: 2026/09/21 12:55:29 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5347lb # [ 114.121664] niks3-server[1026]: 2026/09/21 12:55:29 INFO Completed upload id=345348worker2 # [ 113.191552] nix-grpc-daemon[1341]: time=2026-09-21T12:55:29.485Z level=INFO msg="Upload complete. (309ms)"5349lb # [ 114.124844] niks3-server[1026]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5350worker2 # [ 113.205513] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60656 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15351worker2 # [ 113.221647] nix-daemon[864]: reaped child process 1509, status = succeeded5352client # [ 114.189319] systemd[1]: rs.service: Deactivated successfully.5353client # [ 114.193723] systemd[1]: rs.service: Consumed 176ms CPU time over 8.877s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5354worker2 # [ 113.369155] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=signal_exit5355worker2 # [ 113.594426] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=scheduler_yield5356lb # [ 114.532934] niks3-server[1026]: 2026/09/21 12:55:29 INFO lead: released remote=192.168.1.4:607825357worker2 # [ 113.608903] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=scheduler_restarting addr=lb:500515358worker1 # [ 114.164651] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_restarting addr=lb:500515359worker2 # [ 113.615550] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531845360worker2 # [ 113.620407] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=worker_gone addr=192.168.1.4:500515361worker2 # [ 113.623905] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531845362worker2 # [ 113.627802] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:29Z level=info event=worker_gone addr=192.168.1.3:500515363worker1 # [ 114.363853] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_connected addr=lb:500515364worker2 # [ 113.810297] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 114.367465] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker2 # [ 113.815869] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367lb # [ 114.829628] niks3-server[1026]: 2026/09/21 12:55:30 INFO lead: acquired remote=192.168.1.3:372465368worker1 # [ 114.458458] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_take_over5369worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.15 seconds)5370client: waiting for success: ! systemctl is-active rs5371client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5372client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5373worker1 # [ 114.568445] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_connected addr=lb:500515374worker2 # [ 114.017042] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 114.574080] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 114.022621] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5377client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5378client: must fail: journalctl -u rs | grep -q 'reconnecting'5379client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5380lb: must succeed: curl -sf localhost:9901/clusters5381lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5382worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5383worker1 # [ 114.775340] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_connected addr=lb:500515384worker2 # [ 114.225986] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515385worker1 # [ 114.781729] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker2 # [ 114.230349] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5388worker2 # [ 114.426387] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515389worker1 # [ 114.982096] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515390worker2 # [ 114.430950] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker1 # [ 114.987321] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker2 # [ 114.600303] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=scheduler_restarting5393worker1 # [ 115.189758] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515394worker1 # [ 115.193307] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5395worker2 # [ 114.662404] nix-grpc-daemon[1281]: ts=2026-09-21T12:55:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5396worker2 # [ 114.667647] nix-daemon[864]: reaped child process 1333, status = succeeded5397worker1 # [ 115.390762] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515398worker1 # [ 115.396884] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker1 # [ 115.597862] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=scheduler_connected addr=lb:500515400worker1 # [ 115.606965] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:468985401worker1 # [ 115.612217] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05402lb: must succeed: curl -sf localhost:9901/clusters5403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5405worker2 # [ 115.422926] nix-grpc-daemon[1281]: E0000 00:00:1789995331.714916 1326 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5406worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5407worker2 # [ 115.438826] nix-grpc-daemon[1281]: E0000 00:00:1789995331.732188 1326 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5408worker2 # [ 116.138988] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5409worker2 # [ 116.144984] systemd[1]: Stopped Nix worker-protocol over gRPC.5410worker2 # [ 116.147493] systemd[1]: nix-grpc-daemon.service: Consumed 2.033s CPU time over 31.001s wall clock time, 36.8M memory peak, 63.5K incoming IP traffic, 163.7K outgoing IP traffic.5411worker2 # [ 116.170071] systemd[1]: Starting scheduler-token.service...5412worker2 # [ 116.255036] systemd[1]: scheduler-token.service: Deactivated successfully.5413worker2 # [ 116.259958] systemd[1]: Finished scheduler-token.service.5414worker2 # [ 116.261639] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5415worker2 # [ 116.265718] systemd[1]: Starting Nix worker-protocol over gRPC...5416lb: must succeed: curl -sf localhost:9901/clusters5417worker2 # [ 116.446399] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5418worker2 # [ 116.450217] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5419worker2 # [ 116.452239] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15420worker2 # [ 116.460194] nix-grpc-daemon[1565]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5421worker2 # [ 116.461840] nix-grpc-daemon[1565]: E0000 00:00:1789995332.751217 1565 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5422worker2 # [ 116.470596] nix-grpc-daemon[1565]: E0000 00:00:1789995332.753582 1565 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5423worker2 # [ 116.473696] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5424worker2 # [ 116.477051] systemd[1]: Started Nix worker-protocol over gRPC.5425worker2 # [ 116.480076] nix-grpc-daemon[1565]: E0000 00:00:1789995332.773425 1604 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5426worker2 # [ 116.508095] nix-grpc-daemon[1565]: E0000 00:00:1789995332.791687 1604 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5427worker2 # [ 116.510673] nix-grpc-daemon[1565]: E0000 00:00:1789995332.791980 1604 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5428worker2 # [ 116.520340] nix-grpc-daemon[1565]: E0000 00:00:1789995332.812156 1604 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5430worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5431worker2 # [ 116.530090] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:32Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 117.089451] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=info event=oidc_jwks provider=mock keys=15433worker1 # [ 117.089629] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:468985434worker1 # [ 117.089682] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05435worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5436(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.29 seconds)5437subtest: scheduler restart mid-build: build finishes, no double build5438client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5439client # [ 117.597123] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5440client # Running as unit: mid.service; invocation ID: d6c2488064034ea4a0a73686a1c30f325441client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5442worker1 # [ 117.470684] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469005443worker1 # [ 117.486779] nix-daemon[837]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5444client # [ 117.893648] nix[1204]: don't know how to build these paths:5445client # [ 117.894946] nix[1204]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5446worker1 # [ 117.518645] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05447worker1 # [ 117.526339] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15448worker1 # [ 117.565698] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469005449worker1 # [ 117.590651] nix-grpc-daemon[1111]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)5450worker1 # [ 117.625549] nix-grpc-daemon[1111]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)5451worker1 # [ 117.796947] nix-grpc-daemon[1111]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 3/5)5452worker1 # [ 117.979775] nix-grpc-daemon[1111]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5)5453worker1 # [ 118.249283] nix-grpc-daemon[1111]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5454worker1 # [ 118.251382] nix-grpc-daemon[1111]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5455worker1 # [ 118.272102] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469005456client # [ 118.676054] nix[1204]: copying 1 paths to 'lb:50051'...5457client # [ 118.676281] nix[1204]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5458worker1 # [ 118.288157] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469005459worker1 # [ 118.301440] nix-daemon[837]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5460worker1 # [ 118.326291] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:33Z level=info event=niks3_push_started pid=12305461worker1 # [ 118.536597] nix-grpc-daemon[1230]: time=2026-09-21T12:55:33.728Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5462worker1 # [ 118.719543] nix-daemon[837]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)5463worker1 # [ 118.740973] nix-daemon[837]: reaped child process 1250, status = succeeded5464worker1 # [ 118.919403] nix-daemon[837]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5465worker1 # [ 118.938125] nix-daemon[837]: reaped child process 1255, status = succeeded5466lb # [ 119.330340] niks3-server[1026]: 2026/09/21 12:55:34 INFO Received uploads request method=POST path=/api/pending_closures5467worker1 # [ 118.970370] nix-grpc-daemon[1230]: time=2026-09-21T12:55:34.161Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5468worker1 # [ 118.972416] nix-grpc-daemon[1230]: time=2026-09-21T12:55:34.161Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5469lb # [ 119.383429] niks3-server[1026]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5470worker1 # [ 119.019032] nix-grpc-daemon[1230]: time=2026-09-21T12:55:34.210Z level=INFO msg="Uploading 1 narinfos"5471lb # [ 119.394330] niks3-server[1026]: 2026/09/21 12:55:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5472lb # [ 119.402911] niks3-server[1026]: 2026/09/21 12:55:34 INFO Signed narinfos id=35 count=15473lb # [ 119.415384] niks3-server[1026]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5474lb # [ 119.425273] niks3-server[1026]: 2026/09/21 12:55:34 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5475worker1 # [ 119.058974] nix-grpc-daemon[1230]: time=2026-09-21T12:55:34.251Z level=INFO msg="Upload complete. (518ms)"5476lb # [ 119.435534] niks3-server[1026]: 2026/09/21 12:55:34 INFO Completed upload id=355477worker1 # [ 119.064311] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46900 duration_s=0 paths=1 nar_bytes_in=4245478worker1 # [ 119.076335] nix-grpc-daemon[1111]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469005479worker1 # [ 119.081882] nix-daemon[837]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5480lb # [ 119.460827] niks3-server[1026]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5481worker1 # [ 119.087118] nix-daemon[837]: reaped child process 1228, status = succeeded5482client # [ 119.486491] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5483worker1 # [ 119.101441] nix-daemon[837]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5484lb: must succeed: curl -sf localhost:9901/clusters5485lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5486lb: must succeed: curl -sf localhost:9901/clusters5487lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5488worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5489worker1 # [ 121.205948] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1111 (nix-grpc-daemon) on client request.5490worker1 # [ 121.207635] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5491lb # [ 121.597329] niks3-server[1026]: 2026/09/21 12:55:36 INFO lead: released remote=192.168.1.3:372465492client # [ 121.619964] nix[1204]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5493worker2 # [ 120.677245] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:36Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5494worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5495client: must succeed: sleep 5.05496client # [ 121.626800] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5497worker1 # [ 121.248245] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5498worker1 # [ 121.249951] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5499worker1 # [ 121.251195] systemd[1]: nix-grpc-daemon.service: Consumed 603ms CPU time over 17.018s wall clock time, 34.3M memory peak, 16.8K incoming IP traffic, 26.6K outgoing IP traffic.5500worker1 # [ 121.271777] nix-daemon[837]: reaped child process 1257, status = succeeded5501worker1 # [ 121.273927] nix-daemon[837]: reaped child process 1220, status = succeeded5502worker1 # [ 121.280723] nix-daemon[1259]: unexpected Nix daemon error: error: interrupted by the user5503worker1 # [ 121.290429] nix-daemon[837]: reaped child process 1259, status = failed with exit code 15504worker1 # [ 121.408793] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5505worker1 # [ 121.419111] systemd[1]: Starting Nix worker-protocol over gRPC...5506worker2 # [ 120.877716] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:37Z level=info event=scheduler_connected addr=lb:500515507worker1 # [ 121.625983] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5508worker1 # [ 121.629309] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5509worker1 # [ 121.633454] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15510worker1 # [ 121.643122] nix-grpc-daemon[1296]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5511lb # [ 122.032722] niks3-server[1026]: 2026/09/21 12:55:37 INFO lead: acquired remote=192.168.1.3:493085512worker1 # [ 121.659749] nix-grpc-daemon[1296]: E0000 00:00:1789995336.832699 1296 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5513worker1 # [ 121.669451] nix-grpc-daemon[1296]: E0000 00:00:1789995336.835017 1296 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5514worker1 # [ 121.676978] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5515worker1 # [ 121.712324] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=scheduler_take_over5516worker1 # [ 121.713962] systemd[1]: Started Nix worker-protocol over gRPC.5517worker1 # [ 121.714970] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=oidc_jwks provider=mock keys=15518worker1 # [ 121.719549] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:537905519worker1 # [ 121.722511] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05520worker1 # [ 121.724849] nix-grpc-daemon[1296]: E0000 00:00:1789995336.872722 1335 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5521worker1 # [ 121.731228] nix-grpc-daemon[1296]: E0000 00:00:1789995336.889958 1335 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5522worker1 # [ 121.739288] nix-grpc-daemon[1296]: E0000 00:00:1789995336.890282 1335 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5523worker1 # [ 121.743164] nix-grpc-daemon[1296]: E0000 00:00:1789995336.910253 1335 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5524worker1 # [ 121.746603] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:537905525worker1 # [ 121.749832] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=scheduler_connected addr=lb:500515526worker1 # [ 121.751359] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05527worker1 # [ 121.753389] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05528worker1 # [ 121.755257] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:36Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15529worker2 # [ 121.256230] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345405530worker2 # [ 121.267091] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5531worker2 # [ 121.286996] nix-grpc-daemon[1565]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5532worker2 # [ 121.357679] nix-grpc-daemon[1565]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 2/5)5533worker2 # [ 121.377090] nix-grpc-daemon[1565]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 3/5)5534worker2 # [ 121.396213] nix-grpc-daemon[1565]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 319 ms (attempt 4/5)5535worker2 # [ 121.724952] nix-grpc-daemon[1565]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5536worker2 # [ 121.732883] nix-grpc-daemon[1565]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5537worker2 # [ 121.750144] nix-grpc-daemon[1565]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5538worker2 # [ 121.782124] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5539client # [ 122.743656] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5540worker2 # [ 121.804712] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5541lb # [ 123.769029] envoy[915]: conn peer=[2001:db8:1::1]:36466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1430 ms=86035542lb # [ 123.774472] envoy[915]: conn peer=[2001:db8:1::4]:50624 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=38479 tx=2215 ms=292265543lb # [ 123.774644] envoy[915]: conn peer=[2001:db8:1::3]:36478 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1476 tx=539 ms=167275544client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5545client: waiting for success: ! systemctl is-active mid5546worker2 # [ 126.240221] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:42Z level=info event=niks3_push_started pid=16715547worker2 # [ 126.284811] nix-daemon[864]: reaped child process 1646, status = succeeded5548worker2 # [ 126.467525] nix-grpc-daemon[1671]: time=2026-09-21T12:55:42.760Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5549worker2 # [ 126.651055] nix-daemon[864]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)5550worker2 # [ 126.671489] nix-daemon[864]: reaped child process 1687, status = succeeded5551lb # [ 127.614066] niks3-server[1026]: 2026/09/21 12:55:42 INFO Received uploads request method=POST path=/api/pending_closures5552worker2 # [ 126.691369] nix-grpc-daemon[1671]: time=2026-09-21T12:55:42.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5553worker2 # [ 126.695028] nix-grpc-daemon[1671]: time=2026-09-21T12:55:42.982Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5554lb # [ 127.646179] niks3-server[1026]: 2026/09/21 12:55:42 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5555lb # [ 127.657702] niks3-server[1026]: 2026/09/21 12:55:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5556worker2 # [ 126.732172] nix-grpc-daemon[1671]: time=2026-09-21T12:55:43.024Z level=INFO msg="Uploading 1 narinfos"5557lb # [ 127.664324] niks3-server[1026]: 2026/09/21 12:55:42 INFO Signed narinfos id=36 count=15558lb # [ 127.674080] niks3-server[1026]: 2026/09/21 12:55:42 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5559lb # [ 127.682796] niks3-server[1026]: 2026/09/21 12:55:42 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5560lb # [ 127.688804] niks3-server[1026]: 2026/09/21 12:55:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5561worker2 # [ 126.761800] nix-grpc-daemon[1671]: time=2026-09-21T12:55:43.055Z level=INFO msg="Upload complete. (289ms)"5562lb # [ 127.692203] niks3-server[1026]: 2026/09/21 12:55:42 INFO Completed upload id=365563worker2 # [ 126.768206] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34540 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15564lb # [ 127.707256] niks3-server[1026]: 2026/09/21 12:55:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5565worker2 # [ 126.779663] nix-daemon[864]: reaped child process 1644, status = succeeded5566client # [ 127.753784] systemd[1]: mid.service: Deactivated successfully.5567client # [ 127.757147] systemd[1]: mid.service: Consumed 192ms CPU time over 10.160s wall clock time, 41.2M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5568client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)5569client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5570client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5571(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.59 seconds)5572subtest: interrupting the client stops the build on the worker5573client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5574client # [ 128.161531] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5575client # Running as unit: intr.service; invocation ID: 1b4b1b11d7a0420d949f04be7cae53455576client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5577worker2 # [ 127.485785] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345405578client # [ 128.436368] nix[1240]: don't know how to build these paths:5579client # [ 128.438700] nix[1240]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5580worker1 # [ 128.065083] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:43Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05581worker1 # [ 128.065967] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:43Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15582worker1 # [ 128.124105] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305583worker1 # [ 128.133365] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5584worker1 # [ 128.148955] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)5585worker1 # [ 128.212543] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 2/5)5586worker1 # [ 128.395913] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 292 ms (attempt 3/5)5587worker1 # [ 128.695868] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 709 ms (attempt 4/5)5588worker1 # [ 129.408398] nix-grpc-daemon[1296]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5589worker1 # [ 129.408841] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5590worker1 # [ 129.436403] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305591client # [ 129.845080] nix[1240]: copying 1 paths to 'lb:50051'...5592worker1 # [ 129.454762] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538305593client # [ 129.845439] nix[1240]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5594worker1 # [ 129.471331] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5595worker1 # [ 129.503814] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:44Z level=info event=niks3_push_started pid=13765596worker1 # [ 129.722311] nix-grpc-daemon[1376]: time=2026-09-21T12:55:44.913Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5597worker1 # [ 129.902926] nix-daemon[837]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5598worker1 # [ 129.921806] nix-daemon[837]: reaped child process 1392, status = succeeded5599worker1 # [ 130.104904] nix-daemon[837]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5600worker1 # [ 130.134263] nix-daemon[837]: reaped child process 1399, status = succeeded5601lb # [ 130.525275] niks3-server[1026]: 2026/09/21 12:55:45 INFO Received uploads request method=POST path=/api/pending_closures5602worker1 # [ 130.166789] nix-grpc-daemon[1376]: time=2026-09-21T12:55:45.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5603worker1 # [ 130.168785] nix-grpc-daemon[1376]: time=2026-09-21T12:55:45.357Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5604lb # [ 130.575334] niks3-server[1026]: 2026/09/21 12:55:45 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5605worker1 # [ 130.215811] nix-grpc-daemon[1376]: time=2026-09-21T12:55:45.407Z level=INFO msg="Uploading 1 narinfos"5606lb # [ 130.584930] niks3-server[1026]: 2026/09/21 12:55:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5607lb # [ 130.597665] niks3-server[1026]: 2026/09/21 12:55:45 INFO Signed narinfos id=37 count=15608lb # [ 130.606817] niks3-server[1026]: 2026/09/21 12:55:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5609lb # [ 130.616922] niks3-server[1026]: 2026/09/21 12:55:45 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5610lb # [ 130.626865] niks3-server[1026]: 2026/09/21 12:55:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5611lb # [ 130.633742] niks3-server[1026]: 2026/09/21 12:55:45 INFO Completed upload id=375612worker1 # [ 130.258129] nix-grpc-daemon[1376]: time=2026-09-21T12:55:45.450Z level=INFO msg="Upload complete. (531ms)"5613worker1 # [ 130.268345] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=4245614worker1 # [ 130.284814] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305615worker1 # [ 130.289250] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5616worker1 # [ 130.296202] nix-daemon[837]: reaped child process 1374, status = succeeded5617client # [ 130.693391] nix[1240]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5618worker1 # [ 130.308261] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5619client: must succeed: systemctl kill -s INT intr5620worker1 # [ 132.630142] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:47Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5621client # [ 133.018934] nix[1240]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5622client # [ 133.029127] systemd[1]: intr.service: Sent signal SIGINT to main process 1240 (nix) on client request.5623client # [ 133.039245] nix[1240]: error: interrupted by the user5624client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5625client # [ 133.083896] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5626client # [ 133.089769] systemd[1]: intr.service: Failed with result 'exit-code'.5627client # [ 133.093512] systemd[1]: intr.service: Consumed 197ms CPU time over 4.926s wall clock time, 41.4M memory peak, 6.3K incoming IP traffic, 8.3K outgoing IP traffic.5628worker1 # [ 132.741073] nix-daemon[1405]: unexpected Nix daemon error: error: interrupted by the user5629worker1 # [ 132.747395] nix-daemon[837]: reaped child process 1403, status = succeeded5630worker1 # [ 132.752319] nix-daemon[837]: reaped child process 1405, status = failed with exit code 15631lb # [ 133.770413] envoy[915]: conn peer=[2001:db8:1::1]:39398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1853 ms=98745632lb # [ 133.776699] envoy[915]: conn peer=[2001:db8:1::1]:36056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=850 ms=46225633(finished: subtest: interrupting the client stops the build on the worker, in 6.18 seconds)5634subtest: a deploy mid-build drains: the build finishes, the new generation takes over5635client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5636client # [ 134.373555] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5637client # Running as unit: sw.service; invocation ID: ac5fb2b2c9914e1b916c7a1e6d3c59005638client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5639client # [ 134.634446] nix[1261]: don't know how to build these paths:5640client # [ 134.635650] nix[1261]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5641worker1 # [ 134.244748] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538305642worker1 # [ 134.259008] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:49Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05643worker1 # [ 134.264164] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:49Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15644worker2 # [ 133.753987] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345405645worker2 # [ 133.775096] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345405646client # [ 134.735113] nix[1261]: copying 1 paths to 'lb:50051'...5647client # [ 134.736829] nix[1261]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5648worker2 # [ 133.793661] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345405649worker2 # [ 133.811900] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5650worker2 # [ 134.025304] nix-daemon[864]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)5651worker2 # [ 134.047451] nix-daemon[864]: reaped child process 1726, status = succeeded5652worker2 # [ 134.244357] nix-daemon[864]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)5653worker2 # [ 134.267474] nix-daemon[864]: reaped child process 1731, status = succeeded5654lb # [ 135.213177] niks3-server[1026]: 2026/09/21 12:55:50 INFO Received uploads request method=POST path=/api/pending_closures5655worker2 # [ 134.293120] nix-grpc-daemon[1671]: time=2026-09-21T12:55:50.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5656worker2 # [ 134.294741] nix-grpc-daemon[1671]: time=2026-09-21T12:55:50.585Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5657lb # [ 135.242628] niks3-server[1026]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5658worker2 # [ 134.318169] nix-grpc-daemon[1671]: time=2026-09-21T12:55:50.611Z level=INFO msg="Uploading 1 narinfos"5659lb # [ 135.250702] niks3-server[1026]: 2026/09/21 12:55:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5660lb # [ 135.256269] niks3-server[1026]: 2026/09/21 12:55:50 INFO Signed narinfos id=38 count=15661lb # [ 135.258006] niks3-server[1026]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5662lb # [ 135.265885] niks3-server[1026]: 2026/09/21 12:55:50 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5663worker2 # [ 134.342439] nix-grpc-daemon[1671]: time=2026-09-21T12:55:50.635Z level=INFO msg="Upload complete. (510ms)"5664lb # [ 135.272915] niks3-server[1026]: 2026/09/21 12:55:50 INFO Completed upload id=385665lb # [ 135.276193] niks3-server[1026]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5666worker2 # [ 134.348327] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34540 duration_s=0 paths=1 nar_bytes_in=4165667worker2 # [ 134.356909] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345405668worker2 # [ 134.366869] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5669worker2 # [ 134.371085] nix-daemon[864]: reaped child process 1721, status = succeeded5670client # [ 135.323812] nix[1261]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5671worker2 # [ 134.385205] nix-daemon[864]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5672worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25673worker2 # Checking switch inhibitors... done5674worker2 # [ 137.013479] nixos[1767]: switching to system configuration /nix/store/n5ldl2n7ssxrifcgvi8l27ajdvbwnl5r-nixos-system-worker2-test5675worker2 # activating the configuration...5676worker2 # [ 137.018837] systemd[1]: Stopped target Local File Systems.5677worker2 # [ 137.019799] systemd[1]: Stopped target Remote File Systems.5678worker2 # setting up /etc...5679worker2 # [ 137.855459] systemd[1]: Reload requested from client PID 1767 ('.switch-to-conf') (unit backdoor.service)...5680worker2 # [ 137.857618] systemd[1]: Reloading...5681worker2 # [ 138.834455] systemd[1]: Reloading finished in 974 ms.5682worker2 # restarting sysinit-reactivation.target5683worker2 # [ 138.925552] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5684worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5685worker2 # [ 139.085718] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5686worker2 # [ 139.086956] systemd[1]: Reached target Reactivate sysinit units.5687worker2 # [ 139.087779] systemd[1]: Reloading D-Bus System Message Bus...5688worker2 # [ 139.094340] dbus-broker-launch[503]: Caught SIGHUP, trigger reload.5689worker2 # [ 139.097981] systemd[1]: Reloading Nix worker-protocol over gRPC...5690worker2 # [ 139.126716] dbus-broker-launch[503]: Looking up NSS user entry for 'systemd-timesync'...5691worker2 # [ 139.135979] dbus-broker-launch[503]: NSS returned no entry for 'systemd-timesync'5692worker2 # [ 139.140137] dbus-broker-launch[503]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5693worker2 # [ 139.152350] systemd[1]: Reloaded Nix worker-protocol over gRPC.5694worker2 # [ 139.187087] systemd[1]: Reloaded D-Bus System Message Bus.5695worker2 # [ 139.197060] systemd[1]: Reached target Remote File Systems.5696worker2 # [ 139.201835] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5697worker2 # [ 139.203148] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5698worker2 # [ 139.213647] systemd[1]: Starting linger-users.service...5699worker2 # [ 139.219057] systemd[1]: Reached target Local File Systems.5700worker2 # [ 139.219958] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5701worker2 # [ 139.226358] systemd[1]: Starting Create SUID/SGID Wrappers...5702worker2 # [ 139.233491] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5703worker2 # [ 139.234963] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675704worker2 # [ 139.244321] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5705worker2 # [ 139.246002] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5706worker2 # [ 139.247234] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5707worker2 # [ 139.296847] systemd[1]: linger-users.service: Deactivated successfully.5708worker2 # [ 139.300083] systemd[1]: Finished linger-users.service.5709worker2 # [ 139.481560] nix-grpc-daemon[1565]: ts=2026-09-21T12:55:55Z level=info event=draining5710worker2 # [ 139.788743] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5711worker2 # [ 139.796309] systemd[1]: Finished Create SUID/SGID Wrappers.5712worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5713worker2 # [ 140.047406] nixos[1767]: finished switching to system configuration /nix/store/n5ldl2n7ssxrifcgvi8l27ajdvbwnl5r-nixos-system-worker2-test5714worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.93 seconds)5715worker2: must succeed: pgrep -f 'read -t [9]0 x'5716worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.07 seconds)5717client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25718client # don't know how to build these paths:5719client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5720worker1 # [ 140.935991] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538305721worker1 # [ 140.943438] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305722worker1 # [ 140.973556] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05723worker1 # [ 140.981566] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15724worker1 # [ 141.020932] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305725worker1 # [ 141.037839] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305726client # copying 1 paths to 'lb:50051'...5727worker1 # [ 141.056610] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538305728client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5729worker1 # [ 141.081973] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5730worker1 # [ 141.303048] nix-daemon[837]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5731worker1 # [ 141.324864] nix-daemon[837]: reaped child process 1463, status = succeeded5732worker1 # [ 141.507255] nix-daemon[837]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5733worker1 # [ 141.526061] nix-daemon[837]: reaped child process 1468, status = succeeded5734lb # [ 141.916816] niks3-server[1026]: 2026/09/21 12:55:57 INFO Received uploads request method=POST path=/api/pending_closures5735worker1 # [ 141.552833] nix-grpc-daemon[1376]: time=2026-09-21T12:55:56.744Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5736worker1 # [ 141.554426] nix-grpc-daemon[1376]: time=2026-09-21T12:55:56.744Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5737lb # [ 141.949286] niks3-server[1026]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5738worker1 # [ 141.588611] nix-grpc-daemon[1376]: time=2026-09-21T12:55:56.780Z level=INFO msg="Uploading 1 narinfos"5739lb # [ 141.964512] niks3-server[1026]: 2026/09/21 12:55:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5740lb # [ 141.974863] niks3-server[1026]: 2026/09/21 12:55:57 INFO Signed narinfos id=39 count=15741lb # [ 141.981101] niks3-server[1026]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5742lb # [ 141.990679] niks3-server[1026]: 2026/09/21 12:55:57 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5743lb # [ 141.999458] niks3-server[1026]: 2026/09/21 12:55:57 INFO Completed upload id=395744worker1 # [ 141.622797] nix-grpc-daemon[1376]: time=2026-09-21T12:55:56.814Z level=INFO msg="Upload complete. (521ms)"5745lb # [ 142.002350] niks3-server[1026]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5746worker1 # [ 141.632439] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=5285747worker1 # [ 141.648742] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305748worker1 # [ 141.656544] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5749worker1 # [ 141.662065] nix-daemon[837]: reaped child process 1458, status = succeeded5750client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5751worker1 # [ 141.676306] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5752client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5753client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5754client # farm-a> LOG-farm-a-during-drain5755worker1 # [ 142.504808] nix-daemon[837]: reaped child process 1472, status = succeeded5756worker1 # [ 142.677607] nix-daemon[837]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5757worker1 # [ 142.696323] nix-daemon[837]: reaped child process 1491, status = succeeded5758lb # [ 143.088808] niks3-server[1026]: 2026/09/21 12:55:58 INFO Received uploads request method=POST path=/api/pending_closures5759worker1 # [ 142.725987] nix-grpc-daemon[1376]: time=2026-09-21T12:55:57.916Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5760worker1 # [ 142.727572] nix-grpc-daemon[1376]: time=2026-09-21T12:55:57.916Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5761lb # [ 143.142006] niks3-server[1026]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5762lb # [ 143.153974] niks3-server[1026]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5763lb # [ 143.162601] niks3-server[1026]: 2026/09/21 12:55:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5764lb # [ 143.168731] niks3-server[1026]: 2026/09/21 12:55:58 INFO Signed narinfos id=40 count=15765worker1 # [ 142.793963] nix-grpc-daemon[1376]: time=2026-09-21T12:55:57.984Z level=INFO msg="Uploading 1 narinfos"5766lb # [ 143.172798] niks3-server[1026]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5767lb # [ 143.186048] niks3-server[1026]: 2026/09/21 12:55:58 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5768lb # [ 143.192495] niks3-server[1026]: 2026/09/21 12:55:58 INFO Completed upload id=405769worker1 # [ 142.815770] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.007Z level=INFO msg="Upload complete. (327ms)"5770lb # [ 143.195504] niks3-server[1026]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5771worker1 # [ 142.833686] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15772worker1 # [ 142.851282] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05773worker1 # [ 142.855708] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15774worker1 # [ 142.860507] nix-daemon[837]: reaped child process 1470, status = succeeded5775worker1 # [ 142.881095] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305776worker1 # [ 142.897766] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305777client # copying 1 paths to 'lb:50051'...5778worker1 # [ 142.912469] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.104Z level=INFO msg="All 1 paths already cached"5779worker1 # [ 142.919514] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538305780client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5781worker1 # [ 142.938423] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5782worker1 # [ 143.128180] nix-daemon[837]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)5783worker1 # [ 143.150104] nix-daemon[837]: reaped child process 1500, status = succeeded5784worker1 # [ 143.339773] nix-daemon[837]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)5785worker1 # [ 143.357130] nix-daemon[837]: reaped child process 1505, status = succeeded5786lb # [ 143.750853] niks3-server[1026]: 2026/09/21 12:55:59 INFO Received uploads request method=POST path=/api/pending_closures5787worker1 # [ 143.386282] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.577Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5788worker1 # [ 143.388075] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.577Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5789lb # [ 143.785344] niks3-server[1026]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5790worker1 # [ 143.425451] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.617Z level=INFO msg="Uploading 1 narinfos"5791lb # [ 143.801895] niks3-server[1026]: 2026/09/21 12:55:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5792lb # [ 143.815323] niks3-server[1026]: 2026/09/21 12:55:59 INFO Signed narinfos id=41 count=15793lb # [ 143.823277] niks3-server[1026]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5794lb # [ 143.836684] niks3-server[1026]: 2026/09/21 12:55:59 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5795lb # [ 143.843471] niks3-server[1026]: 2026/09/21 12:55:59 INFO Completed upload id=415796worker1 # [ 143.467627] nix-grpc-daemon[1376]: time=2026-09-21T12:55:58.659Z level=INFO msg="Upload complete. (521ms)"5797lb # [ 143.846603] niks3-server[1026]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5798worker1 # [ 143.478811] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=6405799worker1 # [ 143.494021] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305800worker1 # [ 143.500943] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5801worker1 # [ 143.506846] nix-daemon[837]: reaped child process 1495, status = succeeded5802client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5803client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5804worker1 # [ 143.521537] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5805client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5806client # farm-b> LOG-farm-b-during-drain5807worker1 # [ 144.366588] nix-daemon[837]: reaped child process 1509, status = succeeded5808worker1 # [ 144.568228] nix-daemon[837]: accepted connection from pid 1525, user nix-grpc-daemon (trusted)5809worker1 # [ 144.589630] nix-daemon[837]: reaped child process 1528, status = succeeded5810lb # [ 144.979464] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received uploads request method=POST path=/api/pending_closures5811worker1 # [ 144.614836] nix-grpc-daemon[1376]: time=2026-09-21T12:55:59.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5812worker1 # [ 144.616505] nix-grpc-daemon[1376]: time=2026-09-21T12:55:59.806Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5813lb # [ 145.028552] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5814worker1 # [ 144.661802] nix-grpc-daemon[1376]: time=2026-09-21T12:55:59.853Z level=INFO msg="Uploading 1 narinfos"5815lb # [ 145.039271] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5816lb # [ 145.047181] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5817lb # [ 145.058645] niks3-server[1026]: 2026/09/21 12:56:00 INFO Signed narinfos id=42 count=15818worker1 # [ 144.685151] nix-grpc-daemon[1376]: time=2026-09-21T12:55:59.877Z level=INFO msg="Upload complete. (352ms)"5819lb # [ 145.062276] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5820lb # [ 145.065214] niks3-server[1026]: 2026/09/21 12:56:00 INFO Completed upload id=425821lb # [ 145.068747] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5822lb # [ 145.072579] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5823worker1 # [ 144.698886] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15824worker1 # [ 144.714976] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05825worker1 # [ 144.719145] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15826worker1 # [ 144.723561] nix-daemon[837]: reaped child process 1507, status = succeeded5827worker1 # [ 144.756445] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305828worker1 # [ 144.772352] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305829worker1 # [ 144.788134] nix-grpc-daemon[1376]: time=2026-09-21T12:55:59.980Z level=INFO msg="All 2 paths already cached"5830client # copying 1 paths to 'lb:50051'...5831client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5832worker1 # [ 144.799261] nix-grpc-daemon[1296]: ts=2026-09-21T12:55:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538305833worker1 # [ 144.813858] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5834worker1 # [ 145.018410] nix-daemon[837]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)5835worker1 # [ 145.040288] nix-daemon[837]: reaped child process 1537, status = succeeded5836worker1 # [ 145.219743] nix-daemon[837]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)5837worker1 # [ 145.240419] nix-daemon[837]: reaped child process 1542, status = succeeded5838lb # [ 145.627230] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received uploads request method=POST path=/api/pending_closures5839worker1 # [ 145.261070] nix-grpc-daemon[1376]: time=2026-09-21T12:56:00.452Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5840worker1 # [ 145.262589] nix-grpc-daemon[1376]: time=2026-09-21T12:56:00.453Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5841lb # [ 145.657575] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5842worker1 # [ 145.293924] nix-grpc-daemon[1376]: time=2026-09-21T12:56:00.485Z level=INFO msg="Uploading 1 narinfos"5843lb # [ 145.674450] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5844lb # [ 145.685059] niks3-server[1026]: 2026/09/21 12:56:00 INFO Signed narinfos id=43 count=15845lb # [ 145.688929] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5846lb # [ 145.696500] niks3-server[1026]: 2026/09/21 12:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5847lb # [ 145.705989] niks3-server[1026]: 2026/09/21 12:56:00 INFO Completed upload id=435848worker1 # [ 145.330082] nix-grpc-daemon[1376]: time=2026-09-21T12:56:00.522Z level=INFO msg="Upload complete. (500ms)"5849lb # [ 145.709074] niks3-server[1026]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5850worker1 # [ 145.340419] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=7685851worker1 # [ 145.350841] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305852worker1 # [ 145.358033] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5853worker1 # [ 145.362694] nix-daemon[837]: reaped child process 1532, status = succeeded5854client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5855worker1 # [ 145.373745] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5856client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5857client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5858client # farm-top> LOG-farm-top-during-drain5859worker1 # [ 146.210679] nix-daemon[837]: reaped child process 1546, status = succeeded5860worker1 # [ 146.378099] nix-daemon[837]: accepted connection from pid 1562, user nix-grpc-daemon (trusted)5861worker1 # [ 146.399823] nix-daemon[837]: reaped child process 1565, status = succeeded5862lb # [ 146.789329] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received uploads request method=POST path=/api/pending_closures5863worker1 # [ 146.428484] nix-grpc-daemon[1376]: time=2026-09-21T12:56:01.619Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5864worker1 # [ 146.430025] nix-grpc-daemon[1376]: time=2026-09-21T12:56:01.619Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5865lb # [ 146.836345] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5866lb # [ 146.851018] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5867lb # [ 146.862107] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5868worker1 # [ 146.492716] nix-grpc-daemon[1376]: time=2026-09-21T12:56:01.684Z level=INFO msg="Uploading 1 narinfos"5869lb # [ 146.875468] niks3-server[1026]: 2026/09/21 12:56:02 INFO Signed narinfos id=44 count=15870lb # [ 146.881633] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5871lb # [ 146.892115] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5872lb # [ 146.897950] niks3-server[1026]: 2026/09/21 12:56:02 INFO Completed upload id=445873worker1 # [ 146.522204] nix-grpc-daemon[1376]: time=2026-09-21T12:56:01.714Z level=INFO msg="Upload complete. (336ms)"5874lb # [ 146.901127] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5875worker1 # [ 146.537349] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15876worker1 # [ 146.558635] nix-daemon[837]: reaped child process 1544, status = succeeded5877client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 5.91 seconds)5878client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5879client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.25 seconds)5880worker2: must succeed: pkill -f 'read -t [9]0 x'5881worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.07 seconds)5882client: waiting for success: ! systemctl is-active sw5883worker2 # [ 146.431476] nix-daemon[864]: reaped child process 1735, status = succeeded5884worker2 # [ 146.597390] nix-daemon[864]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)5885worker2 # [ 146.619799] nix-daemon[864]: reaped child process 1910, status = succeeded5886lb # [ 147.561961] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received uploads request method=POST path=/api/pending_closures5887worker2 # [ 146.643198] nix-grpc-daemon[1671]: time=2026-09-21T12:56:02.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5888worker2 # [ 146.644973] nix-grpc-daemon[1671]: time=2026-09-21T12:56:02.935Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5889lb # [ 147.593615] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5890lb # [ 147.608555] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5891worker2 # [ 146.694969] nix-grpc-daemon[1671]: time=2026-09-21T12:56:02.987Z level=INFO msg="Uploading 1 narinfos"5892lb # [ 147.623842] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5893lb # [ 147.634463] niks3-server[1026]: 2026/09/21 12:56:02 INFO Signed narinfos id=45 count=15894lb # [ 147.639698] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5895lb # [ 147.648208] niks3-server[1026]: 2026/09/21 12:56:02 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5896lb # [ 147.655329] niks3-server[1026]: 2026/09/21 12:56:02 INFO Completed upload id=455897worker2 # [ 146.725685] nix-grpc-daemon[1671]: time=2026-09-21T12:56:03.019Z level=INFO msg="Upload complete. (318ms)"5898lb # [ 147.659841] niks3-server[1026]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5899worker2 # [ 146.736374] nix-grpc-daemon[1565]: ts=2026-09-21T12:56:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34540 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15900worker2 # [ 146.748679] nix-daemon[864]: reaped child process 1733, status = succeeded5901client # [ 147.725682] systemd[1]: sw.service: Deactivated successfully.5902client # [ 147.730155] systemd[1]: sw.service: Consumed 193ms CPU time over 13.356s wall clock time, 41.3M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5903client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5904worker2 # [ 147.484186] nix-grpc-daemon[1565]: ts=2026-09-21T12:56:03Z level=info event=signal_exit5905client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5906client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5907worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5908worker2 # [ 147.647006] systemd[1]: Starting scheduler-token.service...5909worker2 # [ 147.753445] systemd[1]: scheduler-token.service: Deactivated successfully.5910worker2 # [ 147.758444] systemd[1]: Finished scheduler-token.service.5911worker2 # [ 147.761180] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 110ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5912worker1 # [ 149.262486] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:537905913worker2 # [ 148.710431] nix-grpc-daemon[1565]: ts=2026-09-21T12:56:04Z level=info event=scheduler_restarting5914worker1 # [ 149.268317] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:04Z level=info event=worker_gone addr=192.168.1.4:500515915worker2 # [ 148.737228] nix-grpc-daemon[1565]: ts=2026-09-21T12:56:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5916worker2 # [ 148.769880] nix-daemon[864]: reaped child process 1637, status = succeeded5917worker2 # [ 149.205648] nix-grpc-daemon[1565]: E0000 00:00:1789995365.498836 1610 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5918worker2 # [ 149.212721] nix-grpc-daemon[1565]: E0000 00:00:1789995365.499092 1610 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5919worker2 # [ 150.517862] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5920worker2 # [ 150.521180] systemd[1]: nix-grpc-daemon.service: Consumed 972ms CPU time over 34.248s wall clock time, 33.3M memory peak, 20.8K incoming IP traffic, 28.6K outgoing IP traffic.5921worker2 # [ 150.544670] systemd[1]: Starting Nix worker-protocol over gRPC...5922worker2 # [ 150.762352] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5923worker2 # [ 150.765219] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5924worker2 # [ 150.769939] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15925worker2 # [ 150.779171] nix-grpc-daemon[1927]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5926worker2 # [ 150.790197] nix-grpc-daemon[1927]: E0000 00:00:1789995367.070210 1927 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5927worker2 # [ 150.794508] nix-grpc-daemon[1927]: E0000 00:00:1789995367.072508 1927 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5928worker2 # [ 150.798791] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5929worker2 # [ 150.812270] systemd[1]: Started Nix worker-protocol over gRPC.5930worker2 # [ 150.813305] nix-grpc-daemon[1927]: E0000 00:00:1789995367.096048 1969 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5931worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.30 seconds)5932lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5933lb: must succeed: curl -sf localhost:9901/clusters5934worker2 # [ 150.841868] nix-grpc-daemon[1927]: E0000 00:00:1789995367.116599 1969 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5935worker2 # [ 150.845239] nix-grpc-daemon[1927]: E0000 00:00:1789995367.117814 1969 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5936worker2 # [ 150.847919] nix-grpc-daemon[1927]: E0000 00:00:1789995367.133059 1969 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5937worker2 # [ 150.860474] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:07Z level=info event=scheduler_connected addr=lb:500515938worker1 # [ 151.419127] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:537905939worker1 # [ 151.427660] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5941lb: must succeed: curl -sf localhost:9901/clusters5942lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)5943lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.20 seconds)5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5946worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5947worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5948(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.84 seconds)5949subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5950client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15951worker1 # [ 153.047259] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538305952worker1 # [ 153.059396] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305953worker1 # [ 153.119486] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05954worker1 # [ 153.125990] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15955worker1 # [ 153.132274] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05956worker1 # [ 153.137607] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05957worker1 # [ 153.154008] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305958worker1 # [ 153.175649] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538305959worker1 # [ 153.196000] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538305960worker1 # [ 153.214769] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5961lb # [ 153.770864] envoy[915]: conn peer=[2001:db8:1::1]:42578 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4074 ms=56215962worker1 # [ 153.432694] nix-daemon[837]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)5963worker1 # [ 153.458593] nix-daemon[837]: reaped child process 1577, status = succeeded5964worker1 # [ 153.665868] nix-daemon[837]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5965worker1 # [ 153.689330] nix-daemon[837]: reaped child process 1582, status = succeeded5966lb # [ 153.776412] envoy[915]: conn peer=[2001:db8:1::1]:33768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1431 ms=130645967lb # [ 153.776559] envoy[915]: conn peer=[2001:db8:1::4]:59500 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2238 tx=448 ms=321875968lb # [ 154.082130] niks3-server[1026]: 2026/09/21 12:56:09 INFO Received uploads request method=POST path=/api/pending_closures5969worker1 # [ 153.730468] nix-grpc-daemon[1376]: time=2026-09-21T12:56:08.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5970worker1 # [ 153.731997] nix-grpc-daemon[1376]: time=2026-09-21T12:56:08.921Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5971lb # [ 154.130220] niks3-server[1026]: 2026/09/21 12:56:09 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5972lb # [ 154.147651] niks3-server[1026]: 2026/09/21 12:56:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5973worker1 # [ 153.777759] nix-grpc-daemon[1376]: time=2026-09-21T12:56:08.969Z level=INFO msg="Uploading 1 narinfos"5974lb # [ 154.156378] niks3-server[1026]: 2026/09/21 12:56:09 INFO Signed narinfos id=46 count=15975lb # [ 154.163989] niks3-server[1026]: 2026/09/21 12:56:09 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5976lb # [ 154.176420] niks3-server[1026]: 2026/09/21 12:56:09 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5977lb # [ 154.183220] niks3-server[1026]: 2026/09/21 12:56:09 INFO Completed upload id=465978worker1 # [ 153.806972] nix-grpc-daemon[1376]: time=2026-09-21T12:56:08.999Z level=INFO msg="Upload complete. (573ms)"5979lb # [ 154.186332] niks3-server[1026]: 2026/09/21 12:56:09 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5980worker1 # [ 153.816391] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=5125981worker1 # [ 153.832213] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538305982worker1 # [ 153.839308] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5983worker1 # [ 153.845866] nix-daemon[837]: reaped child process 1572, status = succeeded5984worker1 # [ 153.862737] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5985worker1 # [ 154.489401] nix-daemon[837]: reaped child process 1586, status = succeeded5986worker1 # [ 154.668134] nix-daemon[837]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)5987worker1 # [ 154.693068] nix-daemon[837]: reaped child process 1605, status = succeeded5988lb # [ 155.083117] niks3-server[1026]: 2026/09/21 12:56:10 INFO Received uploads request method=POST path=/api/pending_closures5989worker1 # [ 154.720163] nix-grpc-daemon[1376]: time=2026-09-21T12:56:09.910Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5990worker1 # [ 154.721833] nix-grpc-daemon[1376]: time=2026-09-21T12:56:09.910Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5991lb # [ 155.126905] niks3-server[1026]: 2026/09/21 12:56:10 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5992lb # [ 155.143218] niks3-server[1026]: 2026/09/21 12:56:10 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5993lb # [ 155.158841] niks3-server[1026]: 2026/09/21 12:56:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5994worker1 # [ 154.789730] nix-grpc-daemon[1376]: time=2026-09-21T12:56:09.980Z level=INFO msg="Uploading 1 narinfos"5995lb # [ 155.167379] niks3-server[1026]: 2026/09/21 12:56:10 INFO Signed narinfos id=47 count=15996lb # [ 155.172307] niks3-server[1026]: 2026/09/21 12:56:10 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5997lb # [ 155.183362] niks3-server[1026]: 2026/09/21 12:56:10 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5998lb # [ 155.190128] niks3-server[1026]: 2026/09/21 12:56:10 INFO Completed upload id=475999worker1 # [ 154.814540] nix-grpc-daemon[1376]: time=2026-09-21T12:56:10.006Z level=INFO msg="Upload complete. (339ms)"6000lb # [ 155.193329] niks3-server[1026]: 2026/09/21 12:56:10 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6001worker1 # [ 154.837478] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16002worker1 # [ 154.856616] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16003worker1 # [ 154.868492] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306004worker1 # [ 154.875836] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06005worker1 # [ 154.889243] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306006worker1 # [ 154.900252] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306007worker1 # [ 154.905619] nix-daemon[837]: reaped child process 1584, status = succeeded6008worker1 # [ 154.907342] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6009worker1 # [ 155.115549] nix-daemon[837]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6010worker1 # [ 155.140323] nix-daemon[837]: reaped child process 1614, status = succeeded6011worker1 # [ 155.346002] nix-daemon[837]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6012worker1 # [ 155.368773] nix-daemon[837]: reaped child process 1619, status = succeeded6013lb # [ 155.757775] niks3-server[1026]: 2026/09/21 12:56:11 INFO Received uploads request method=POST path=/api/pending_closures6014worker1 # [ 155.392153] nix-grpc-daemon[1376]: time=2026-09-21T12:56:10.584Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6015worker1 # [ 155.393936] nix-grpc-daemon[1376]: time=2026-09-21T12:56:10.584Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6016lb # [ 155.799246] niks3-server[1026]: 2026/09/21 12:56:11 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6017lb # [ 155.812613] niks3-server[1026]: 2026/09/21 12:56:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6018worker1 # [ 155.444377] nix-grpc-daemon[1376]: time=2026-09-21T12:56:10.636Z level=INFO msg="Uploading 1 narinfos"6019lb # [ 155.822499] niks3-server[1026]: 2026/09/21 12:56:11 INFO Signed narinfos id=48 count=16020lb # [ 155.829789] niks3-server[1026]: 2026/09/21 12:56:11 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6021lb # [ 155.842303] niks3-server[1026]: 2026/09/21 12:56:11 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6022lb # [ 155.849638] niks3-server[1026]: 2026/09/21 12:56:11 INFO Completed upload id=486023worker1 # [ 155.473889] nix-grpc-daemon[1376]: time=2026-09-21T12:56:10.666Z level=INFO msg="Upload complete. (562ms)"6024lb # [ 155.853181] niks3-server[1026]: 2026/09/21 12:56:11 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6025worker1 # [ 155.484402] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=5126026worker1 # [ 155.499547] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306027worker1 # [ 155.507370] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6028worker1 # [ 155.513550] nix-daemon[837]: reaped child process 1609, status = succeeded6029worker1 # [ 155.530823] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6030worker1 # [ 156.974849] nix-daemon[837]: reaped child process 1623, status = succeeded6031worker1 # [ 157.160493] nix-daemon[837]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6032worker1 # [ 157.182644] nix-daemon[837]: reaped child process 1642, status = succeeded6033lb # [ 157.570590] niks3-server[1026]: 2026/09/21 12:56:12 INFO Received uploads request method=POST path=/api/pending_closures6034worker1 # [ 157.228670] nix-grpc-daemon[1376]: time=2026-09-21T12:56:12.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6035worker1 # [ 157.230754] nix-grpc-daemon[1376]: time=2026-09-21T12:56:12.419Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6036lb # [ 157.649527] niks3-server[1026]: 2026/09/21 12:56:12 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6037worker1 # [ 157.286215] nix-grpc-daemon[1376]: time=2026-09-21T12:56:12.478Z level=INFO msg="Uploading 1 narinfos"6038lb # [ 157.664410] niks3-server[1026]: 2026/09/21 12:56:12 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6039lb # [ 157.675328] niks3-server[1026]: 2026/09/21 12:56:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6040lb # [ 157.682488] niks3-server[1026]: 2026/09/21 12:56:12 INFO Signed narinfos id=49 count=16041lb # [ 157.691166] niks3-server[1026]: 2026/09/21 12:56:12 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6042worker1 # [ 157.314810] nix-grpc-daemon[1376]: time=2026-09-21T12:56:12.506Z level=INFO msg="Upload complete. (370ms)"6043lb # [ 157.694677] niks3-server[1026]: 2026/09/21 12:56:12 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6044lb # [ 157.700332] niks3-server[1026]: 2026/09/21 12:56:12 INFO Completed upload id=496045lb # [ 157.703299] niks3-server[1026]: 2026/09/21 12:56:12 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6046worker1 # [ 157.336698] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16047worker1 # [ 157.355050] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16048worker1 # [ 157.360397] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06049worker1 # [ 157.363855] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306050worker1 # [ 157.368360] nix-daemon[837]: reaped child process 1621, status = succeeded6051worker1 # [ 157.379186] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306052worker1 # [ 157.395454] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306053worker1 # [ 157.409557] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6054worker1 # [ 157.624266] nix-daemon[837]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6055worker1 # [ 157.646664] nix-daemon[837]: reaped child process 1651, status = succeeded6056worker1 # [ 157.846402] nix-daemon[837]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)6057worker1 # [ 157.867484] nix-daemon[837]: reaped child process 1656, status = succeeded6058lb # [ 158.258144] niks3-server[1026]: 2026/09/21 12:56:13 INFO Received uploads request method=POST path=/api/pending_closures6059worker1 # [ 157.891862] nix-grpc-daemon[1376]: time=2026-09-21T12:56:13.083Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6060worker1 # [ 157.893757] nix-grpc-daemon[1376]: time=2026-09-21T12:56:13.083Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6061lb # [ 158.287651] niks3-server[1026]: 2026/09/21 12:56:13 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6062lb # [ 158.303027] niks3-server[1026]: 2026/09/21 12:56:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6063worker1 # [ 157.934837] nix-grpc-daemon[1376]: time=2026-09-21T12:56:13.126Z level=INFO msg="Uploading 1 narinfos"6064lb # [ 158.316268] niks3-server[1026]: 2026/09/21 12:56:13 INFO Signed narinfos id=50 count=16065lb # [ 158.319011] niks3-server[1026]: 2026/09/21 12:56:13 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6066lb # [ 158.330825] niks3-server[1026]: 2026/09/21 12:56:13 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6067lb # [ 158.337096] niks3-server[1026]: 2026/09/21 12:56:13 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6068lb # [ 158.342577] niks3-server[1026]: 2026/09/21 12:56:13 INFO Completed upload id=506069worker1 # [ 157.966760] nix-grpc-daemon[1376]: time=2026-09-21T12:56:13.158Z level=INFO msg="Upload complete. (542ms)"6070worker1 # [ 157.976252] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=5126071worker1 # [ 157.992486] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306072worker1 # [ 157.999497] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6073worker1 # [ 158.005567] nix-daemon[837]: reaped child process 1646, status = succeeded6074worker1 # [ 158.021081] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6075worker1 # [ 159.263841] nix-daemon[837]: reaped child process 1660, status = succeeded6076worker1 # [ 159.456212] nix-daemon[837]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6077worker1 # [ 159.477517] nix-daemon[837]: reaped child process 1679, status = succeeded6078lb # [ 159.869582] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received uploads request method=POST path=/api/pending_closures6079worker1 # [ 159.504620] nix-grpc-daemon[1376]: time=2026-09-21T12:56:14.696Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6080worker1 # [ 159.507766] nix-grpc-daemon[1376]: time=2026-09-21T12:56:14.698Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6081lb # [ 159.910605] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6082lb # [ 159.923562] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6083lb # [ 159.936407] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6084worker1 # [ 159.567304] nix-grpc-daemon[1376]: time=2026-09-21T12:56:14.758Z level=INFO msg="Uploading 1 narinfos"6085lb # [ 159.946864] niks3-server[1026]: 2026/09/21 12:56:15 INFO Signed narinfos id=51 count=16086lb # [ 159.951214] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6087lb # [ 159.959222] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6088lb # [ 159.965817] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6089lb # [ 159.969440] niks3-server[1026]: 2026/09/21 12:56:15 INFO Completed upload id=516090worker1 # [ 159.593725] nix-grpc-daemon[1376]: time=2026-09-21T12:56:14.785Z level=INFO msg="Upload complete. (360ms)"6091worker1 # [ 159.613534] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16092worker1 # [ 159.636662] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16093worker1 # [ 159.640445] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306094worker1 # [ 159.643326] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06095worker1 # [ 159.648310] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306096worker1 # [ 159.652767] nix-daemon[837]: reaped child process 1658, status = succeeded6097worker1 # [ 159.656817] nix-grpc-daemon[1376]: time=2026-09-21T12:56:14.848Z level=INFO msg="All 1 paths already cached"6098worker1 # [ 159.666478] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306099worker1 # [ 159.678360] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6100worker1 # [ 159.895969] nix-daemon[837]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6101worker1 # [ 159.920996] nix-daemon[837]: reaped child process 1688, status = succeeded6102worker1 # [ 160.116757] nix-daemon[837]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6103worker1 # [ 160.139549] nix-daemon[837]: reaped child process 1693, status = succeeded6104lb # [ 160.528285] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received uploads request method=POST path=/api/pending_closures6105worker1 # [ 160.175653] nix-grpc-daemon[1376]: time=2026-09-21T12:56:15.367Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6106worker1 # [ 160.177674] nix-grpc-daemon[1376]: time=2026-09-21T12:56:15.367Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6107lb # [ 160.578506] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6108worker1 # [ 160.213944] nix-grpc-daemon[1376]: time=2026-09-21T12:56:15.405Z level=INFO msg="Uploading 1 narinfos"6109lb # [ 160.595271] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6110lb # [ 160.603646] niks3-server[1026]: 2026/09/21 12:56:15 INFO Signed narinfos id=52 count=16111lb # [ 160.609445] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6112lb # [ 160.617953] niks3-server[1026]: 2026/09/21 12:56:15 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6113worker1 # [ 160.245841] nix-grpc-daemon[1376]: time=2026-09-21T12:56:15.437Z level=INFO msg="Upload complete. (551ms)"6114lb # [ 160.624639] niks3-server[1026]: 2026/09/21 12:56:15 INFO Completed upload id=526115lb # [ 160.629037] niks3-server[1026]: 2026/09/21 12:56:15 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6116worker1 # [ 160.256421] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=6246117worker1 # [ 160.274928] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306118worker1 # [ 160.283259] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6119worker1 # [ 160.290713] nix-daemon[837]: reaped child process 1683, status = succeeded6120worker1 # [ 160.308228] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6121worker1 # [ 161.359275] nix-daemon[837]: reaped child process 1697, status = succeeded6122worker1 # [ 161.563644] nix-daemon[837]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6123worker1 # [ 161.585804] nix-daemon[837]: reaped child process 1716, status = succeeded6124lb # [ 161.977224] niks3-server[1026]: 2026/09/21 12:56:17 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 161.620299] nix-grpc-daemon[1376]: time=2026-09-21T12:56:16.807Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6126worker1 # [ 161.622141] nix-grpc-daemon[1376]: time=2026-09-21T12:56:16.807Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6127lb # [ 162.036776] niks3-server[1026]: 2026/09/21 12:56:17 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6128worker1 # [ 161.672204] nix-grpc-daemon[1376]: time=2026-09-21T12:56:16.863Z level=INFO msg="Uploading 1 narinfos"6129lb # [ 162.049048] niks3-server[1026]: 2026/09/21 12:56:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6130lb # [ 162.058009] niks3-server[1026]: 2026/09/21 12:56:17 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6131lb # [ 162.065015] niks3-server[1026]: 2026/09/21 12:56:17 INFO Signed narinfos id=53 count=16132lb # [ 162.071769] niks3-server[1026]: 2026/09/21 12:56:17 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6133lb # [ 162.077051] niks3-server[1026]: 2026/09/21 12:56:17 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6134lb # [ 162.081401] niks3-server[1026]: 2026/09/21 12:56:17 INFO Completed upload id=536135worker1 # [ 161.700398] nix-grpc-daemon[1376]: time=2026-09-21T12:56:16.892Z level=INFO msg="Upload complete. (377ms)"6136lb # [ 162.082981] niks3-server[1026]: 2026/09/21 12:56:17 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6137worker1 # [ 161.719743] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16138worker1 # [ 161.737835] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16139worker1 # [ 161.743601] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06140worker1 # [ 161.747794] nix-daemon[837]: reaped child process 1695, status = succeeded6141worker1 # [ 161.761255] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306142worker1 # [ 161.778060] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306143worker1 # [ 161.794424] nix-grpc-daemon[1376]: time=2026-09-21T12:56:16.986Z level=INFO msg="All 1 paths already cached"6144worker1 # [ 161.805098] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306145worker1 # [ 161.819267] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6146worker1 # [ 162.031503] nix-daemon[837]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6147worker1 # [ 162.058591] nix-daemon[837]: reaped child process 1725, status = succeeded6148worker1 # [ 162.244842] nix-daemon[837]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6149worker1 # [ 162.267019] nix-daemon[837]: reaped child process 1730, status = succeeded6150lb # [ 162.655147] niks3-server[1026]: 2026/09/21 12:56:17 INFO Received uploads request method=POST path=/api/pending_closures6151worker1 # [ 162.291611] nix-grpc-daemon[1376]: time=2026-09-21T12:56:17.483Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6152worker1 # [ 162.293387] nix-grpc-daemon[1376]: time=2026-09-21T12:56:17.483Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6153lb # [ 162.694719] niks3-server[1026]: 2026/09/21 12:56:17 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6154lb # [ 162.712926] niks3-server[1026]: 2026/09/21 12:56:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6155worker1 # [ 162.342495] nix-grpc-daemon[1376]: time=2026-09-21T12:56:17.534Z level=INFO msg="Uploading 1 narinfos"6156lb # [ 162.725087] niks3-server[1026]: 2026/09/21 12:56:18 INFO Signed narinfos id=54 count=16157lb # [ 162.733002] niks3-server[1026]: 2026/09/21 12:56:18 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6158lb # [ 162.743138] niks3-server[1026]: 2026/09/21 12:56:18 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6159worker1 # [ 162.373178] nix-grpc-daemon[1376]: time=2026-09-21T12:56:17.565Z level=INFO msg="Upload complete. (538ms)"6160lb # [ 162.750215] niks3-server[1026]: 2026/09/21 12:56:18 INFO Completed upload id=546161lb # [ 162.753372] niks3-server[1026]: 2026/09/21 12:56:18 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6162worker1 # [ 162.384408] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=6246163worker1 # [ 162.400324] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306164worker1 # [ 162.408724] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6165worker1 # [ 162.414704] nix-daemon[837]: reaped child process 1720, status = succeeded6166worker1 # [ 162.428625] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6167worker1 # [ 163.465542] nix-daemon[837]: reaped child process 1734, status = succeeded6168worker1 # [ 163.672916] nix-daemon[837]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6169worker1 # [ 163.696487] nix-daemon[837]: reaped child process 1753, status = succeeded6170lb # [ 164.107547] niks3-server[1026]: 2026/09/21 12:56:19 INFO Received uploads request method=POST path=/api/pending_closures6171worker1 # [ 163.744668] nix-grpc-daemon[1376]: time=2026-09-21T12:56:18.935Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6172worker1 # [ 163.746455] nix-grpc-daemon[1376]: time=2026-09-21T12:56:18.935Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6173lb # [ 164.156826] niks3-server[1026]: 2026/09/21 12:56:19 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6174lb # [ 164.172282] niks3-server[1026]: 2026/09/21 12:56:19 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6175worker1 # [ 163.816356] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.008Z level=INFO msg="Uploading 1 narinfos"6176lb # [ 164.194488] niks3-server[1026]: 2026/09/21 12:56:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6177lb # [ 164.199321] niks3-server[1026]: 2026/09/21 12:56:19 INFO Signed narinfos id=55 count=16178lb # [ 164.202667] niks3-server[1026]: 2026/09/21 12:56:19 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6179lb # [ 164.209165] niks3-server[1026]: 2026/09/21 12:56:19 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6180lb # [ 164.215444] niks3-server[1026]: 2026/09/21 12:56:19 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6181lb # [ 164.219007] niks3-server[1026]: 2026/09/21 12:56:19 INFO Completed upload id=556182worker1 # [ 163.843227] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.035Z level=INFO msg="Upload complete. (399ms)"6183worker1 # [ 163.857568] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16184worker1 # [ 163.876374] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16185worker1 # [ 163.881519] nix-daemon[837]: reaped child process 1732, status = succeeded6186worker1 # [ 163.897601] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306187worker1 # [ 163.904857] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06188worker1 # [ 163.913340] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306189worker1 # [ 163.924888] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.117Z level=INFO msg="All 1 paths already cached"6190worker1 # [ 163.932361] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306191worker1 # [ 163.943408] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6192worker1 # [ 164.142508] nix-daemon[837]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6193worker1 # [ 164.163843] nix-daemon[837]: reaped child process 1762, status = succeeded6194worker1 # [ 164.348104] nix-daemon[837]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6195worker1 # [ 164.372896] nix-daemon[837]: reaped child process 1767, status = succeeded6196lb # [ 164.760426] niks3-server[1026]: 2026/09/21 12:56:20 INFO Received uploads request method=POST path=/api/pending_closures6197worker1 # [ 164.396730] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.588Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6198worker1 # [ 164.398356] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.588Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6199lb # [ 164.796049] niks3-server[1026]: 2026/09/21 12:56:20 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6200lb # [ 164.812449] niks3-server[1026]: 2026/09/21 12:56:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6201worker1 # [ 164.442120] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.634Z level=INFO msg="Uploading 1 narinfos"6202lb # [ 164.825116] niks3-server[1026]: 2026/09/21 12:56:20 INFO Signed narinfos id=56 count=16203lb # [ 164.832406] niks3-server[1026]: 2026/09/21 12:56:20 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6204lb # [ 164.841815] niks3-server[1026]: 2026/09/21 12:56:20 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6205lb # [ 164.849337] niks3-server[1026]: 2026/09/21 12:56:20 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6206lb # [ 164.854190] niks3-server[1026]: 2026/09/21 12:56:20 INFO Completed upload id=566207worker1 # [ 164.478693] nix-grpc-daemon[1376]: time=2026-09-21T12:56:19.670Z level=INFO msg="Upload complete. (521ms)"6208worker1 # [ 164.489005] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=6246209worker1 # [ 164.500880] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306210worker1 # [ 164.513664] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6211worker1 # [ 164.523962] nix-daemon[837]: reaped child process 1757, status = succeeded6212worker1 # [ 164.536546] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6213worker1 # [ 165.773897] nix-daemon[837]: reaped child process 1771, status = succeeded6214worker1 # [ 166.459357] nix-daemon[837]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6215worker1 # [ 166.484476] nix-daemon[837]: reaped child process 1790, status = succeeded6216lb # [ 166.889283] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received uploads request method=POST path=/api/pending_closures6217worker1 # [ 166.520560] nix-grpc-daemon[1376]: time=2026-09-21T12:56:21.712Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6218worker1 # [ 166.522230] nix-grpc-daemon[1376]: time=2026-09-21T12:56:21.712Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6219lb # [ 166.931014] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6220lb # [ 166.948166] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6221worker1 # [ 166.579234] nix-grpc-daemon[1376]: time=2026-09-21T12:56:21.771Z level=INFO msg="Uploading 1 narinfos"6222lb # [ 166.959135] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6223lb # [ 166.966997] niks3-server[1026]: 2026/09/21 12:56:22 INFO Signed narinfos id=57 count=16224lb # [ 166.972417] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6225lb # [ 166.980630] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6226lb # [ 166.985792] niks3-server[1026]: 2026/09/21 12:56:22 INFO Completed upload id=576227worker1 # [ 166.609794] nix-grpc-daemon[1376]: time=2026-09-21T12:56:21.801Z level=INFO msg="Upload complete. (860ms)"6228lb # [ 166.993743] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6229worker1 # [ 166.618291] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16230worker1 # [ 166.628781] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16231worker1 # [ 166.631550] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306232worker1 # [ 166.639129] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06233worker1 # [ 166.643206] nix-daemon[837]: reaped child process 1769, status = succeeded6234worker1 # [ 166.646788] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306235worker1 # [ 166.656354] nix-grpc-daemon[1376]: time=2026-09-21T12:56:21.848Z level=INFO msg="All 2 paths already cached"6236worker1 # [ 166.664131] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306237worker1 # [ 166.672551] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6238worker1 # [ 166.893721] nix-daemon[837]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6239worker1 # [ 166.923813] nix-daemon[837]: reaped child process 1799, status = succeeded6240worker1 # [ 167.128242] nix-daemon[837]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6241worker1 # [ 167.157392] nix-daemon[837]: reaped child process 1804, status = succeeded6242lb # [ 167.542270] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received uploads request method=POST path=/api/pending_closures6243worker1 # [ 167.178059] nix-grpc-daemon[1376]: time=2026-09-21T12:56:22.369Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6244worker1 # [ 167.179683] nix-grpc-daemon[1376]: time=2026-09-21T12:56:22.369Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6245lb # [ 167.582070] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6246lb # [ 167.604155] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6247worker1 # [ 167.236118] nix-grpc-daemon[1376]: time=2026-09-21T12:56:22.427Z level=INFO msg="Uploading 1 narinfos"6248lb # [ 167.617161] niks3-server[1026]: 2026/09/21 12:56:22 INFO Signed narinfos id=58 count=16249lb # [ 167.624366] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6250lb # [ 167.636140] niks3-server[1026]: 2026/09/21 12:56:22 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6251lb # [ 167.644331] niks3-server[1026]: 2026/09/21 12:56:22 INFO Completed upload id=586252worker1 # [ 167.267610] nix-grpc-daemon[1376]: time=2026-09-21T12:56:22.459Z level=INFO msg="Upload complete. (583ms)"6253lb # [ 167.647624] niks3-server[1026]: 2026/09/21 12:56:22 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6254worker1 # [ 167.281832] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=7526255worker1 # [ 167.296560] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306256worker1 # [ 167.311113] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6257worker1 # [ 167.317653] nix-daemon[837]: reaped child process 1794, status = succeeded6258worker1 # [ 167.337065] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6259worker1 # [ 168.975011] nix-daemon[837]: reaped child process 1808, status = succeeded6260worker1 # [ 169.164293] nix-daemon[837]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6261worker1 # [ 169.193951] nix-daemon[837]: reaped child process 1827, status = succeeded6262lb # [ 169.585352] niks3-server[1026]: 2026/09/21 12:56:24 INFO Received uploads request method=POST path=/api/pending_closures6263worker1 # [ 169.224611] nix-grpc-daemon[1376]: time=2026-09-21T12:56:24.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6264worker1 # [ 169.226243] nix-grpc-daemon[1376]: time=2026-09-21T12:56:24.415Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6265lb # [ 169.646949] niks3-server[1026]: 2026/09/21 12:56:24 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6266worker1 # [ 169.287638] nix-grpc-daemon[1376]: time=2026-09-21T12:56:24.479Z level=INFO msg="Uploading 1 narinfos"6267lb # [ 169.663526] niks3-server[1026]: 2026/09/21 12:56:24 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6268lb # [ 169.674450] niks3-server[1026]: 2026/09/21 12:56:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6269lb # [ 169.680309] niks3-server[1026]: 2026/09/21 12:56:24 INFO Signed narinfos id=59 count=16270lb # [ 169.682196] niks3-server[1026]: 2026/09/21 12:56:24 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6271lb # [ 169.687146] niks3-server[1026]: 2026/09/21 12:56:24 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6272lb # [ 169.694989] niks3-server[1026]: 2026/09/21 12:56:24 INFO Completed upload id=596273worker1 # [ 169.318472] nix-grpc-daemon[1376]: time=2026-09-21T12:56:24.510Z level=INFO msg="Upload complete. (365ms)"6274lb # [ 169.697416] niks3-server[1026]: 2026/09/21 12:56:24 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6275worker1 # [ 169.337826] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16276worker1 # [ 169.352550] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:24Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16277worker1 # [ 169.363788] nix-daemon[837]: reaped child process 1806, status = succeeded6278worker1 # [ 169.374316] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306279worker1 # [ 169.394375] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306280worker1 # [ 169.417720] nix-grpc-daemon[1376]: time=2026-09-21T12:56:24.609Z level=INFO msg="All 2 paths already cached"6281worker1 # [ 169.432183] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306282worker1 # [ 169.451061] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6283worker1 # [ 169.699900] nix-daemon[837]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6284worker1 # [ 169.730168] nix-daemon[837]: reaped child process 1834, status = succeeded6285worker1 # [ 169.933277] nix-daemon[837]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6286worker1 # [ 169.964219] nix-daemon[837]: reaped child process 1839, status = succeeded6287lb # [ 170.353110] niks3-server[1026]: 2026/09/21 12:56:25 INFO Received uploads request method=POST path=/api/pending_closures6288worker1 # [ 169.991632] nix-grpc-daemon[1376]: time=2026-09-21T12:56:25.183Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6289worker1 # [ 169.994119] nix-grpc-daemon[1376]: time=2026-09-21T12:56:25.185Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6290worker1 # [ 170.041456] nix-grpc-daemon[1376]: time=2026-09-21T12:56:25.233Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 170.406506] niks3-server[1026]: 2026/09/21 12:56:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6292lb # [ 170.422971] niks3-server[1026]: 2026/09/21 12:56:25 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6293lb # [ 170.436797] niks3-server[1026]: 2026/09/21 12:56:25 INFO Signed narinfos id=60 count=16294lb # [ 170.439037] niks3-server[1026]: 2026/09/21 12:56:25 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6295lb # [ 170.452449] niks3-server[1026]: 2026/09/21 12:56:25 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6296lb # [ 170.465235] niks3-server[1026]: 2026/09/21 12:56:25 INFO Completed upload id=606297lb # [ 170.473602] niks3-server[1026]: 2026/09/21 12:56:25 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6298worker1 # [ 170.096805] nix-grpc-daemon[1376]: time=2026-09-21T12:56:25.288Z level=INFO msg="Upload complete. (622ms)"6299worker1 # [ 170.103164] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=7526300worker1 # [ 170.114929] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306301worker1 # [ 170.119876] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6302worker1 # [ 170.124242] nix-daemon[837]: reaped child process 1829, status = succeeded6303worker1 # [ 170.136306] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6304worker1 # [ 171.767337] nix-daemon[837]: reaped child process 1843, status = succeeded6305worker1 # [ 171.962068] nix-daemon[837]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6306worker1 # [ 171.987970] nix-daemon[837]: reaped child process 1862, status = succeeded6307lb # [ 172.381805] niks3-server[1026]: 2026/09/21 12:56:27 INFO Received uploads request method=POST path=/api/pending_closures6308worker1 # [ 172.020639] nix-grpc-daemon[1376]: time=2026-09-21T12:56:27.211Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6309worker1 # [ 172.022474] nix-grpc-daemon[1376]: time=2026-09-21T12:56:27.211Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6310lb # [ 172.436735] niks3-server[1026]: 2026/09/21 12:56:27 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6311lb # [ 172.444780] niks3-server[1026]: 2026/09/21 12:56:27 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6312lb # [ 172.454769] niks3-server[1026]: 2026/09/21 12:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6313worker1 # [ 172.085266] nix-grpc-daemon[1376]: time=2026-09-21T12:56:27.276Z level=INFO msg="Uploading 1 narinfos"6314lb # [ 172.462760] niks3-server[1026]: 2026/09/21 12:56:27 INFO Signed narinfos id=61 count=16315lb # [ 172.467144] niks3-server[1026]: 2026/09/21 12:56:27 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6316lb # [ 172.477918] niks3-server[1026]: 2026/09/21 12:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6317lb # [ 172.485104] niks3-server[1026]: 2026/09/21 12:56:27 INFO Completed upload id=616318worker1 # [ 172.108524] nix-grpc-daemon[1376]: time=2026-09-21T12:56:27.300Z level=INFO msg="Upload complete. (361ms)"6319lb # [ 172.488668] niks3-server[1026]: 2026/09/21 12:56:27 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6320worker1 # [ 172.128840] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16321worker1 # [ 172.144837] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:27Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16322worker1 # [ 172.153382] nix-daemon[837]: reaped child process 1841, status = succeeded6323worker1 # [ 172.169512] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306324worker1 # [ 172.186421] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538306325worker1 # [ 172.206225] nix-grpc-daemon[1376]: time=2026-09-21T12:56:27.398Z level=INFO msg="All 2 paths already cached"6326worker1 # [ 172.218981] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538306327worker1 # [ 172.238170] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6328worker1 # [ 172.473641] nix-daemon[837]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6329worker1 # [ 172.506347] nix-daemon[837]: reaped child process 1869, status = succeeded6330worker1 # [ 172.750933] nix-daemon[837]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6331worker1 # [ 172.783176] nix-daemon[837]: reaped child process 1874, status = succeeded6332lb # [ 173.175822] niks3-server[1026]: 2026/09/21 12:56:28 INFO Received uploads request method=POST path=/api/pending_closures6333worker1 # [ 172.814590] nix-grpc-daemon[1376]: time=2026-09-21T12:56:28.006Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6334worker1 # [ 172.817634] nix-grpc-daemon[1376]: time=2026-09-21T12:56:28.006Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6335lb # [ 173.210551] niks3-server[1026]: 2026/09/21 12:56:28 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6336worker1 # [ 172.845951] nix-grpc-daemon[1376]: time=2026-09-21T12:56:28.038Z level=INFO msg="Uploading 1 narinfos"6337lb # [ 173.222228] niks3-server[1026]: 2026/09/21 12:56:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6338lb # [ 173.226957] niks3-server[1026]: 2026/09/21 12:56:28 INFO Signed narinfos id=62 count=16339lb # [ 173.234179] niks3-server[1026]: 2026/09/21 12:56:28 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6340lb # [ 173.245141] niks3-server[1026]: 2026/09/21 12:56:28 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6341worker1 # [ 172.875398] nix-grpc-daemon[1376]: time=2026-09-21T12:56:28.067Z level=INFO msg="Upload complete. (618ms)"6342lb # [ 173.252849] niks3-server[1026]: 2026/09/21 12:56:28 INFO Completed upload id=626343lb # [ 173.255953] niks3-server[1026]: 2026/09/21 12:56:28 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6344worker1 # [ 172.884658] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=0 paths=1 nar_bytes_in=7526345worker1 # [ 172.896327] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538306346worker1 # [ 172.904614] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6347worker1 # [ 172.910420] nix-daemon[837]: reaped child process 1864, status = succeeded6348worker1 # [ 172.923684] nix-daemon[837]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)6349worker1 # [ 174.559966] nix-daemon[837]: reaped child process 1878, status = succeeded6350worker1 # [ 174.761752] nix-daemon[837]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6351worker1 # [ 174.785085] nix-daemon[837]: reaped child process 1897, status = succeeded6352lb # [ 175.175940] niks3-server[1026]: 2026/09/21 12:56:30 INFO Received uploads request method=POST path=/api/pending_closures6353worker1 # [ 174.818902] nix-grpc-daemon[1376]: time=2026-09-21T12:56:30.009Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6354worker1 # [ 174.821378] nix-grpc-daemon[1376]: time=2026-09-21T12:56:30.009Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6355lb # [ 175.225701] niks3-server[1026]: 2026/09/21 12:56:30 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6356worker1 # [ 174.864241] nix-grpc-daemon[1376]: time=2026-09-21T12:56:30.056Z level=INFO msg="Uploading 1 narinfos"6357lb # [ 175.241027] niks3-server[1026]: 2026/09/21 12:56:30 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6358lb # [ 175.251621] niks3-server[1026]: 2026/09/21 12:56:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6359lb # [ 175.258448] niks3-server[1026]: 2026/09/21 12:56:30 INFO Signed narinfos id=63 count=16360lb # [ 175.264848] niks3-server[1026]: 2026/09/21 12:56:30 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6361worker1 # [ 174.894247] nix-grpc-daemon[1376]: time=2026-09-21T12:56:30.086Z level=INFO msg="Upload complete. (349ms)"6362lb # [ 175.270853] niks3-server[1026]: 2026/09/21 12:56:30 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6363lb # [ 175.276067] niks3-server[1026]: 2026/09/21 12:56:30 INFO Completed upload id=636364lb # [ 175.279494] niks3-server[1026]: 2026/09/21 12:56:30 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6365worker1 # [ 174.912824] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53830 duration_s=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16366worker1 # [ 174.929843] nix-daemon[837]: reaped child process 1876, status = succeeded6367client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 22.25 seconds)6368worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6369worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6370client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6371client # [ 175.570887] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6372client # Running as unit: gpu.service; invocation ID: 943dca5e40144d41a96bf21eeafe89826373client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.12 seconds)6374lb: must succeed: curl -sf localhost:9901/clusters6375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.39 seconds)6376lb: must succeed: curl -sf localhost:9901/clusters6377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6378worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6379worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6380worker2 # [ 175.196750] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:563506381worker2 # [ 175.206110] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6382client # [ 176.176200] nix[1341]: don't know how to build these paths:6383client # [ 176.177556] nix[1341]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6384worker2 # [ 175.244594] nix-grpc-daemon[1927]: ts=2026-09-21T12:56:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563506385worker2 # [ 175.278660] nix-grpc-daemon[1927]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)6386worker2 # [ 175.304983] nix-grpc-daemon[1927]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 2/5)6387worker2 # [ 175.454047] nix-grpc-daemon[1927]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 371 ms (attempt 3/5)6388worker2 # [ 175.827951] nix-grpc-daemon[1927]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 4/5)6389worker2 # [ 176.014146] nix-grpc-daemon[1927]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6390worker2 # [ 176.022822] nix-grpc-daemon[1927]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6391worker1 # [ 176.611827] nix-grpc-daemon[1296]: ts=2026-09-21T12:56:31Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06392client # [ 177.001351] nix[1341]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6393lb: must succeed: curl -sf localhost:9901/clusters6394lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6395lb: must succeed: curl -sf localhost:9901/clusters6396lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6397worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6398worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6399client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6400lb # [ 183.857135] envoy[915]: conn peer=[2001:db8:1::1]:40376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27131 tx=11438 ms=218876401client # [ 207.031473] nix[1341]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6402client # [ 207.294866] nix[1341]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6403client # [ 207.512556] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6404client # [ 207.535629] systemd[1]: gpu.service: Failed with result 'exit-code'.6405client # [ 207.559886] systemd[1]: gpu.service: Consumed 367ms CPU time over 31.941s wall clock time, 35.5M memory peak, 4K written to disk, 5.4K incoming IP traffic, 5.2K outgoing IP traffic.6406client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 31.44 seconds)6407client: must succeed: journalctl -u gpu -o cat6408client: (finished: must succeed: journalctl -u gpu -o cat, in 0.79 seconds)6409lb: must succeed: curl -sf localhost:9901/clusters6410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 1.24 seconds)6411lb: must succeed: curl -sf localhost:9901/clusters6412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.80 seconds)6413worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6414worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.92 seconds)6415(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 59.38 seconds)6416subtest: low disk drains a worker and builds go to the other6417worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6418lb # [ 213.898860] envoy[915]: conn peer=[2001:db8:1::1]:47584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=914 tx=518 ms=311626419worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 20.04 seconds)6420worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6421worker1 # [ 232.293282] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:27Z level=info event=unhealthy reason=min_free6422worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.18 seconds)6423client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26424client # don't know how to build these paths:6425client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6426worker2 # [ 232.105366] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362106427worker1 # [ 232.719546] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:27Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06428worker1 # [ 232.727307] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16429worker2 # [ 232.115944] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362106430worker2 # [ 232.218122] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106431worker2 # [ 232.249983] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362106432client # copying 1 paths to 'lb:50051'...6433client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6434worker2 # [ 232.275694] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362106435worker2 # [ 232.291408] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6436worker2 # [ 232.329302] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:28Z level=info event=niks3_push_started pid=19936437worker2 # [ 232.574029] nix-grpc-daemon[1993]: time=2026-09-21T12:57:28.866Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6438worker2 # [ 232.817323] nix-daemon[864]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6439worker2 # [ 232.852816] nix-daemon[864]: reaped child process 2009, status = succeeded6440worker2 # [ 233.084183] nix-daemon[864]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6441worker2 # [ 233.112271] nix-daemon[864]: reaped child process 2014, status = succeeded6442lb # [ 234.053774] niks3-server[1026]: 2026/09/21 12:57:29 INFO Received uploads request method=POST path=/api/pending_closures6443worker2 # [ 233.132355] nix-grpc-daemon[1993]: time=2026-09-21T12:57:29.425Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6444worker2 # [ 233.137712] nix-grpc-daemon[1993]: time=2026-09-21T12:57:29.427Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6445lb # [ 234.100490] niks3-server[1026]: 2026/09/21 12:57:29 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6446lb # [ 234.113436] niks3-server[1026]: 2026/09/21 12:57:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6447worker2 # [ 233.184633] nix-grpc-daemon[1993]: time=2026-09-21T12:57:29.477Z level=INFO msg="Uploading 1 narinfos"6448lb # [ 234.117245] niks3-server[1026]: 2026/09/21 12:57:29 INFO Signed narinfos id=64 count=16449lb # [ 234.121062] niks3-server[1026]: 2026/09/21 12:57:29 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6450lb # [ 234.130603] niks3-server[1026]: 2026/09/21 12:57:29 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6451lb # [ 234.137345] niks3-server[1026]: 2026/09/21 12:57:29 INFO Completed upload id=646452worker2 # [ 233.208090] nix-grpc-daemon[1993]: time=2026-09-21T12:57:29.501Z level=INFO msg="Upload complete. (629ms)"6453lb # [ 234.140475] niks3-server[1026]: 2026/09/21 12:57:29 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6454worker2 # [ 233.217548] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=0 paths=1 nar_bytes_in=5126455worker2 # [ 233.228104] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106456worker2 # [ 233.240609] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6457worker2 # [ 233.245762] nix-daemon[864]: reaped child process 1991, status = succeeded6458client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6459client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6460worker2 # [ 233.271280] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6461client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6462client # farm-a> LOG-farm-a-drain6463worker2 # [ 234.325015] nix-daemon[864]: reaped child process 2018, status = succeeded6464worker2 # [ 234.509638] nix-daemon[864]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6465worker2 # [ 234.549498] nix-daemon[864]: reaped child process 2037, status = succeeded6466lb # [ 235.490212] niks3-server[1026]: 2026/09/21 12:57:30 INFO Received uploads request method=POST path=/api/pending_closures6467worker2 # [ 234.578294] nix-grpc-daemon[1993]: time=2026-09-21T12:57:30.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker2 # [ 234.588592] nix-grpc-daemon[1993]: time=2026-09-21T12:57:30.873Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6469lb # [ 235.559992] niks3-server[1026]: 2026/09/21 12:57:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6470lb # [ 235.572326] niks3-server[1026]: 2026/09/21 12:57:30 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6471worker2 # [ 234.644193] nix-grpc-daemon[1993]: time=2026-09-21T12:57:30.934Z level=INFO msg="Uploading 1 narinfos"6472lb # [ 235.578424] niks3-server[1026]: 2026/09/21 12:57:30 INFO Signed narinfos id=65 count=16473lb # [ 235.583284] niks3-server[1026]: 2026/09/21 12:57:30 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6474lb # [ 235.588734] niks3-server[1026]: 2026/09/21 12:57:30 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6475lb # [ 235.597469] niks3-server[1026]: 2026/09/21 12:57:30 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6476lb # [ 235.604113] niks3-server[1026]: 2026/09/21 12:57:30 INFO Completed upload id=656477worker2 # [ 234.674254] nix-grpc-daemon[1993]: time=2026-09-21T12:57:30.967Z level=INFO msg="Upload complete. (387ms)"6478lb # [ 235.608459] niks3-server[1026]: 2026/09/21 12:57:30 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6479worker2 # [ 234.685268] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16480worker1 # [ 235.258517] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:30Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06481worker2 # [ 234.714238] nix-daemon[864]: reaped child process 2016, status = succeeded6482worker1 # [ 235.259414] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:30Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16483worker2 # [ 234.753690] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106484worker2 # [ 234.780805] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362106485worker2 # [ 234.803427] nix-grpc-daemon[1993]: time=2026-09-21T12:57:31.096Z level=INFO msg="All 1 paths already cached"6486client # copying 1 paths to 'lb:50051'...6487client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6488worker2 # [ 234.818067] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362106489worker2 # [ 234.830811] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6490worker2 # [ 235.080074] nix-daemon[864]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6491worker2 # [ 235.117406] nix-daemon[864]: reaped child process 2044, status = succeeded6492worker2 # [ 235.314693] nix-daemon[864]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6493worker2 # [ 235.346806] nix-daemon[864]: reaped child process 2049, status = succeeded6494lb # [ 236.292795] niks3-server[1026]: 2026/09/21 12:57:31 INFO Received uploads request method=POST path=/api/pending_closures6495worker2 # [ 235.376430] nix-grpc-daemon[1993]: time=2026-09-21T12:57:31.669Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6496worker2 # [ 235.378859] nix-grpc-daemon[1993]: time=2026-09-21T12:57:31.671Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6497lb # [ 236.341236] niks3-server[1026]: 2026/09/21 12:57:31 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6498lb # [ 236.353287] niks3-server[1026]: 2026/09/21 12:57:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6499worker2 # [ 235.430962] nix-grpc-daemon[1993]: time=2026-09-21T12:57:31.723Z level=INFO msg="Uploading 1 narinfos"6500lb # [ 236.363441] niks3-server[1026]: 2026/09/21 12:57:31 INFO Signed narinfos id=66 count=16501lb # [ 236.375073] niks3-server[1026]: 2026/09/21 12:57:31 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6502lb # [ 236.390587] niks3-server[1026]: 2026/09/21 12:57:31 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6503lb # [ 236.397811] niks3-server[1026]: 2026/09/21 12:57:31 INFO Completed upload id=666504worker2 # [ 235.470375] nix-grpc-daemon[1993]: time=2026-09-21T12:57:31.763Z level=INFO msg="Upload complete. (620ms)"6505lb # [ 236.403896] niks3-server[1026]: 2026/09/21 12:57:31 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6506worker2 # [ 235.484246] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=0 paths=1 nar_bytes_in=6326507worker2 # [ 235.499430] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106508worker2 # [ 235.519017] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6509worker2 # [ 235.526825] nix-daemon[864]: reaped child process 2039, status = succeeded6510client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6511client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6512worker2 # [ 235.548296] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6513client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6514client # farm-b> LOG-farm-b-drain6515worker2 # [ 237.417323] nix-daemon[864]: reaped child process 2053, status = succeeded6516worker2 # [ 237.629273] nix-daemon[864]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6517worker2 # [ 237.657003] nix-daemon[864]: reaped child process 2072, status = succeeded6518lb # [ 238.602341] niks3-server[1026]: 2026/09/21 12:57:33 INFO Received uploads request method=POST path=/api/pending_closures6519worker2 # [ 237.694633] nix-grpc-daemon[1993]: time=2026-09-21T12:57:33.986Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6520worker2 # [ 237.696486] nix-grpc-daemon[1993]: time=2026-09-21T12:57:33.986Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6521lb # [ 238.658077] niks3-server[1026]: 2026/09/21 12:57:33 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6522lb # [ 238.673575] niks3-server[1026]: 2026/09/21 12:57:33 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6523worker2 # [ 237.751826] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.044Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 238.685324] niks3-server[1026]: 2026/09/21 12:57:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6525lb # [ 238.696598] niks3-server[1026]: 2026/09/21 12:57:33 INFO Signed narinfos id=67 count=16526lb # [ 238.700529] niks3-server[1026]: 2026/09/21 12:57:33 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6527lb # [ 238.707701] niks3-server[1026]: 2026/09/21 12:57:33 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6528lb # [ 238.714522] niks3-server[1026]: 2026/09/21 12:57:34 INFO Completed upload id=676529lb # [ 238.717165] niks3-server[1026]: 2026/09/21 12:57:34 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6530worker2 # [ 237.785458] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.078Z level=INFO msg="Upload complete. (416ms)"6531worker1 # [ 238.359061] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:33Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06532worker1 # [ 238.359295] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:33Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16533worker2 # [ 237.801488] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=2 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16534worker2 # [ 237.844551] nix-daemon[864]: reaped child process 2051, status = succeeded6535worker2 # [ 237.850448] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106536worker2 # [ 237.863652] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362106537worker2 # [ 237.879569] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.173Z level=INFO msg="All 2 paths already cached"6538client # copying 1 paths to 'lb:50051'...6539worker2 # [ 237.890537] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362106540client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6541worker2 # [ 237.901681] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6542worker2 # [ 238.142796] nix-daemon[864]: accepted connection from pid 2076, user nix-grpc-daemon (trusted)6543worker2 # [ 238.184735] nix-daemon[864]: reaped child process 2079, status = succeeded6544worker2 # [ 238.386713] nix-daemon[864]: accepted connection from pid 2081, user nix-grpc-daemon (trusted)6545worker2 # [ 238.425728] nix-daemon[864]: reaped child process 2084, status = succeeded6546lb # [ 239.361746] niks3-server[1026]: 2026/09/21 12:57:34 INFO Received uploads request method=POST path=/api/pending_closures6547worker2 # [ 238.444247] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6548worker2 # [ 238.448377] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.740Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6549lb # [ 239.414064] niks3-server[1026]: 2026/09/21 12:57:34 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6550worker2 # [ 238.499018] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.792Z level=INFO msg="Uploading 1 narinfos"6551lb # [ 239.428211] niks3-server[1026]: 2026/09/21 12:57:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6552lb # [ 239.438574] niks3-server[1026]: 2026/09/21 12:57:34 INFO Signed narinfos id=68 count=16553lb # [ 239.443507] niks3-server[1026]: 2026/09/21 12:57:34 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6554lb # [ 239.455382] niks3-server[1026]: 2026/09/21 12:57:34 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6555lb # [ 239.462502] niks3-server[1026]: 2026/09/21 12:57:34 INFO Completed upload id=686556lb # [ 239.465600] niks3-server[1026]: 2026/09/21 12:57:34 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6557worker2 # [ 238.533183] nix-grpc-daemon[1993]: time=2026-09-21T12:57:34.826Z level=INFO msg="Upload complete. (610ms)"6558worker2 # [ 238.545363] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=0 paths=1 nar_bytes_in=7606559worker2 # [ 238.560668] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362106560worker2 # [ 238.573163] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6561worker2 # [ 238.592741] nix-daemon[864]: reaped child process 2074, status = succeeded6562worker2 # [ 238.605828] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6563client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6564client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6565client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6566client # farm-top> LOG-farm-top-drain6567worker2 # [ 240.087951] nix-daemon[864]: reaped child process 2088, status = succeeded6568worker2 # [ 240.249157] nix-daemon[864]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)6569worker2 # [ 240.296869] nix-daemon[864]: reaped child process 2107, status = succeeded6570lb # [ 241.234667] niks3-server[1026]: 2026/09/21 12:57:36 INFO Received uploads request method=POST path=/api/pending_closures6571worker2 # [ 240.319045] nix-grpc-daemon[1993]: time=2026-09-21T12:57:36.612Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6572worker2 # [ 240.322940] nix-grpc-daemon[1993]: time=2026-09-21T12:57:36.614Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6573lb # [ 241.285598] niks3-server[1026]: 2026/09/21 12:57:36 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6574lb # [ 241.295580] niks3-server[1026]: 2026/09/21 12:57:36 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6575lb # [ 241.308827] niks3-server[1026]: 2026/09/21 12:57:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6576worker2 # [ 240.380144] nix-grpc-daemon[1993]: time=2026-09-21T12:57:36.672Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 241.317156] niks3-server[1026]: 2026/09/21 12:57:36 INFO Signed narinfos id=69 count=16578lb # [ 241.319248] niks3-server[1026]: 2026/09/21 12:57:36 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6579lb # [ 241.330109] niks3-server[1026]: 2026/09/21 12:57:36 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6580lb # [ 241.339623] niks3-server[1026]: 2026/09/21 12:57:36 INFO Completed upload id=696581worker2 # [ 240.408341] nix-grpc-daemon[1993]: time=2026-09-21T12:57:36.701Z level=INFO msg="Upload complete. (382ms)"6582lb # [ 241.343209] niks3-server[1026]: 2026/09/21 12:57:36 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6583worker2 # [ 240.423433] nix-grpc-daemon[1927]: ts=2026-09-21T12:57:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36210 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16584worker2 # [ 240.459405] nix-daemon[864]: reaped child process 2086, status = succeeded6585client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 8.67 seconds)6586client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6587client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.28 seconds)6588worker1: must succeed: rm /nix/.rw-store/fill6589worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)6590worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6591worker1 # [ 241.555388] nix-grpc-daemon[1296]: ts=2026-09-21T12:57:36Z level=info event=healthy reason=min_free6592worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 0.05 seconds)6593(finished: subtest: low disk drains a worker and builds go to the other, in 29.45 seconds)6594(finished: run the VM test script, in 247.94 seconds)6595test script finished in 248.38s6596cleanup6597kill QemuMachine (pid 49)6598client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6599kill QemuMachine (pid 50)6600lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6601kill QemuMachine (pid 51)6602worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6603kill QemuMachine (pid 52)6604worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6605(finished: cleanup, in 3.21 seconds)6606additionally exposed symbols:6607 client, lb, worker1, worker2,6608 vlan1,6609 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh