vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #469
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14client: QEMU running (pid 50)15worker1: QEMU running (pid 51)16worker1 # Disk image does not exist, creating the virtualisation disk image...17lb # Disk image does not exist, creating the virtualisation disk image...18lb: QEMU running (pid 49)19client # Formatting '/build/vm-state-client/tmp.2X2qSWdrsn', fmt=raw size=107374182420worker2 # Formatting '/build/vm-state-worker2/tmp.LvhZC8jwtE', fmt=raw size=107374182421worker2: QEMU running (pid 52)22client # mke2fs 1.47.4 (6-Mar-2025)23worker2 # mke2fs 1.47.4 (6-Mar-2025)24lb # Formatting '/build/vm-state-lb/tmp.eyFhjpWBVa', fmt=raw size=107374182425worker1 # Formatting '/build/vm-state-worker1/tmp.Zq6wS3yVKC', fmt=raw size=107374182426(finished: start all VMs, in 0.49 seconds)27client # Discarding device blocks: 0/262144 done28worker2 # Discarding device blocks: 0/262144 done29lb # mke2fs 1.47.4 (6-Mar-2025)30worker1 # mke2fs 1.47.4 (6-Mar-2025)31client: waiting for unit network-addresses-eth1.service32client # Creating filesystem with 262144 4k blocks and 65536 inodes33worker1 # Discarding device blocks: 0/262144 done34client: waiting for the VM to finish booting35client # Filesystem UUID: 3d9162f0-02a6-4fd9-a510-3cdf3892677d36worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes37lb # Discarding device blocks: 0/262144 done38worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes39client # Superblock backups stored on blocks:40worker1 # Filesystem UUID: f7c5d3cd-a4c0-4abb-872c-160b098e5d0441lb # Creating filesystem with 262144 4k blocks and 65536 inodes42worker2 # Filesystem UUID: e47d3285-a155-4fd1-a63a-734ecaee58fd43client # 32768, 98304, 163840, 22937644worker1 # Superblock backups stored on blocks:45lb # Filesystem UUID: ff69be9e-844c-4a61-a44f-dbd929a9fa0846worker2 # Superblock backups stored on blocks:47client # 48worker1 # 32768, 98304, 163840, 22937649lb # Superblock backups stored on blocks:50worker2 # 32768, 98304, 163840, 22937651client # Allocating group tables: 0/8 done52worker1 # 53lb # 32768, 98304, 163840, 22937654worker2 # 55client # Writing inode tables: 0/8 done56worker1 # Allocating group tables: 0/8 done57lb # 58worker2 # Allocating group tables: 0/8 done59client # Creating journal (8192 blocks): done60worker1 # Writing inode tables: 0/8 done61lb # Allocating group tables: 0/8 done62worker2 # Writing inode tables: 0/8 done63client # Writing superblocks and filesystem accounting information: 0/8 done64worker1 # Creating journal (8192 blocks): done65lb # Writing inode tables: 0/8 done66worker2 # Creating journal (8192 blocks): done67client # 68worker1 # Writing superblocks and filesystem accounting information: 0/8 done69lb # Creating journal (8192 blocks): done70worker2 # Writing superblocks and filesystem accounting information: 0/8 done71client # Virtualisation disk image created.72worker1 # 73lb # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # Virtualisation disk image created.75lb # 76worker2 # 77lb # Virtualisation disk image created.78worker2 # Virtualisation disk image created.79client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681client # [ 0.000000] KASLR enabled82client # [ 0.000000] random: crng init done83client # [ 0.000000] Machine model: linux,dummy-virt84client # [ 0.000000] efi: UEFI not found.85client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88client # [ 0.000000] Zone ranges:89client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90client # [ 0.000000] DMA32 empty91client # [ 0.000000] Normal empty92client # [ 0.000000] Device empty93client # [ 0.000000] Movable zone start for each node94client # [ 0.000000] Early memory node ranges95client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098client # [ 0.000000] psci: probing for conduit method from DT.99client # [ 0.000000] psci: PSCIv1.3 detected in firmware.100client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101client # [ 0.000000] psci: Trusted OS migration not required102client # [ 0.000000] psci: SMC Calling Convention v1.1103client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105client # [ 0.000000] Detected PIPT I-cache on CPU0106client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107client # [ 0.000000] CPU features: detected: GICv3 CPU interface108client # [ 0.000000] CPU features: detected: Spectre-v4109client # [ 0.000000] CPU features: detected: Spectre-BHB110client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112client # [ 0.000000] alternatives: applying boot alternatives113client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test/init regInfo=/nix/store/qv89ps7hlc2xvxz8pzqw4jziggm25i3g-closure-info/registration console=ttyAMA0,115200n8 console=tty0114client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qv89ps7hlc2xvxz8pzqw4jziggm25i3g-closure-info/registration", will be passed to user space.115client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119client # [ 0.000000] software IO TLB: area num 1.120client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122client # [ 0.000000] Fallback order for Node 0: 0123client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124client # [ 0.000000] Policy zone: DMA125client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127client # [ 0.000000] allocated 3145728 bytes of page_ext128client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129client # [ 0.000000] ftrace: allocated 294 pages with 4 groups130client # [ 0.000000] rcu: Hierarchical RCU implementation.131client # [ 0.000000] rcu: RCU event tracing is enabled.132client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.133client # [ 0.000000] Trampoline variant of Tasks RCU enabled.134client # [ 0.000000] Rude variant of Tasks RCU enabled.135client # [ 0.000000] Tracing variant of Tasks RCU enabled.136client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.137client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1138client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.139client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.140client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.141client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0142client # [ 0.000000] GICv3: 256 SPIs implemented143client # [ 0.000000] GICv3: 0 Extended SPIs implemented144client # [ 0.000000] Root IRQ handler: gic_handle_irq145client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI146worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]147client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0148client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000149client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]150worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026151worker2 # [ 0.000000] KASLR enabled152worker2 # [ 0.000000] random: crng init done153client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)154worker2 # [ 0.000000] Machine model: linux,dummy-virt155worker2 # [ 0.000000] efi: UEFI not found.156worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]157client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)158worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT159client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000160worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]161worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026162worker1 # [ 0.000000] KASLR enabled163client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000164worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]165worker1 # [ 0.000000] random: crng init done166worker2 # [ 0.000000] Zone ranges:167worker1 # [ 0.000000] Machine model: linux,dummy-virt168client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.169lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]170worker1 # [ 0.000000] efi: UEFI not found.171worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]172worker2 # [ 0.000000] DMA32 empty173client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns174worker2 # [ 0.000000] Normal empty175worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT176worker2 # [ 0.000000] Device empty177client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).178lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026179worker2 # [ 0.000000] Movable zone start for each node180worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]181lb # [ 0.000000] KASLR enabled182worker2 # [ 0.000000] Early memory node ranges183lb # [ 0.000000] random: crng init done184worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]185worker1 # [ 0.000000] Zone ranges:186client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns187lb # [ 0.000000] Machine model: linux,dummy-virt188worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]189lb # [ 0.000000] efi: UEFI not found.190worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]191client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns192worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]193worker1 # [ 0.000000] DMA32 empty194lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT195client # [ 0.000036] arm-pv: using stolen time PV196worker1 # [ 0.000000] Normal empty197worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000198worker1 # [ 0.000000] Device empty199lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]200worker2 # [ 0.000000] psci: probing for conduit method from DT.201worker1 # [ 0.000000] Movable zone start for each node202client # [ 0.000665] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)203worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.204worker1 # [ 0.000000] Early memory node ranges205lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]206client # [ 0.000825] Console: colour dummy device 80x25207lb # [ 0.000000] Zone ranges:208worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs209worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]210client # [ 0.000832] printk: legacy console [tty0] enabled211lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]212worker2 # [ 0.000000] psci: Trusted OS migration not required213lb # [ 0.000000] DMA32 empty214worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]215worker2 # [ 0.000000] psci: SMC Calling Convention v1.1216lb # [ 0.000000] Normal empty217client # [ 0.001026] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)218lb # [ 0.000000] Device empty219worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000220client # [ 0.001033] pid_max: default: 32768 minimum: 301221lb # [ 0.000000] Movable zone start for each node222worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)223worker1 # [ 0.000000] psci: probing for conduit method from DT.224lb # [ 0.000000] Early memory node ranges225worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.226client # [ 0.001105] LSM: initializing lsm=capability,landlock,yama,bpf,ima227worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296228lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]229client # [ 0.001275] landlock: Up and running.230worker2 # [ 0.000000] Detected PIPT I-cache on CPU0231worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs232client # [ 0.001277] Yama: becoming mindful.233worker1 # [ 0.000000] psci: Trusted OS migration not required234lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]235client # [ 0.001934] LSM support for eBPF active236worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)237worker1 # [ 0.000000] psci: SMC Calling Convention v1.1238lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000239client # [ 0.002090] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)240worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface241lb # [ 0.000000] psci: probing for conduit method from DT.242worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)243worker2 # [ 0.000000] CPU features: detected: Spectre-v4244lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.245client # [ 0.002127] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)246worker2 # [ 0.000000] CPU features: detected: Spectre-BHB247worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296248lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs249client # [ 0.003277] cacheinfo: Unable to detect cache hierarchy for CPU 0250worker1 # [ 0.000000] Detected PIPT I-cache on CPU0251worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38252lb # [ 0.000000] psci: Trusted OS migration not required253client # [ 0.004069] rcu: Hierarchical SRCU implementation.254worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23255lb # [ 0.000000] psci: SMC Calling Convention v1.1256worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)257client # [ 0.004073] rcu: Max phase no-delay instances is 1000.258worker2 # [ 0.000000] alternatives: applying boot alternatives259client # [ 0.005299] fsl-mc MSI: its@8080000 domain created260lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)261client # [ 0.005389] EFI services will not be available.262worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface263lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296264client # [ 0.005500] smp: Bringing up secondary CPUs ...265worker1 # [ 0.000000] CPU features: detected: Spectre-v4266lb # [ 0.000000] Detected PIPT I-cache on CPU0267client # [ 0.005509] smp: Brought up 1 node, 1 CPU268worker1 # [ 0.000000] CPU features: detected: Spectre-BHB269client # [ 0.005512] SMP: Total of 1 processors activated.270worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38271lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)272client # [ 0.005518] CPU: All CPU(s) started at EL1273lb # [ 0.000000] CPU features: detected: GICv3 CPU interface274worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23275client # [ 0.005529] CPU features: detected: Branch Target Identification276worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wykyb0qy7gxax0kbx52lhrandvn5hhwl-nixos-system-worker2-test/init regInfo=/nix/store/vijscrzsxl5jgm579rc9xqvnsymkza5v-closure-info/registration console=ttyAMA0,115200n8 console=tty0277lb # [ 0.000000] CPU features: detected: Spectre-v4278worker1 # [ 0.000000] alternatives: applying boot alternatives279client # [ 0.005534] CPU features: detected: ARMv8.4 Translation Table Level280lb # [ 0.000000] CPU features: detected: Spectre-BHB281lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38282client # [ 0.005537] CPU features: detected: Instruction cache invalidation not required for I/D coherence283worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/vijscrzsxl5jgm579rc9xqvnsymkza5v-closure-info/registration", will be passed to user space.284lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23285client # [ 0.005540] CPU features: detected: Data cache clean to the PoU not required for I/D coherence286worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes287lb # [ 0.000000] alternatives: applying boot alternatives288client # [ 0.005544] CPU features: detected: Common not Private translations289worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)290worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/85sl11dh0zky10jc8x3p4y9xadvl89kj-nixos-system-worker1-test/init regInfo=/nix/store/sf77vbcbghgk9hk2xbbk4q5n9r9ql4qy-closure-info/registration console=ttyAMA0,115200n8 console=tty0291client # [ 0.005547] CPU features: detected: CRC32 instructions292worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)293client # [ 0.005550] CPU features: detected: Data cache clean to Point of Deep Persistence294worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB295worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sf77vbcbghgk9hk2xbbk4q5n9r9ql4qy-closure-info/registration", will be passed to user space.296client # [ 0.005553] CPU features: detected: Data cache clean to Point of Persistence297worker2 # [ 0.000000] software IO TLB: area num 1.298worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes299worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB300lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sy3pchz5vg22ylc3whvkp4s8qngi8wmd-nixos-system-lb-test/init regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration console=ttyAMA0,115200n8 console=tty0301client # [ 0.005556] CPU features: detected: Data independent timing control (DIT)302client # [ 0.005560] CPU features: detected: E0PD303worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)304worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)305client # [ 0.005562] CPU features: detected: Enhanced Counter Virtualization306worker2 # [ 0.000000] Fallback order for Node 0: 0307worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)308lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration", will be passed to user space.309client # [ 0.005565] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)310worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216311worker2 # [ 0.000000] Policy zone: DMA312worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB313lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes314client # [ 0.005568] CPU features: detected: Enhanced Virtualization Traps315worker1 # [ 0.000000] software IO TLB: area num 1.316client # [ 0.005571] CPU features: detected: Fine Grained Traps317worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off318lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)319worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB320worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1321client # [ 0.005575] CPU features: detected: Generic authentication (architected QARMA5 algorithm)322lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)323worker2 # [ 0.000000] allocated 3145728 bytes of page_ext324worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)325client # [ 0.005580] CPU features: detected: RCpc load-acquire (LDAPR)326worker1 # [ 0.000000] Fallback order for Node 0: 0327worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages328client # [ 0.005583] CPU features: detected: LSE atomic instructions329worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups330lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB331worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216332lb # [ 0.000000] software IO TLB: area num 1.333worker1 # [ 0.000000] Policy zone: DMA334client # [ 0.005586] CPU features: detected: Privileged Access Never335worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.336client # [ 0.005589] CPU features: detected: PMUv3337worker2 # [ 0.000000] rcu: RCU event tracing is enabled.338lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB339worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off340client # [ 0.005591] CPU features: detected: RAS Extension Support341worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.342worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1343lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)344client # [ 0.005594] CPU features: detected: RASv1p1 Extension Support345worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.346worker1 # [ 0.000000] allocated 3145728 bytes of page_ext347lb # [ 0.000000] Fallback order for Node 0: 0348client # [ 0.005597] CPU features: detected: Random Number Generator349worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.350worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages351lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216352client # [ 0.005599] CPU features: detected: Speculation barrier (SB)353worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.354lb # [ 0.000000] Policy zone: DMA355worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups356client # [ 0.005602] CPU features: detected: Stage-2 Force Write-Back357worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.358worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.359lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off360worker1 # [ 0.000000] rcu: RCU event tracing is enabled.361client # [ 0.005605] CPU features: detected: TLB range maintenance instructions362worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1363lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1364worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.365client # [ 0.005610] CPU features: detected: Speculative Store Bypassing Safe (SSBS)366lb # [ 0.000000] allocated 3145728 bytes of page_ext367worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.368worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.369client # [ 0.005653] alternatives: applying system-wide alternatives370lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages371worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.372lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups373client # [ 0.008665] CPU features: detected: BBM Level 2 without TLB conflict abort374worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.375worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.376lb # [ 0.000000] rcu: Hierarchical RCU implementation.377lb # [ 0.000000] rcu: RCU event tracing is enabled.378worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.379worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.380client # [ 0.008845] Memory: 1406508K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)381lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.382worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0383worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1384client # [ 0.009284] devtmpfs: initialized385worker2 # [ 0.000000] GICv3: 256 SPIs implemented386lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.387worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.388worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented389lb # [ 0.000000] Rude variant of Tasks RCU enabled.390client # [ 0.011192] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)391worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq392lb # [ 0.000000] Tracing variant of Tasks RCU enabled.393client # [ 0.011245] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).394worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.395worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI396lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.397worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0398client # [ 0.011422] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL399worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.400lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1401worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000402worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0403worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]404client # [ 0.011427] 0 pages in range for non-PLT usage405worker1 # [ 0.000000] GICv3: 256 SPIs implemented406lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.407client # [ 0.011428] 508288 pages in range for PLT usage408worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented409worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)410client # [ 0.011536] pinctrl core: initialized pinctrl subsystem411worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq412lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.413client # [ 0.012337] DMI not present or invalid.414worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI415worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)416client # [ 0.016027] NET: Registered PF_NETLINK/PF_ROUTE protocol family417worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0418lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.419worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000420lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0421worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000422client # [ 0.018362] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations423lb # [ 0.000000] GICv3: 256 SPIs implemented424worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]425worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000426client # [ 0.018558] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations427lb # [ 0.000000] GICv3: 0 Extended SPIs implemented428worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.429lb # [ 0.000000] Root IRQ handler: gic_handle_irq430worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)431client # [ 0.018762] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations432lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI433client # [ 0.018792] audit: initializing netlink subsys (disabled)434worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns435lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0436worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)437client # [ 0.019376] thermal_sys: Registered thermal governor 'fair_share'438worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).439lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000440worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000441client # [ 0.019378] thermal_sys: Registered thermal governor 'bang_bang'442lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]443worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000444worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns445client # [ 0.019382] thermal_sys: Registered thermal governor 'step_wise'446lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)447worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.448client # [ 0.019388] thermal_sys: Registered thermal governor 'user_space'449worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns450worker2 # [ 0.000029] arm-pv: using stolen time PV451client # [ 0.019391] thermal_sys: Registered thermal governor 'power_allocator'452lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)453worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns454worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).455lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000456client # [ 0.019433] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1457worker2 # [ 0.000395] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)458client # [ 0.019441] cpuidle: using governor ladder459worker2 # [ 0.000558] Console: colour dummy device 80x25460lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000461client # [ 0.019448] cpuidle: using governor menu462worker2 # [ 0.000565] printk: legacy console [tty0] enabled463worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns464lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.465client # [ 0.019645] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.466worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns467client # [ 0.019663] ASID allocator initialised with 65536 entries468worker2 # [ 0.000754] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)469worker1 # [ 0.000029] arm-pv: using stolen time PV470lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns471client # [ 0.020918] Serial: AMBA PL011 UART driver472worker2 # [ 0.000760] pid_max: default: 32768 minimum: 301473lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).474worker2 # [ 0.000837] LSM: initializing lsm=capability,landlock,yama,bpf,ima475worker1 # [ 0.000384] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)476client # [ 0.026320] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1477worker2 # [ 0.000964] landlock: Up and running.478worker1 # [ 0.000536] Console: colour dummy device 80x25479client # [ 0.026462] printk: console [ttyAMA0] enabled480worker2 # [ 0.000967] Yama: becoming mindful.481lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns482worker1 # [ 0.000546] printk: legacy console [tty0] enabled483worker2 # [ 0.001394] LSM support for eBPF active484lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns485worker2 # [ 0.001538] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)486lb # [ 0.000029] arm-pv: using stolen time PV487worker1 # [ 0.000746] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)488worker1 # [ 0.000753] pid_max: default: 32768 minimum: 301489worker2 # [ 0.001571] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)490lb # [ 0.000400] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)491worker1 # [ 0.000832] LSM: initializing lsm=capability,landlock,yama,bpf,ima492worker2 # [ 0.002703] cacheinfo: Unable to detect cache hierarchy for CPU 0493lb # [ 0.000562] Console: colour dummy device 80x25494worker1 # [ 0.000961] landlock: Up and running.495worker2 # [ 0.003419] rcu: Hierarchical SRCU implementation.496worker1 # [ 0.000964] Yama: becoming mindful.497lb # [ 0.000570] printk: legacy console [tty0] enabled498worker2 # [ 0.003423] rcu: Max phase no-delay instances is 1000.499worker1 # [ 0.001393] LSM support for eBPF active500worker2 # [ 0.004645] fsl-mc MSI: its@8080000 domain created501worker1 # [ 0.001537] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)502lb # [ 0.000777] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)503worker2 # [ 0.004734] EFI services will not be available.504lb # [ 0.000784] pid_max: default: 32768 minimum: 301505worker2 # [ 0.004797] smp: Bringing up secondary CPUs ...506worker1 # [ 0.001574] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)507worker2 # [ 0.004805] smp: Brought up 1 node, 1 CPU508lb # [ 0.000859] LSM: initializing lsm=capability,landlock,yama,bpf,ima509worker1 # [ 0.002681] cacheinfo: Unable to detect cache hierarchy for CPU 0510worker2 # [ 0.004808] SMP: Total of 1 processors activated.511lb # [ 0.000992] landlock: Up and running.512lb # [ 0.000994] Yama: becoming mindful.513worker1 # [ 0.003423] rcu: Hierarchical SRCU implementation.514worker2 # [ 0.004814] CPU: All CPU(s) started at EL1515client # [ 0.145327] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages516lb # [ 0.001451] LSM support for eBPF active517worker1 # [ 0.003427] rcu: Max phase no-delay instances is 1000.518worker2 # [ 0.004825] CPU features: detected: Branch Target Identification519client # [ 0.145347] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page520worker1 # [ 0.004641] fsl-mc MSI: its@8080000 domain created521lb # [ 0.001592] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)522worker2 # [ 0.004830] CPU features: detected: ARMv8.4 Translation Table Level523worker1 # [ 0.004731] EFI services will not be available.524client # [ 0.145353] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages525lb # [ 0.001625] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)526worker1 # [ 0.004842] smp: Bringing up secondary CPUs ...527client # [ 0.145357] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page528worker2 # [ 0.004833] CPU features: detected: Instruction cache invalidation not required for I/D coherence529worker1 # [ 0.004850] smp: Brought up 1 node, 1 CPU530lb # [ 0.002765] cacheinfo: Unable to detect cache hierarchy for CPU 0531worker1 # [ 0.004852] SMP: Total of 1 processors activated.532client # [ 0.145362] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages533lb # [ 0.003464] rcu: Hierarchical SRCU implementation.534worker2 # [ 0.004837] CPU features: detected: Data cache clean to the PoU not required for I/D coherence535worker1 # [ 0.004858] CPU: All CPU(s) started at EL1536client # [ 0.145366] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page537lb # [ 0.003467] rcu: Max phase no-delay instances is 1000.538worker2 # [ 0.004840] CPU features: detected: Common not Private translations539worker1 # [ 0.004869] CPU features: detected: Branch Target Identification540lb # [ 0.004631] fsl-mc MSI: its@8080000 domain created541client # [ 0.145370] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages542worker2 # [ 0.004843] CPU features: detected: CRC32 instructions543lb # [ 0.004727] EFI services will not be available.544worker1 # [ 0.004874] CPU features: detected: ARMv8.4 Translation Table Level545client # [ 0.145374] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page546worker2 # [ 0.004846] CPU features: detected: Data cache clean to Point of Deep Persistence547lb # [ 0.004809] smp: Bringing up secondary CPUs ...548lb # [ 0.004817] smp: Brought up 1 node, 1 CPU549worker1 # [ 0.004877] CPU features: detected: Instruction cache invalidation not required for I/D coherence550worker2 # [ 0.004850] CPU features: detected: Data cache clean to Point of Persistence551lb # [ 0.004820] SMP: Total of 1 processors activated.552lb # [ 0.004825] CPU: All CPU(s) started at EL1553worker2 # [ 0.004853] CPU features: detected: Data independent timing control (DIT)554worker1 # [ 0.004881] CPU features: detected: Data cache clean to the PoU not required for I/D coherence555worker2 # [ 0.004856] CPU features: detected: E0PD556client # [ 0.152523] fbcon: Taking over console557lb # [ 0.004836] CPU features: detected: Branch Target Identification558worker1 # [ 0.004884] CPU features: detected: Common not Private translations559client # [ 0.152540] ACPI: Interpreter disabled.560worker2 # [ 0.004859] CPU features: detected: Enhanced Counter Virtualization561worker1 # [ 0.004887] CPU features: detected: CRC32 instructions562lb # [ 0.004841] CPU features: detected: ARMv8.4 Translation Table Level563worker2 # [ 0.004862] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)564worker1 # [ 0.004890] CPU features: detected: Data cache clean to Point of Deep Persistence565lb # [ 0.004844] CPU features: detected: Instruction cache invalidation not required for I/D coherence566worker2 # [ 0.004865] CPU features: detected: Enhanced Virtualization Traps567worker1 # [ 0.004894] CPU features: detected: Data cache clean to Point of Persistence568worker2 # [ 0.004868] CPU features: detected: Fine Grained Traps569lb # [ 0.004848] CPU features: detected: Data cache clean to the PoU not required for I/D coherence570worker1 # [ 0.004897] CPU features: detected: Data independent timing control (DIT)571lb # [ 0.004852] CPU features: detected: Common not Private translations572worker1 # [ 0.004900] CPU features: detected: E0PD573worker2 # [ 0.004872] CPU features: detected: Generic authentication (architected QARMA5 algorithm)574lb # [ 0.004855] CPU features: detected: CRC32 instructions575worker2 # [ 0.004877] CPU features: detected: RCpc load-acquire (LDAPR)576worker1 # [ 0.004902] CPU features: detected: Enhanced Counter Virtualization577worker2 # [ 0.004880] CPU features: detected: LSE atomic instructions578lb # [ 0.004858] CPU features: detected: Data cache clean to Point of Deep Persistence579worker1 # [ 0.004905] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)580worker2 # [ 0.004883] CPU features: detected: Privileged Access Never581lb # [ 0.004861] CPU features: detected: Data cache clean to Point of Persistence582worker1 # [ 0.004908] CPU features: detected: Enhanced Virtualization Traps583worker2 # [ 0.004885] CPU features: detected: PMUv3584worker1 # [ 0.004911] CPU features: detected: Fine Grained Traps585worker2 # [ 0.004888] CPU features: detected: RAS Extension Support586lb # [ 0.004864] CPU features: detected: Data independent timing control (DIT)587lb # [ 0.004867] CPU features: detected: E0PD588worker2 # [ 0.004891] CPU features: detected: RASv1p1 Extension Support589worker1 # [ 0.004914] CPU features: detected: Generic authentication (architected QARMA5 algorithm)590lb # [ 0.004870] CPU features: detected: Enhanced Counter Virtualization591worker2 # [ 0.004893] CPU features: detected: Random Number Generator592worker1 # [ 0.004920] CPU features: detected: RCpc load-acquire (LDAPR)593worker2 # [ 0.004896] CPU features: detected: Speculation barrier (SB)594client # [ 0.161510] iommu: Default domain type: Translated595lb # [ 0.004873] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)596worker1 # [ 0.004922] CPU features: detected: LSE atomic instructions597worker2 # [ 0.004899] CPU features: detected: Stage-2 Force Write-Back598client # [ 0.161522] iommu: DMA domain TLB invalidation policy: strict mode599lb # [ 0.004876] CPU features: detected: Enhanced Virtualization Traps600worker1 # [ 0.004925] CPU features: detected: Privileged Access Never601client # [ 0.161877] SCSI subsystem initialized602worker1 # [ 0.004928] CPU features: detected: PMUv3603worker2 # [ 0.004902] CPU features: detected: TLB range maintenance instructions604lb # [ 0.004879] CPU features: detected: Fine Grained Traps605worker1 # [ 0.004930] CPU features: detected: RAS Extension Support606worker2 # [ 0.004906] CPU features: detected: Speculative Store Bypassing Safe (SSBS)607lb # [ 0.004883] CPU features: detected: Generic authentication (architected QARMA5 algorithm)608client # [ 0.163786] usbcore: registered new interface driver usbfs609worker1 # [ 0.004933] CPU features: detected: RASv1p1 Extension Support610lb # [ 0.004888] CPU features: detected: RCpc load-acquire (LDAPR)611client # [ 0.163818] usbcore: registered new interface driver hub612worker1 # [ 0.004935] CPU features: detected: Random Number Generator613worker2 # [ 0.004943] alternatives: applying system-wide alternatives614client # [ 0.163847] usbcore: registered new device driver usb615lb # [ 0.004891] CPU features: detected: LSE atomic instructions616worker1 # [ 0.004938] CPU features: detected: Speculation barrier (SB)617client # [ 0.164113] pps_core: LinuxPPS API ver. 1 registered618worker2 # [ 0.007825] CPU features: detected: BBM Level 2 without TLB conflict abort619lb # [ 0.004894] CPU features: detected: Privileged Access Never620worker1 # [ 0.004941] CPU features: detected: Stage-2 Force Write-Back621lb # [ 0.004897] CPU features: detected: PMUv3622client # [ 0.164120] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>623lb # [ 0.004900] CPU features: detected: RAS Extension Support624client # [ 0.164129] PTP clock support registered625worker2 # [ 0.007958] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)626client # [ 0.164174] EDAC MC: Ver: 3.0.0627lb # [ 0.004902] CPU features: detected: RASv1p1 Extension Support628worker2 # [ 0.008412] devtmpfs: initialized629lb # [ 0.004905] CPU features: detected: Random Number Generator630client # [ 0.168603] scmi_core: SCMI protocol bus registered631worker2 # [ 0.010147] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)632lb # [ 0.004908] CPU features: detected: Speculation barrier (SB)633client # [ 0.169549] FPGA manager framework634worker2 # [ 0.010169] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).635lb # [ 0.004911] CPU features: detected: Stage-2 Force Write-Back636client # [ 0.170521] vgaarb: loaded637worker2 # [ 0.010348] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL638lb # [ 0.004914] CPU features: detected: TLB range maintenance instructions639worker2 # [ 0.010352] 0 pages in range for non-PLT usage640client # [ 0.171141] clocksource: Switched to clocksource arch_sys_counter641lb # [ 0.004918] CPU features: detected: Speculative Store Bypassing Safe (SSBS)642worker2 # [ 0.010353] 508288 pages in range for PLT usage643lb # [ 0.004953] alternatives: applying system-wide alternatives644worker2 # [ 0.010459] pinctrl core: initialized pinctrl subsystem645worker2 # [ 0.011239] DMI not present or invalid.646lb # [ 0.007886] CPU features: detected: BBM Level 2 without TLB conflict abort647worker2 # [ 0.014190] NET: Registered PF_NETLINK/PF_ROUTE protocol family648worker2 # [ 0.016403] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations649client # [ 0.174332] VFS: Disk quotas dquot_6.6.0650lb # [ 0.008069] Memory: 1406552K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)651worker1 # [ 0.004944] CPU features: detected: TLB range maintenance instructions652lb # [ 0.008432] devtmpfs: initialized653client # [ 0.174367] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)654worker2 # [ 0.016580] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations655worker1 # [ 0.004949] CPU features: detected: Speculative Store Bypassing Safe (SSBS)656lb # [ 0.010243] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)657worker2 # [ 0.016775] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations658worker1 # [ 0.004986] alternatives: applying system-wide alternatives659worker2 # [ 0.016796] audit: initializing netlink subsys (disabled)660lb # [ 0.010265] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).661worker1 # [ 0.007925] CPU features: detected: BBM Level 2 without TLB conflict abort662worker2 # [ 0.017397] thermal_sys: Registered thermal governor 'fair_share'663lb # [ 0.010441] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL664client # [ 0.177875] netfs: FS-Cache loaded665worker2 # [ 0.017399] thermal_sys: Registered thermal governor 'bang_bang'666lb # [ 0.010446] 0 pages in range for non-PLT usage667client # [ 0.177993] pnp: PnP ACPI: disabled668lb # [ 0.010447] 508288 pages in range for PLT usage669worker2 # [ 0.017403] thermal_sys: Registered thermal governor 'step_wise'670worker1 # [ 0.008139] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)671worker1 # [ 0.008509] devtmpfs: initialized672lb # [ 0.010550] pinctrl core: initialized pinctrl subsystem673worker2 # [ 0.017408] thermal_sys: Registered thermal governor 'user_space'674lb # [ 0.011332] DMI not present or invalid.675worker1 # [ 0.010257] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)676worker2 # [ 0.017410] thermal_sys: Registered thermal governor 'power_allocator'677lb # [ 0.014490] NET: Registered PF_NETLINK/PF_ROUTE protocol family678worker1 # [ 0.010282] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).679worker2 # [ 0.017442] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1680lb # [ 0.016642] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations681worker2 # [ 0.017452] cpuidle: using governor ladder682worker1 # [ 0.010476] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL683worker2 # [ 0.017458] cpuidle: using governor menu684lb # [ 0.016813] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations685worker1 # [ 0.010480] 0 pages in range for non-PLT usage686worker2 # [ 0.017647] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.687worker1 # [ 0.010481] 508288 pages in range for PLT usage688client # [ 0.181957] NET: Registered PF_INET protocol family689worker2 # [ 0.017662] ASID allocator initialised with 65536 entries690worker1 # [ 0.010588] pinctrl core: initialized pinctrl subsystem691worker2 # [ 0.018861] Serial: AMBA PL011 UART driver692worker1 # [ 0.011343] DMI not present or invalid.693client # [ 0.182213] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)694worker1 # [ 0.014479] NET: Registered PF_NETLINK/PF_ROUTE protocol family695worker2 # [ 0.023999] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1696worker2 # [ 0.024140] printk: console [ttyAMA0] enabled697worker1 # [ 0.016664] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations698worker1 # [ 0.016835] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations699worker1 # [ 0.017031] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations700worker1 # [ 0.017055] audit: initializing netlink subsys (disabled)701worker1 # [ 0.017640] thermal_sys: Registered thermal governor 'fair_share'702worker1 # [ 0.017642] thermal_sys: Registered thermal governor 'bang_bang'703worker1 # [ 0.017646] thermal_sys: Registered thermal governor 'step_wise'704worker1 # [ 0.017651] thermal_sys: Registered thermal governor 'user_space'705lb # [ 0.017007] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations706worker1 # [ 0.017653] thermal_sys: Registered thermal governor 'power_allocator'707lb # [ 0.017028] audit: initializing netlink subsys (disabled)708lb # [ 0.017599] thermal_sys: Registered thermal governor 'fair_share'709worker1 # [ 0.017676] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1710worker1 # [ 0.017686] cpuidle: using governor ladder711lb # [ 0.017602] thermal_sys: Registered thermal governor 'bang_bang'712worker1 # [ 0.017691] cpuidle: using governor menu713lb # [ 0.017605] thermal_sys: Registered thermal governor 'step_wise'714worker1 # [ 0.017879] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.715lb # [ 0.017610] thermal_sys: Registered thermal governor 'user_space'716worker2 # [ 0.142844] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages717worker1 # [ 0.017894] ASID allocator initialised with 65536 entries718lb # [ 0.017613] thermal_sys: Registered thermal governor 'power_allocator'719worker1 # [ 0.019092] Serial: AMBA PL011 UART driver720worker2 # [ 0.142860] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page721lb # [ 0.017635] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1722worker2 # [ 0.142865] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages723worker1 # [ 0.024144] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1724lb # [ 0.017645] cpuidle: using governor ladder725worker1 # [ 0.024288] printk: console [ttyAMA0] enabled726worker2 # [ 0.142870] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page727lb # [ 0.017651] cpuidle: using governor menu728worker2 # [ 0.142874] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages729lb # [ 0.017837] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.730worker1 # [ 0.143251] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731worker2 # [ 0.142878] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page732lb # [ 0.017851] ASID allocator initialised with 65536 entries733worker1 # [ 0.143270] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page734lb # [ 0.019010] Serial: AMBA PL011 UART driver735worker2 # [ 0.142882] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages736worker1 # [ 0.143275] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages737worker2 # [ 0.142886] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page738lb # [ 0.024272] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1739worker1 # [ 0.143280] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page740lb # [ 0.024417] printk: console [ttyAMA0] enabled741worker1 # [ 0.143284] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages742lb # [ 0.143978] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages743worker1 # [ 0.143288] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page744worker2 # [ 0.150029] fbcon: Taking over console745lb # [ 0.143999] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page746worker2 # [ 0.150045] ACPI: Interpreter disabled.747worker1 # [ 0.143293] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages748worker2 # [ 0.151849] iommu: Default domain type: Translated749lb # [ 0.144005] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages750worker1 # [ 0.143297] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page751worker2 # [ 0.151859] iommu: DMA domain TLB invalidation policy: strict mode752lb # [ 0.144009] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page753worker2 # [ 0.153510] SCSI subsystem initialized754lb # [ 0.144013] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages755lb # [ 0.144018] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page756worker1 # [ 0.150536] fbcon: Taking over console757worker1 # [ 0.150553] ACPI: Interpreter disabled.758lb # [ 0.144022] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages759lb # [ 0.144026] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page760lb # [ 0.151373] fbcon: Taking over console761lb # [ 0.151387] ACPI: Interpreter disabled.762lb # [ 0.153230] iommu: Default domain type: Translated763worker2 # [ 0.159063] usbcore: registered new interface driver usbfs764lb # [ 0.153240] iommu: DMA domain TLB invalidation policy: strict mode765worker2 # [ 0.159096] usbcore: registered new interface driver hub766worker2 # [ 0.159113] usbcore: registered new device driver usb767lb # [ 0.154969] SCSI subsystem initialized768worker2 # [ 0.159378] pps_core: LinuxPPS API ver. 1 registered769worker1 # [ 0.159148] iommu: Default domain type: Translated770worker2 # [ 0.159384] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>771worker2 # [ 0.159401] PTP clock support registered772worker1 # [ 0.159159] iommu: DMA domain TLB invalidation policy: strict mode773worker2 # [ 0.159454] EDAC MC: Ver: 3.0.0774worker1 # [ 0.160885] SCSI subsystem initialized775worker2 # [ 0.164047] scmi_core: SCMI protocol bus registered776worker1 # [ 0.161549] usbcore: registered new interface driver usbfs777worker2 # [ 0.165004] FPGA manager framework778worker2 # [ 0.165980] vgaarb: loaded779worker1 # [ 0.161573] usbcore: registered new interface driver hub780worker1 # [ 0.161598] usbcore: registered new device driver usb781worker2 # [ 0.166607] clocksource: Switched to clocksource arch_sys_counter782worker1 # [ 0.161869] pps_core: LinuxPPS API ver. 1 registered783client # [ 0.210270] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)784worker1 # [ 0.161875] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>785worker1 # [ 0.161885] PTP clock support registered786client # [ 0.210316] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)787worker1 # [ 0.161925] EDAC MC: Ver: 3.0.0788lb # [ 0.162458] usbcore: registered new interface driver usbfs789lb # [ 0.162491] usbcore: registered new interface driver hub790worker1 # [ 0.166530] scmi_core: SCMI protocol bus registered791client # [ 0.210354] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)792lb # [ 0.162509] usbcore: registered new device driver usb793worker2 # [ 0.170415] VFS: Disk quotas dquot_6.6.0794client # [ 0.210424] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)795lb # [ 0.162786] pps_core: LinuxPPS API ver. 1 registered796worker1 # [ 0.167453] FPGA manager framework797worker2 # [ 0.170448] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)798worker1 # [ 0.168402] vgaarb: loaded799client # [ 0.210569] TCP: Hash tables configured (established 16384 bind 16384)800lb # [ 0.162792] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>801client # [ 0.210651] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)802lb # [ 0.162803] PTP clock support registered803lb # [ 0.162842] EDAC MC: Ver: 3.0.0804worker1 # [ 0.169029] clocksource: Switched to clocksource arch_sys_counter805client # [ 0.210688] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)806worker1 # [ 0.169606] VFS: Disk quotas dquot_6.6.0807lb # [ 0.167389] scmi_core: SCMI protocol bus registered808client # [ 0.210723] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)809worker2 # [ 0.173984] netfs: FS-Cache loaded810worker1 # [ 0.169636] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)811worker2 # [ 0.174105] pnp: PnP ACPI: disabled812client # [ 0.210808] NET: Registered PF_UNIX/PF_LOCAL protocol family813lb # [ 0.168311] FPGA manager framework814worker1 # [ 0.171886] netfs: FS-Cache loaded815client # [ 0.210831] NET: Registered PF_XDP protocol family816lb # [ 0.169265] vgaarb: loaded817worker1 # [ 0.172433] pnp: PnP ACPI: disabled818client # [ 0.210848] PCI: CLS 0 bytes, default 64819client # [ 0.211089] Trying to unpack rootfs image as initramfs...820lb # [ 0.169899] clocksource: Switched to clocksource arch_sys_counter821lb # [ 0.170485] VFS: Disk quotas dquot_6.6.0822lb # [ 0.170509] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)823worker2 # [ 0.178012] NET: Registered PF_INET protocol family824lb # [ 0.172877] netfs: FS-Cache loaded825lb # [ 0.172992] pnp: PnP ACPI: disabled826worker2 # [ 0.178272] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)827client # [ 0.228556] kvm [1]: HYP mode not available828worker1 # [ 0.177886] NET: Registered PF_INET protocol family829worker1 # [ 0.178146] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)830lb # [ 0.178952] NET: Registered PF_INET protocol family831lb # [ 0.179235] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832worker2 # [ 0.207719] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833worker2 # [ 0.207766] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834lb # [ 0.209881] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)835worker1 # [ 0.206883] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)836worker2 # [ 0.207797] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)837worker1 # [ 0.206932] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)838worker2 # [ 0.207866] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)839worker1 # [ 0.206960] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)840worker2 # [ 0.208008] TCP: Hash tables configured (established 16384 bind 16384)841worker1 # [ 0.207029] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)842worker2 # [ 0.208093] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)843worker1 # [ 0.207173] TCP: Hash tables configured (established 16384 bind 16384)844worker2 # [ 0.208128] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)845worker1 # [ 0.207272] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)846worker2 # [ 0.208164] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847worker2 # [ 0.208255] NET: Registered PF_UNIX/PF_LOCAL protocol family848worker1 # [ 0.207309] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)849worker2 # [ 0.208302] NET: Registered PF_XDP protocol family850worker1 # [ 0.207345] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)851worker2 # [ 0.208320] PCI: CLS 0 bytes, default 64852worker1 # [ 0.207439] NET: Registered PF_UNIX/PF_LOCAL protocol family853worker2 # [ 0.208571] Trying to unpack rootfs image as initramfs...854worker1 # [ 0.207463] NET: Registered PF_XDP protocol family855lb # [ 0.211147] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)856worker1 # [ 0.207480] PCI: CLS 0 bytes, default 64857worker1 # [ 0.207754] Trying to unpack rootfs image as initramfs...858lb # [ 0.211182] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)859lb # [ 0.211263] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)860worker2 # [ 0.224227] kvm [1]: HYP mode not available861lb # [ 0.211405] TCP: Hash tables configured (established 16384 bind 16384)862lb # [ 0.211497] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)863worker1 # [ 0.222668] kvm [1]: HYP mode not available864lb # [ 0.211540] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.211576] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866lb # [ 0.211670] NET: Registered PF_UNIX/PF_LOCAL protocol family867lb # [ 0.211736] NET: Registered PF_XDP protocol family868lb # [ 0.211754] PCI: CLS 0 bytes, default 64869lb # [ 0.212000] Trying to unpack rootfs image as initramfs...870lb # [ 0.233958] kvm [1]: HYP mode not available871client # [ 0.319781] Initialise system trusted keyrings872client # [ 0.320547] workingset: timestamp_bits=42 max_order=19 bucket_order=0873client # [ 0.321794] squashfs: version 4.0 (2009/01/31) Phillip Lougher874client # [ 0.322557] 9p: Installing v9fs 9p2000 file system support875client # [ 0.351313] Key type asymmetric registered876client # [ 0.351345] Asymmetric key parser 'x509' registered877client # [ 0.351450] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878client # [ 0.353562] io scheduler mq-deadline registered879client # [ 0.353573] io scheduler kyber registered880client # [ 0.363317] pl061_gpio 9030000.pl061: PL061 GPIO chip registered881client # [ 0.364755] ledtrig-cpu: registered to indicate activity on CPUs882client # [ 0.365154] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:883client # [ 0.365172] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000884client # [ 0.365185] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000885worker2 # [ 0.319234] Initialise system trusted keyrings886client # [ 0.365193] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000887worker2 # [ 0.320003] workingset: timestamp_bits=42 max_order=19 bucket_order=0888client # [ 0.365214] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits889worker2 # [ 0.321274] squashfs: version 4.0 (2009/01/31) Phillip Lougher890client # [ 0.365240] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]891worker2 # [ 0.322031] 9p: Installing v9fs 9p2000 file system support892client # [ 0.365317] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00893client # [ 0.365327] pci_bus 0000:00: root bus resource [bus 00-ff]894client # [ 0.365333] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]895client # [ 0.365338] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]896client # [ 0.365343] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]897client # [ 0.365401] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint898client # [ 0.365854] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint899client # [ 0.366036] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]900worker1 # [ 0.325623] Initialise system trusted keyrings901client # [ 0.366053] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]902worker1 # [ 0.326369] workingset: timestamp_bits=42 max_order=19 bucket_order=0903client # [ 0.366088] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]904client # [ 0.366116] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]905worker1 # [ 0.327642] squashfs: version 4.0 (2009/01/31) Phillip Lougher906client # [ 0.366895] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint907worker1 # [ 0.328423] 9p: Installing v9fs 9p2000 file system support908client # [ 0.367100] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]909client # [ 0.367117] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]910lb # [ 0.331504] Initialise system trusted keyrings911lb # [ 0.332252] workingset: timestamp_bits=42 max_order=19 bucket_order=0912lb # [ 0.333590] squashfs: version 4.0 (2009/01/31) Phillip Lougher913client # [ 0.391240] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]914lb # [ 0.334442] 9p: Installing v9fs 9p2000 file system support915client # [ 0.391807] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint916client # [ 0.392004] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]917client # [ 0.392020] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]918client # [ 0.392050] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]919worker2 # [ 0.350807] Key type asymmetric registered920worker2 # [ 0.350834] Asymmetric key parser 'x509' registered921client # [ 0.392498] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint922worker2 # [ 0.350915] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)923client # [ 0.392677] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]924worker2 # [ 0.353018] io scheduler mq-deadline registered925client # [ 0.392693] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]926worker2 # [ 0.353028] io scheduler kyber registered927client # [ 0.392725] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928client # [ 0.393194] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint929client # [ 0.393373] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]930client # [ 0.393389] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]931client # [ 0.393418] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]932client # [ 0.393862] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint933client # [ 0.394043] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]934client # [ 0.394058] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]935client # [ 0.394087] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]936worker1 # [ 0.357201] Key type asymmetric registered937lb # [ 0.354466] Key type asymmetric registered938worker1 # [ 0.357234] Asymmetric key parser 'x509' registered939lb # [ 0.354493] Asymmetric key parser 'x509' registered940client # [ 0.394549] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint941worker1 # [ 0.357345] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)942client # [ 0.394731] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]943lb # [ 0.354563] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)944worker1 # [ 0.359469] io scheduler mq-deadline registered945client # [ 0.394748] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]946worker1 # [ 0.359480] io scheduler kyber registered947client # [ 0.394777] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]948worker2 # [ 0.362809] pl061_gpio 9030000.pl061: PL061 GPIO chip registered949client # [ 0.394793] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]950client # [ 0.395305] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint951client # [ 0.395488] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]952worker2 # [ 0.364354] ledtrig-cpu: registered to indicate activity on CPUs953client # [ 0.395517] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]954worker2 # [ 0.364774] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:955client # [ 0.395990] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint956lb # [ 0.361959] io scheduler mq-deadline registered957client # [ 0.396177] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]958lb # [ 0.361977] io scheduler kyber registered959worker2 # [ 0.364795] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000960client # [ 0.396214] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]961worker2 # [ 0.364807] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000962client # [ 0.396608] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint963worker2 # [ 0.364816] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964client # [ 0.396793] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]965worker2 # [ 0.364838] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966client # [ 0.397042] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint967client # [ 0.397339] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]968worker2 # [ 0.364866] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]969worker1 # [ 0.369397] pl061_gpio 9030000.pl061: PL061 GPIO chip registered970client # [ 0.397355] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]971worker2 # [ 0.364961] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972client # [ 0.397385] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973worker2 # [ 0.364970] pci_bus 0000:00: root bus resource [bus 00-ff]974worker2 # [ 0.364976] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]975client # [ 0.397835] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint976client # [ 0.398025] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]977worker2 # [ 0.364981] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]978worker1 # [ 0.370826] ledtrig-cpu: registered to indicate activity on CPUs979client # [ 0.398040] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]980worker2 # [ 0.364986] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]981worker1 # [ 0.371232] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:982client # [ 0.398070] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker2 # [ 0.365044] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint984worker1 # [ 0.371251] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000985client # [ 0.398643] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned986worker2 # [ 0.365481] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint987client # [ 0.398655] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned988worker1 # [ 0.371264] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000989worker2 # [ 0.365668] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]990client # [ 0.398660] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned991worker2 # [ 0.365684] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]992worker1 # [ 0.371273] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000993lb # [ 0.374058] pl061_gpio 9030000.pl061: PL061 GPIO chip registered994client # [ 0.398705] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned995worker2 # [ 0.365714] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker1 # [ 0.371307] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits997worker2 # [ 0.365729] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]998client # [ 0.398751] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned999worker1 # [ 0.371332] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1000worker2 # [ 0.366185] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1001client # [ 0.398795] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1002worker1 # [ 0.371408] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001003worker2 # [ 0.366369] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1004worker1 # [ 0.371437] pci_bus 0000:00: root bus resource [bus 00-ff]1005client # [ 0.398840] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1006worker2 # [ 0.366384] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1007worker1 # [ 0.371453] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1008worker2 # [ 0.366413] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1009client # [ 0.398885] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1010worker1 # [ 0.371458] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1011client # [ 0.398931] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1012worker1 # [ 0.371463] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1013client # [ 0.398976] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1014worker1 # [ 0.371521] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1015client # [ 0.399022] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1016lb # [ 0.375397] ledtrig-cpu: registered to indicate activity on CPUs1017worker1 # [ 0.371976] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1018client # [ 0.399068] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1019worker1 # [ 0.372162] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1020lb # [ 0.375788] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1021worker1 # [ 0.372179] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1022client # [ 0.399133] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1023lb # [ 0.375805] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001024worker1 # [ 0.372209] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025lb # [ 0.375817] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001026worker1 # [ 0.372225] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1027worker1 # [ 0.372702] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1028lb # [ 0.375825] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001029worker1 # [ 0.372895] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1030lb # [ 0.375847] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1031worker1 # [ 0.372911] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1032worker1 # [ 0.372941] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033lb # [ 0.375874] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1034lb # [ 0.375971] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001035lb # [ 0.375981] pci_bus 0000:00: root bus resource [bus 00-ff]1036worker1 # [ 0.392178] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1037lb # [ 0.375987] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1038worker1 # [ 0.392364] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1039lb # [ 0.375992] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1040worker1 # [ 0.392380] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1041lb # [ 0.375998] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1042worker1 # [ 0.392409] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043worker2 # [ 0.390950] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1044client # [ 0.447243] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1045lb # [ 0.376054] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1046worker2 # [ 0.391148] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1047worker1 # [ 0.392861] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1048client # [ 0.447278] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1049worker2 # [ 0.391164] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1050lb # [ 0.376489] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051client # [ 0.447300] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1052lb # [ 0.376686] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1053worker2 # [ 0.391194] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054client # [ 0.447322] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1055lb # [ 0.376703] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1056worker2 # [ 0.391658] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1057client # [ 0.447348] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1058lb # [ 0.376733] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059worker2 # [ 0.391839] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1060client # [ 0.447371] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1061lb # [ 0.376749] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1062worker2 # [ 0.391854] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1063client # [ 0.447394] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1064worker2 # [ 0.391886] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065lb # [ 0.377210] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1066client # [ 0.447416] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1067lb # [ 0.377393] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1068worker2 # [ 0.392347] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1069client # [ 0.447438] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1070lb # [ 0.377408] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1071worker2 # [ 0.392532] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1072client # [ 0.447461] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1073lb # [ 0.377437] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker2 # [ 0.392547] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1075client # [ 0.447485] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1076worker2 # [ 0.392577] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.377936] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1078client # [ 0.447509] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1079lb # [ 0.378125] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1080worker2 # [ 0.393031] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1081client # [ 0.447531] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1082lb # [ 0.378152] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1083worker2 # [ 0.393213] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1084client # [ 0.447552] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1085lb # [ 0.378187] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker2 # [ 0.393229] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1087client # [ 0.447573] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1088worker2 # [ 0.393258] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089lb # [ 0.378670] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1090client # [ 0.447594] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1091lb # [ 0.378854] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1092worker2 # [ 0.393743] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1093client # [ 0.447615] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1094lb # [ 0.378869] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.393930] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1096client # [ 0.447637] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1097lb # [ 0.378901] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker2 # [ 0.393945] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1099client # [ 0.447659] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1100worker2 # [ 0.393975] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101lb # [ 0.379368] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102client # [ 0.447680] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1103lb # [ 0.379550] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1104worker2 # [ 0.393990] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1105client # [ 0.447702] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1106lb # [ 0.379565] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1107client # [ 0.447730] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1108worker2 # [ 0.394457] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1109lb # [ 0.379595] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110worker2 # [ 0.394661] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1111client # [ 0.447740] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1112worker1 # [ 0.405109] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1113worker2 # [ 0.394694] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114client # [ 0.447745] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1115lb # [ 0.380049] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1116worker1 # [ 0.405135] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1117lb # [ 0.380230] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1118client # [ 0.448916] pci 0000:00:0a.0: enabling device (0000 -> 0002)1119worker2 # [ 0.395151] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1120worker1 # [ 0.405168] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121lb # [ 0.380246] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker2 # [ 0.395339] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1123lb # [ 0.380275] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124worker1 # [ 0.405657] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125worker2 # [ 0.395368] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1126worker1 # [ 0.405846] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1127lb # [ 0.380771] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1128worker2 # [ 0.395775] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1129worker1 # [ 0.405862] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.380955] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1131worker2 # [ 0.395957] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1132worker1 # [ 0.405892] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133lb # [ 0.380970] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1134worker2 # [ 0.396204] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1135lb # [ 0.381000] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136worker1 # [ 0.406345] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1137worker2 # [ 0.396476] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1138lb # [ 0.381016] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1139worker1 # [ 0.406528] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1140worker2 # [ 0.396494] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1141worker1 # [ 0.406543] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1142lb # [ 0.381470] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1143worker2 # [ 0.396522] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144worker1 # [ 0.406573] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145lb # [ 0.381658] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1146worker2 # [ 0.396983] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1147worker1 # [ 0.407070] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1148lb # [ 0.381688] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker2 # [ 0.397169] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1150worker1 # [ 0.407254] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1151worker2 # [ 0.397185] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1152worker1 # [ 0.407270] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1153worker2 # [ 0.397214] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154worker1 # [ 0.407299] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155worker2 # [ 0.397786] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1156worker1 # [ 0.407315] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1157worker2 # [ 0.397798] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1158worker1 # [ 0.407795] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1159worker2 # [ 0.397804] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1160worker1 # [ 0.407986] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker1 # [ 0.408015] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162worker2 # [ 0.397851] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1163worker2 # [ 0.397900] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1164worker1 # [ 0.408470] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1165worker1 # [ 0.408660] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker2 # [ 0.397947] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1167worker1 # [ 0.408690] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker2 # [ 0.397994] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1169worker1 # [ 0.409116] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1170worker2 # [ 0.398040] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1171worker1 # [ 0.409298] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.398086] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1173worker1 # [ 0.409552] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1174worker2 # [ 0.398133] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1175worker1 # [ 0.409825] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1176lb # [ 0.426210] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1177worker1 # [ 0.409843] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1178lb # [ 0.426419] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1179worker2 # [ 0.398179] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1180worker1 # [ 0.409877] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181lb # [ 0.426449] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182worker2 # [ 0.398225] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1183worker1 # [ 0.410343] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1184lb # [ 0.426850] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1185worker2 # [ 0.398288] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1186worker1 # [ 0.410525] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1187lb # [ 0.427032] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1188worker2 # [ 0.398333] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1189worker1 # [ 0.410541] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1190lb # [ 0.427288] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1191worker2 # [ 0.398355] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1192worker1 # [ 0.410571] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1193lb # [ 0.427569] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1194worker2 # [ 0.398376] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1195worker1 # [ 0.411146] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1196lb # [ 0.427587] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1197worker2 # [ 0.398399] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1198worker1 # [ 0.411158] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1199lb # [ 0.427616] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200worker2 # [ 0.398423] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1201worker1 # [ 0.411164] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1202lb # [ 0.428073] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1203worker2 # [ 0.398444] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1204client # [ 0.494097] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1205lb # [ 0.428257] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1206worker1 # [ 0.411208] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1207worker2 # [ 0.398465] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1208lb # [ 0.428273] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker1 # [ 0.411255] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1210worker2 # [ 0.398486] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1211lb # [ 0.428302] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker2 # [ 0.398508] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1213worker1 # [ 0.411302] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214lb # [ 0.428916] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1215worker2 # [ 0.398531] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1216worker1 # [ 0.411348] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217lb # [ 0.428928] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1218worker2 # [ 0.398554] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1219client # [ 0.497069] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1220worker1 # [ 0.411394] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1221lb # [ 0.428934] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1222worker2 # [ 0.398577] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1223worker1 # [ 0.411440] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1224worker2 # [ 0.398598] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1225lb # [ 0.428986] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1226worker1 # [ 0.411491] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1227lb # [ 0.429033] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1228worker1 # [ 0.411539] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1229lb # [ 0.429079] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1230worker1 # [ 0.411599] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1231lb # [ 0.429126] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1232worker1 # [ 0.411671] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1233lb # [ 0.429173] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1234worker1 # [ 0.411717] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1235lb # [ 0.429232] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1236worker1 # [ 0.411739] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1237lb # [ 0.429287] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1238worker1 # [ 0.411760] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1239lb # [ 0.429333] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1240worker1 # [ 0.411782] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1241worker1 # [ 0.411806] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1242lb # [ 0.429390] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1243worker1 # [ 0.411828] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1244worker2 # [ 0.454665] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1245lb # [ 0.429476] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1246worker2 # [ 0.454712] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1247worker1 # [ 0.411850] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1248lb # [ 0.429523] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1249worker2 # [ 0.454737] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1250worker1 # [ 0.411872] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1251lb # [ 0.429545] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1252worker2 # [ 0.454771] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1253worker1 # [ 0.411894] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254lb # [ 0.429565] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1255worker2 # [ 0.454794] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1256worker1 # [ 0.411916] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1257lb # [ 0.429587] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1258worker2 # [ 0.454817] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1259worker1 # [ 0.411938] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1260lb # [ 0.429612] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1261worker2 # [ 0.454848] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1262worker1 # [ 0.411962] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1263lb # [ 0.429634] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1264worker2 # [ 0.454870] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1265worker1 # [ 0.411983] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1266worker2 # [ 0.454901] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1267lb # [ 0.429656] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1268client # [ 0.507224] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1269worker1 # [ 0.412005] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1270worker2 # [ 0.454912] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1271lb # [ 0.429679] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1272client # [ 0.509133] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1273worker1 # [ 0.412027] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1274worker2 # [ 0.454917] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1275client # [ 0.510984] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1276lb # [ 0.429700] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1277worker1 # [ 0.412049] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1278worker2 # [ 0.456146] pci 0000:00:0a.0: enabling device (0000 -> 0002)1279lb # [ 0.429722] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1280worker1 # [ 0.412070] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1281lb # [ 0.429745] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1282worker1 # [ 0.412091] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1283client # [ 0.513078] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1284lb # [ 0.429767] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1285worker1 # [ 0.412113] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1286lb # [ 0.429789] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1287worker1 # [ 0.412135] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1288lb # [ 0.429810] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1289worker1 # [ 0.412157] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1290worker1 # [ 0.412185] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1291lb # [ 0.429831] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1292worker1 # [ 0.412195] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1293lb # [ 0.429852] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1294lb # [ 0.429873] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1295worker1 # [ 0.412199] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1296lb # [ 0.429894] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1297client # [ 0.523335] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1298client # [ 0.525365] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1299worker1 # [ 0.477459] pci 0000:00:0a.0: enabling device (0000 -> 0002)1300lb # [ 0.474039] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1301lb # [ 0.474066] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1302lb # [ 0.474089] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1303lb # [ 0.474122] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1304lb # [ 0.474132] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1305lb # [ 0.474137] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1306client # [ 0.535317] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1307lb # [ 0.475334] pci 0000:00:0a.0: enabling device (0000 -> 0002)1308client # [ 0.537796] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1309worker2 # [ 0.490137] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310client # [ 0.549276] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1311worker2 # [ 0.500052] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1312worker1 # [ 0.498656] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1313worker1 # [ 0.500916] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1314worker2 # [ 0.504408] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker2 # [ 0.506462] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316lb # [ 0.501533] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1317client # [ 0.560426] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1318client # [ 0.563085] msm_serial: driver initialized1319client # [ 0.563784] SuperH (H)SCI(F) driver initialized1320client # [ 0.563845] STM32 USART driver initialized1321worker1 # [ 0.511757] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1322lb # [ 0.511383] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1323worker2 # [ 0.520241] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1324worker2 # [ 0.522246] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1325worker2 # [ 0.525972] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1326lb # [ 0.515784] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1327lb # [ 0.517748] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1328lb # [ 0.519806] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1329worker1 # [ 0.521699] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1330worker1 # [ 0.525381] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1331worker1 # [ 0.527335] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1332worker2 # [ 0.534375] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1333lb # [ 0.530220] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1334lb # [ 0.532283] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1335worker1 # [ 0.537632] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336worker1 # [ 0.540063] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337client # [ 0.596360] loop: module loaded1338client # [ 0.596547] virtio_blk virtio5: 1/0/0 default/read/poll queues1339worker2 # [ 0.544728] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340client # [ 0.597282] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1341worker2 # [ 0.547736] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1342worker2 # [ 0.550027] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1343lb # [ 0.542152] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1344lb # [ 0.545214] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1345worker1 # [ 0.550496] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346worker1 # [ 0.552327] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1347client # [ 0.607800] megasas: 07.734.00.00-rc11348client # [ 0.608567] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1349worker1 # [ 0.556393] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1350client # [ 0.610644] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001351client # [ 0.610672] Intel/Sharp Extended Query Table at 0x00311352lb # [ 0.556320] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1353lb # [ 0.559585] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1354client # [ 0.615642] Using buffer write method1355worker2 # [ 0.567837] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1356client # [ 0.615719] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1357client # [ 0.617540] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001358worker2 # [ 0.570466] msm_serial: driver initialized1359client # [ 0.617599] Intel/Sharp Extended Query Table at 0x00311360worker2 # [ 0.571242] SuperH (H)SCI(F) driver initialized1361worker2 # [ 0.571306] STM32 USART driver initialized1362worker1 # [ 0.574468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363lb # [ 0.575201] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1364client # [ 0.635205] Using buffer write method1365client # [ 0.635239] Concatenating MTD devices:1366worker1 # [ 0.577341] msm_serial: driver initialized1367client # [ 0.635243] (0): "0.flash"1368worker1 # [ 0.577485] SuperH (H)SCI(F) driver initialized1369client # [ 0.635247] (1): "0.flash"1370worker1 # [ 0.577540] STM32 USART driver initialized1371client # [ 0.635251] into device "0.flash"1372lb # [ 0.577979] msm_serial: driver initialized1373lb # [ 0.578132] SuperH (H)SCI(F) driver initialized1374lb # [ 0.578188] STM32 USART driver initialized1375worker2 # [ 0.604492] loop: module loaded1376worker2 # [ 0.604677] virtio_blk virtio5: 1/0/0 default/read/poll queues1377worker2 # [ 0.605441] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1378worker2 # [ 0.611303] megasas: 07.734.00.00-rc11379worker2 # [ 0.611991] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1380worker1 # [ 0.612128] loop: module loaded1381worker1 # [ 0.612322] virtio_blk virtio5: 1/0/0 default/read/poll queues1382worker2 # [ 0.614296] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001383worker2 # [ 0.614320] Intel/Sharp Extended Query Table at 0x00311384lb # [ 0.610899] loop: module loaded1385worker1 # [ 0.614220] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386lb # [ 0.611088] virtio_blk virtio5: 1/0/0 default/read/poll queues1387lb # [ 0.611849] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1388worker1 # [ 0.617629] megasas: 07.734.00.00-rc11389worker1 # [ 0.618337] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1390worker2 # [ 0.623727] Using buffer write method1391worker2 # [ 0.623802] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1392lb # [ 0.622650] megasas: 07.734.00.00-rc11393lb # [ 0.623489] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394worker1 # [ 0.629111] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395worker2 # [ 0.634651] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001396worker1 # [ 0.629139] Intel/Sharp Extended Query Table at 0x00311397worker2 # [ 0.634710] Intel/Sharp Extended Query Table at 0x00311398lb # [ 0.625362] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001399worker1 # [ 0.630879] Using buffer write method1400worker2 # [ 0.636243] Using buffer write method1401lb # [ 0.625396] Intel/Sharp Extended Query Table at 0x00311402worker2 # [ 0.636272] Concatenating MTD devices:1403worker2 # [ 0.636276] (0): "0.flash"1404lb # [ 0.627010] Using buffer write method1405worker1 # [ 0.630951] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1406worker2 # [ 0.636280] (1): "0.flash"1407worker2 # [ 0.636283] into device "0.flash"1408worker1 # [ 0.632675] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409lb # [ 0.627070] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1410worker1 # [ 0.632707] Intel/Sharp Extended Query Table at 0x00311411lb # [ 0.628872] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412lb # [ 0.628909] Intel/Sharp Extended Query Table at 0x00311413worker1 # [ 0.646241] Using buffer write method1414worker1 # [ 0.646329] Concatenating MTD devices:1415worker1 # [ 0.646347] (0): "0.flash"1416worker1 # [ 0.646354] (1): "0.flash"1417worker1 # [ 0.646357] into device "0.flash"1418lb # [ 0.648530] Using buffer write method1419lb # [ 0.648593] Concatenating MTD devices:1420lb # [ 0.648610] (0): "0.flash"1421lb # [ 0.648615] (1): "0.flash"1422lb # [ 0.648618] into device "0.flash"1423client # [ 0.889170] Freeing initrd memory: 26160K1424client # [ 0.896051] tun: Universal TUN/TAP device driver, 1.61425client # [ 0.899975] thunder_xcv, ver 1.01426client # [ 0.900019] thunder_bgx, ver 1.01427client # [ 0.900044] nicpf, ver 1.01428client # [ 0.900597] e1000: Intel(R) PRO/1000 Network Driver1429client # [ 0.900606] e1000: Copyright (c) 1999-2006 Intel Corporation.1430client # [ 0.900633] e1000e: Intel(R) PRO/1000 Network Driver1431client # [ 0.900641] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432client # [ 0.900676] igb: Intel(R) Gigabit Ethernet Network Driver1433client # [ 0.900681] igb: Copyright (c) 2007-2014 Intel Corporation.1434client # [ 0.900704] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435client # [ 0.900710] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436client # [ 0.900841] sky2: driver version 1.301437client # [ 0.902545] usbcore: registered new interface driver usb-storage1438client # [ 0.902607] usbcore: registered new interface driver usbserial_generic1439client # [ 0.902624] usbserial: USB Serial support registered for generic1440client # [ 0.903921] ehci-pci 0000:00:0a.0: EHCI Host Controller1441client # [ 0.903957] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11442client # [ 0.904253] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001443client # [ 0.915924] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001444client # [ 0.916284] hub 1-0:1.0: USB hub found1445client # [ 0.916301] hub 1-0:1.0: 6 ports detected1446client # [ 0.918450] hv_vmbus: registering driver hyperv_keyboard1447client # [ 0.920098] rtc-pl031 9010000.pl031: registered as rtc01448client # [ 0.920126] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T10:42:09 UTC (1789987329)1449client # [ 0.920434] i2c_dev: i2c /dev entries driver1450client # [ 0.925698] sdhci: Secure Digital Host Controller Interface driver1451client # [ 0.925714] sdhci: Copyright(c) Pierre Ossman1452client # [ 0.926001] Synopsys Designware Multimedia Card Interface Driver1453client # [ 0.926382] sdhci-pltfm: SDHCI platform and OF driver helper1454client # [ 0.930943] hid: raw HID events driver (C) Jiri Kosina1455client # [ 0.931882] usbcore: registered new interface driver usbhid1456client # [ 0.931892] usbhid: USB HID core driver1457client # [ 0.934253] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1458client # [ 0.937026] drop_monitor: Initializing network drop monitor service1459client # [ 0.937265] NET: Registered PF_INET6 protocol family1460client # [ 0.939445] Segment Routing with IPv61461client # [ 0.939468] In-situ OAM (IOAM) with IPv61462client # [ 0.939501] NET: Registered PF_PACKET protocol family1463worker2 # [ 0.888933] Freeing initrd memory: 26148K1464client # [ 0.941107] 9pnet: Installing 9P2000 support1465worker2 # [ 0.895332] tun: Universal TUN/TAP device driver, 1.61466client # [ 0.943833] Key type dns_resolver registered1467worker1 # [ 0.895795] Freeing initrd memory: 26148K1468client # [ 0.950477] registered taskstats version 11469client # [ 0.950669] Loading compiled-in X.509 certificates1470worker2 # [ 0.899145] thunder_xcv, ver 1.01471worker2 # [ 0.899191] thunder_bgx, ver 1.01472worker2 # [ 0.899215] nicpf, ver 1.01473worker2 # [ 0.899870] e1000: Intel(R) PRO/1000 Network Driver1474lb # [ 0.899093] Freeing initrd memory: 26140K1475worker1 # [ 0.902379] tun: Universal TUN/TAP device driver, 1.61476worker2 # [ 0.899881] e1000: Copyright (c) 1999-2006 Intel Corporation.1477worker2 # [ 0.899907] e1000e: Intel(R) PRO/1000 Network Driver1478worker2 # [ 0.899914] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1479worker2 # [ 0.899943] igb: Intel(R) Gigabit Ethernet Network Driver1480worker2 # [ 0.899948] igb: Copyright (c) 2007-2014 Intel Corporation.1481client # [ 0.959560] Demotion targets for Node 0: null1482client # [ 0.959683] Key type .fscrypt registered1483worker2 # [ 0.899971] igbvf: Intel(R) Gigabit Virtual Function Network Driver1484client # [ 0.959689] Key type fscrypt-provisioning registered1485worker2 # [ 0.899976] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1486client # [ 0.959793] ima: No TPM chip found, activating TPM-bypass!1487worker2 # [ 0.900125] sky2: driver version 1.301488client # [ 0.959819] ima: Allocated hash algorithm: sha11489worker2 # [ 0.901788] usbcore: registered new interface driver usb-storage1490client # [ 0.959845] ima: No architecture policies found1491worker2 # [ 0.901851] usbcore: registered new interface driver usbserial_generic1492lb # [ 0.905667] tun: Universal TUN/TAP device driver, 1.61493worker2 # [ 0.901868] usbserial: USB Serial support registered for generic1494client # [ 0.964107] input: gpio-keys as /devices/platform/gpio-keys/input/input01495worker1 # [ 0.906223] thunder_xcv, ver 1.01496worker2 # [ 0.902493] hv_vmbus: registering driver hyperv_keyboard1497worker1 # [ 0.906269] thunder_bgx, ver 1.01498worker1 # [ 0.906295] nicpf, ver 1.01499worker2 # [ 0.903480] ehci-pci 0000:00:0a.0: EHCI Host Controller1500worker1 # [ 0.906831] e1000: Intel(R) PRO/1000 Network Driver1501worker2 # [ 0.903518] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11502worker1 # [ 0.906840] e1000: Copyright (c) 1999-2006 Intel Corporation.1503worker2 # [ 0.903801] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001504worker1 # [ 0.906864] e1000e: Intel(R) PRO/1000 Network Driver1505worker1 # [ 0.906871] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1506worker2 # [ 0.916043] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001507worker1 # [ 0.906906] igb: Intel(R) Gigabit Ethernet Network Driver1508worker2 # [ 0.916436] hub 1-0:1.0: USB hub found1509worker1 # [ 0.906914] igb: Copyright (c) 2007-2014 Intel Corporation.1510worker2 # [ 0.916468] hub 1-0:1.0: 6 ports detected1511worker1 # [ 0.906936] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512worker1 # [ 0.906944] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1513worker1 # [ 0.907072] sky2: driver version 1.301514worker1 # [ 0.908721] usbcore: registered new interface driver usb-storage1515worker2 # [ 0.919443] rtc-pl031 9010000.pl031: registered as rtc01516lb # [ 0.910004] thunder_xcv, ver 1.01517lb # [ 0.910060] thunder_bgx, ver 1.01518worker1 # [ 0.908820] usbcore: registered new interface driver usbserial_generic1519lb # [ 0.910087] nicpf, ver 1.01520worker2 # [ 0.919473] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T10:42:09 UTC (1789987329)1521worker1 # [ 0.908835] usbserial: USB Serial support registered for generic1522lb # [ 0.910878] e1000: Intel(R) PRO/1000 Network Driver1523worker2 # [ 0.919783] i2c_dev: i2c /dev entries driver1524worker1 # [ 0.909818] ehci-pci 0000:00:0a.0: EHCI Host Controller1525lb # [ 0.910888] e1000: Copyright (c) 1999-2006 Intel Corporation.1526lb # [ 0.910913] e1000e: Intel(R) PRO/1000 Network Driver1527worker1 # [ 0.909850] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11528lb # [ 0.910920] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1529worker1 # [ 0.910021] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001530lb # [ 0.910954] igb: Intel(R) Gigabit Ethernet Network Driver1531lb # [ 0.910960] igb: Copyright (c) 2007-2014 Intel Corporation.1532worker1 # [ 0.921792] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001533lb # [ 0.910985] igbvf: Intel(R) Gigabit Virtual Function Network Driver1534worker1 # [ 0.922203] hub 1-0:1.0: USB hub found1535worker1 # [ 0.922225] hub 1-0:1.0: 6 ports detected1536worker2 # [ 0.924977] sdhci: Secure Digital Host Controller Interface driver1537lb # [ 0.910990] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1538worker2 # [ 0.924992] sdhci: Copyright(c) Pierre Ossman1539lb # [ 0.911130] sky2: driver version 1.301540worker2 # [ 0.925274] Synopsys Designware Multimedia Card Interface Driver1541lb # [ 0.912971] usbcore: registered new interface driver usb-storage1542worker1 # [ 0.924761] hv_vmbus: registering driver hyperv_keyboard1543worker2 # [ 0.925671] sdhci-pltfm: SDHCI platform and OF driver helper1544lb # [ 0.913067] usbcore: registered new interface driver usbserial_generic1545lb # [ 0.913083] usbserial: USB Serial support registered for generic1546lb # [ 0.913736] hv_vmbus: registering driver hyperv_keyboard1547worker2 # [ 0.930374] hid: raw HID events driver (C) Jiri Kosina1548worker1 # [ 0.926365] rtc-pl031 9010000.pl031: registered as rtc01549lb # [ 0.914542] ehci-pci 0000:00:0a.0: EHCI Host Controller1550client # [ 0.982861] clk: Disabling unused clocks1551worker1 # [ 0.926401] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T10:42:09 UTC (1789987329)1552lb # [ 0.914572] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11553client # [ 0.982884] PM: genpd: Disabling unused power domains1554worker1 # [ 0.926718] i2c_dev: i2c /dev entries driver1555worker2 # [ 0.931312] usbcore: registered new interface driver usbhid1556worker2 # [ 0.931323] usbhid: USB HID core driver1557lb # [ 0.914772] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001558lb # [ 0.926540] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001559lb # [ 0.926903] hub 1-0:1.0: USB hub found1560lb # [ 0.926924] hub 1-0:1.0: 6 ports detected1561worker2 # [ 0.933605] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1562worker1 # [ 0.931818] sdhci: Secure Digital Host Controller Interface driver1563worker2 # [ 0.936312] drop_monitor: Initializing network drop monitor service1564worker1 # [ 0.931834] sdhci: Copyright(c) Pierre Ossman1565client # [ 0.988531] Freeing unused kernel memory: 4736K1566worker2 # [ 0.936460] NET: Registered PF_INET6 protocol family1567worker1 # [ 0.932119] Synopsys Designware Multimedia Card Interface Driver1568client # [ 0.988727] Run /init as init process1569lb # [ 0.930607] rtc-pl031 9010000.pl031: registered as rtc01570worker1 # [ 0.932493] sdhci-pltfm: SDHCI platform and OF driver helper1571worker2 # [ 0.938502] Segment Routing with IPv61572lb # [ 0.930651] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T10:42:09 UTC (1789987329)1573worker2 # [ 0.938523] In-situ OAM (IOAM) with IPv61574lb # [ 0.930956] i2c_dev: i2c /dev entries driver1575worker2 # [ 0.938583] NET: Registered PF_PACKET protocol family1576worker1 # [ 0.936981] hid: raw HID events driver (C) Jiri Kosina1577worker1 # [ 0.938023] usbcore: registered new interface driver usbhid1578worker2 # [ 0.940429] 9pnet: Installing 9P2000 support1579worker1 # [ 0.938037] usbhid: USB HID core driver1580worker2 # [ 0.942545] Key type dns_resolver registered1581lb # [ 0.936179] sdhci: Secure Digital Host Controller Interface driver1582worker1 # [ 0.940389] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1583lb # [ 0.936194] sdhci: Copyright(c) Pierre Ossman1584lb # [ 0.936470] Synopsys Designware Multimedia Card Interface Driver1585lb # [ 0.936870] sdhci-pltfm: SDHCI platform and OF driver helper1586worker1 # [ 0.943063] drop_monitor: Initializing network drop monitor service1587worker1 # [ 0.943238] NET: Registered PF_INET6 protocol family1588lb # [ 0.941635] hid: raw HID events driver (C) Jiri Kosina1589worker1 # [ 0.945448] Segment Routing with IPv61590worker1 # [ 0.945474] In-situ OAM (IOAM) with IPv61591worker1 # [ 0.945523] NET: Registered PF_PACKET protocol family1592lb # [ 0.942558] usbcore: registered new interface driver usbhid1593lb # [ 0.942569] usbhid: USB HID core driver1594worker2 # [ 0.950216] registered taskstats version 11595worker2 # [ 0.950395] Loading compiled-in X.509 certificates1596worker1 # [ 0.947175] 9pnet: Installing 9P2000 support1597lb # [ 0.944916] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1598client # [ 1.005512] systemd[1]: Successfully made /usr/ read-only.1599lb # [ 0.947632] drop_monitor: Initializing network drop monitor service1600lb # [ 0.947804] NET: Registered PF_INET6 protocol family1601worker1 # [ 0.949989] Key type dns_resolver registered1602lb # [ 0.950140] Segment Routing with IPv61603lb # [ 0.950162] In-situ OAM (IOAM) with IPv61604lb # [ 0.950208] NET: Registered PF_PACKET protocol family1605lb # [ 0.951886] 9pnet: Installing 9P2000 support1606worker2 # [ 0.959734] Demotion targets for Node 0: null1607worker2 # [ 0.959849] Key type .fscrypt registered1608worker2 # [ 0.959855] Key type fscrypt-provisioning registered1609worker2 # [ 0.959960] ima: No TPM chip found, activating TPM-bypass!1610worker2 # [ 0.959980] ima: Allocated hash algorithm: sha11611worker2 # [ 0.960014] ima: No architecture policies found1612lb # [ 0.954768] Key type dns_resolver registered1613worker1 # [ 0.957347] registered taskstats version 11614worker2 # [ 0.964400] input: gpio-keys as /devices/platform/gpio-keys/input/input01615worker1 # [ 0.957537] Loading compiled-in X.509 certificates1616worker1 # [ 0.966428] Demotion targets for Node 0: null1617worker1 # [ 0.966554] Key type .fscrypt registered1618worker1 # [ 0.966560] Key type fscrypt-provisioning registered1619worker1 # [ 0.966659] ima: No TPM chip found, activating TPM-bypass!1620lb # [ 0.962149] registered taskstats version 11621worker1 # [ 0.966678] ima: Allocated hash algorithm: sha11622lb # [ 0.962359] Loading compiled-in X.509 certificates1623worker1 # [ 0.966699] ima: No architecture policies found1624worker1 # [ 0.970759] input: gpio-keys as /devices/platform/gpio-keys/input/input01625lb # [ 0.971717] Demotion targets for Node 0: null1626lb # [ 0.971867] Key type .fscrypt registered1627lb # [ 0.971877] Key type fscrypt-provisioning registered1628lb # [ 0.971994] ima: No TPM chip found, activating TPM-bypass!1629lb # [ 0.972016] ima: Allocated hash algorithm: sha11630lb # [ 0.972039] ima: No architecture policies found1631lb # [ 0.976087] input: gpio-keys as /devices/platform/gpio-keys/input/input01632worker2 # [ 0.982588] clk: Disabling unused clocks1633worker2 # [ 0.983183] PM: genpd: Disabling unused power domains1634worker2 # [ 0.987000] Freeing unused kernel memory: 4736K1635worker2 # [ 0.987223] Run /init as init process1636worker1 # [ 0.989110] clk: Disabling unused clocks1637worker1 # [ 0.989135] PM: genpd: Disabling unused power domains1638worker1 # [ 0.993563] Freeing unused kernel memory: 4736K1639worker1 # [ 0.993871] Run /init as init process1640lb # [ 0.995096] clk: Disabling unused clocks1641lb # [ 0.995128] PM: genpd: Disabling unused power domains1642worker2 # [ 1.005003] systemd[1]: Successfully made /usr/ read-only.1643lb # [ 0.999568] Freeing unused kernel memory: 4736K1644lb # [ 0.999764] Run /init as init process1645worker1 # [ 1.011440] systemd[1]: Successfully made /usr/ read-only.1646lb # [ 1.016266] systemd[1]: Successfully made /usr/ read-only.1647client # [ 1.163210] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker2 # [ 1.162716] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.169093] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.173982] usb 1-1: new high-speed USB device number 2 using ehci-pci1651client # [ 1.315452] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652client # [ 1.340364] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1653client # [ 1.346691] systemd[1]: Detected virtualization qemu.1654client # [ 1.347740] systemd[1]: Detected architecture arm64.1655client # [ 1.348730] systemd[1]: Running in initrd.1656client # [ 1.350116] systemd[1]: Initializing machine ID from random generator.1657client # [ 1.351543] systemd[1]: Hostname set to <client>.1658worker2 # [ 1.315148] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11659worker1 # [ 1.325653] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660lb # [ 1.328611] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661worker2 # [ 1.339597] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662worker2 # [ 1.351839] systemd[1]: Detected virtualization qemu.1663worker1 # [ 1.346225] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1664client # [ 1.403427] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01665worker1 # [ 1.351596] systemd[1]: Detected virtualization qemu.1666worker2 # [ 1.353922] systemd[1]: Detected architecture arm64.1667worker1 # [ 1.352592] systemd[1]: Detected architecture arm64.1668worker1 # [ 1.353557] systemd[1]: Running in initrd.1669worker2 # [ 1.355955] systemd[1]: Running in initrd.1670worker1 # [ 1.354925] systemd[1]: Initializing machine ID from random generator.1671worker1 # [ 1.356302] systemd[1]: Hostname set to <worker1>.1672worker2 # [ 1.358800] systemd[1]: Initializing machine ID from random generator.1673worker2 # [ 1.361866] systemd[1]: Hostname set to <worker2>.1674lb # [ 1.350978] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1675lb # [ 1.357941] systemd[1]: Detected virtualization qemu.1676lb # [ 1.359022] systemd[1]: Detected architecture arm64.1677lb # [ 1.360120] systemd[1]: Running in initrd.1678lb # [ 1.361705] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.363381] systemd[1]: Hostname set to <lb>.1680worker1 # [ 1.405349] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker2 # [ 1.407113] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.414152] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683client # [ 1.527211] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker1 # [ 1.525086] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker2 # [ 1.534649] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.533985] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker1 # [ 1.614302] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688client # [ 1.673519] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689client # [ 1.688628] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690client # [ 1.693646] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691lb # [ 1.651257] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker1 # [ 1.677156] systemd[1]: Queued start job for default target Initrd Default Target.1693worker1 # [ 1.680565] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21694worker1 # [ 1.682563] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01695worker2 # [ 1.687836] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696worker1 # [ 1.687286] systemd[1]: Created slice Slice /system/modprobe.1697worker1 # [ 1.688498] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1698worker1 # [ 1.689705] systemd[1]: Expecting device /dev/disk/by-label/nixos...1699worker1 # [ 1.689738] systemd[1]: Reached target Path Units.1700worker1 # [ 1.689756] systemd[1]: Reached target Slice Units.1701worker1 # [ 1.689773] systemd[1]: Reached target Swaps.1702worker1 # [ 1.689790] systemd[1]: Reached target Timer Units.1703worker1 # [ 1.689958] systemd[1]: Listening on D-Bus System Message Bus Socket.1704worker1 # [ 1.690087] systemd[1]: Listening on Journal Socket (/dev/log).1705worker1 # [ 1.690209] systemd[1]: Listening on Journal Sockets.1706worker1 # [ 1.690318] systemd[1]: Listening on udev Control Socket.1707worker1 # [ 1.690420] systemd[1]: Listening on udev Kernel Socket.1708worker1 # [ 1.690439] systemd[1]: Reached target Socket Units.1709lb # [ 1.693267] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21710worker2 # [ 1.698179] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21711worker1 # [ 1.699315] systemd[1]: Starting Create List of Static Device Nodes...1712lb # [ 1.698169] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01713worker2 # [ 1.706577] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01714worker1 # [ 1.708858] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1715worker1 # [ 1.710067] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1716worker1 # [ 1.725869] systemd[1]: Mounting Kernel Configuration File System...1717worker1 # [ 1.741261] systemd[1]: Starting Journal Service...1718client # [ 1.796956] systemd[1]: Queued start job for default target Initrd Default Target.1719worker1 # [ 1.743680] systemd[1]: Starting Load Kernel Modules...1720worker1 # [ 1.744465] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1721client # [ 1.805774] systemd[1]: Created slice Slice /system/modprobe.1722client # [ 1.806911] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1723client # [ 1.808333] systemd[1]: Expecting device /dev/disk/by-label/nixos...1724client # [ 1.809313] systemd[1]: Reached target Path Units.1725client # [ 1.810064] systemd[1]: Reached target Slice Units.1726client # [ 1.810838] systemd[1]: Reached target Swaps.1727client # [ 1.811594] systemd[1]: Reached target Timer Units.1728worker1 # [ 1.757412] systemd[1]: Starting Coldplug All udev Devices...1729client # [ 1.812539] systemd[1]: Listening on D-Bus System Message Bus Socket.1730client # [ 1.813686] systemd[1]: Listening on Journal Socket (/dev/log).1731client # [ 1.814757] systemd[1]: Listening on Journal Sockets.1732client # [ 1.815794] systemd[1]: Listening on udev Control Socket.1733client # [ 1.815919] systemd[1]: Listening on udev Kernel Socket.1734client # [ 1.815940] systemd[1]: Reached target Socket Units.1735lb # [ 1.761825] systemd[1]: Queued start job for default target Initrd Default Target.1736client # [ 1.820023] systemd[1]: Starting Create List of Static Device Nodes...1737client # [ 1.830381] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1738lb # [ 1.773234] systemd[1]: Created slice Slice /system/modprobe.1739client # [ 1.831650] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740lb # [ 1.774745] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1741lb # [ 1.776178] systemd[1]: Expecting device /dev/disk/by-label/nixos...1742worker1 # [ 1.777170] systemd[1]: Finished Create List of Static Device Nodes.1743lb # [ 1.777321] systemd[1]: Reached target Path Units.1744worker1 # [ 1.778132] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1745lb # [ 1.778300] systemd[1]: Reached target Slice Units.1746worker1 # [ 1.778424] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1747lb # [ 1.779211] systemd[1]: Reached target Swaps.1748worker1 # [ 1.778659] systemd[1]: Mounted Kernel Configuration File System.1749lb # [ 1.780026] systemd[1]: Reached target Timer Units.1750lb # [ 1.781285] systemd[1]: Listening on D-Bus System Message Bus Socket.1751lb # [ 1.782740] systemd[1]: Listening on Journal Socket (/dev/log).1752lb # [ 1.783998] systemd[1]: Listening on Journal Sockets.1753lb # [ 1.785108] systemd[1]: Listening on udev Control Socket.1754lb # [ 1.786313] systemd[1]: Listening on udev Kernel Socket.1755lb # [ 1.787251] systemd[1]: Reached target Socket Units.1756lb # [ 1.790969] systemd[1]: Starting Create List of Static Device Nodes...1757client # [ 1.844642] systemd[1]: Mounting Kernel Configuration File System...1758worker1 # [ 1.797418] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1759lb # [ 1.802157] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1760lb # [ 1.803325] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1761client # [ 1.863407] systemd[1]: Starting Journal Service...1762lb # [ 1.810214] systemd[1]: Mounting Kernel Configuration File System...1763client # [ 1.872522] systemd[1]: Starting Load Kernel Modules...1764client # [ 1.873399] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1765worker2 # [ 1.822314] systemd[1]: Queued start job for default target Initrd Default Target.1766worker1 # [ 1.820509] systemd-journald[73]: Collecting audit messages is disabled.1767worker2 # [ 1.834194] systemd[1]: Created slice Slice /system/modprobe.1768worker2 # [ 1.835450] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1769worker2 # [ 1.836640] systemd[1]: Expecting device /dev/disk/by-label/nixos...1770worker2 # [ 1.837601] systemd[1]: Reached target Path Units.1771worker2 # [ 1.838325] systemd[1]: Reached target Slice Units.1772worker2 # [ 1.839201] systemd[1]: Reached target Swaps.1773worker2 # [ 1.839867] systemd[1]: Reached target Timer Units.1774lb # [ 1.830402] systemd[1]: Starting Journal Service...1775worker2 # [ 1.840932] systemd[1]: Listening on D-Bus System Message Bus Socket.1776worker1 # [ 1.833919] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1777worker2 # [ 1.842070] systemd[1]: Listening on Journal Socket (/dev/log).1778worker1 # [ 1.839175] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1779worker2 # [ 1.843171] systemd[1]: Listening on Journal Sockets.1780worker1 # [ 1.841800] systemd[1]: Starting Create Static Device Nodes in /dev...1781worker2 # [ 1.843325] systemd[1]: Listening on udev Control Socket.1782worker2 # [ 1.843436] systemd[1]: Listening on udev Kernel Socket.1783worker2 # [ 1.843459] systemd[1]: Reached target Socket Units.1784worker2 # [ 1.848393] systemd[1]: Starting Create List of Static Device Nodes...1785lb # [ 1.845009] systemd[1]: Starting Load Kernel Modules...1786worker2 # [ 1.855667] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1787worker2 # [ 1.856806] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1788client # [ 1.907738] systemd[1]: Starting Coldplug All udev Devices...1789worker1 # [ 1.853115] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1790worker1 # [ 1.857868] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01791worker1 # [ 1.858107] [drm] features: -virgl +edid -resource_blob -host_visible1792lb # [ 1.858164] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1793worker1 # [ 1.858117] [drm] features: -context_init1794worker1 # [ 1.858881] [drm] number of scanouts: 11795worker1 # [ 1.858899] [drm] number of cap sets: 01796worker2 # [ 1.867803] systemd[1]: Mounting Kernel Configuration File System...1797lb # [ 1.870155] systemd[1]: Starting Coldplug All udev Devices...1798client # [ 1.927353] systemd[1]: Finished Create List of Static Device Nodes.1799client # [ 1.928716] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1800client # [ 1.929057] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1801worker2 # [ 1.883102] systemd[1]: Starting Journal Service...1802worker1 # [ 1.873538] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1803worker1 # [ 1.873561] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01804client # [ 1.936676] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1805client # [ 1.939717] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1806lb # [ 1.878121] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1807client # [ 1.940987] systemd[1]: Mounted Kernel Configuration File System.1808lb # [ 1.887600] systemd[1]: Finished Create List of Static Device Nodes.1809lb # [ 1.889448] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1810client # [ 1.948157] systemd-journald[73]: Collecting audit messages is disabled.1811client # [ 1.949105] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1812lb # [ 1.898407] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1813lb # [ 1.898910] systemd[1]: Mounted Kernel Configuration File System.1814worker2 # [ 1.906983] systemd[1]: Starting Load Kernel Modules...1815worker2 # [ 1.907824] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1816lb # [ 1.901613] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1817lb # [ 1.904134] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1818client # [ 1.963815] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01819client # [ 1.964079] [drm] features: -virgl +edid -resource_blob -host_visible1820worker1 # [ 1.909753] systemd[1]: Finished Create Static Device Nodes in /dev.1821client # [ 1.964090] [drm] features: -context_init1822worker1 # [ 1.910195] systemd[1]: Reached target Preparation for Local File Systems.1823worker1 # [ 1.910230] systemd[1]: Reached target Local File Systems.1824lb # [ 1.910292] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01825lb # [ 1.910592] [drm] features: -virgl +edid -resource_blob -host_visible1826lb # [ 1.910617] [drm] features: -context_init1827client # [ 1.964868] [drm] number of scanouts: 11828lb # [ 1.911374] [drm] number of scanouts: 11829client # [ 1.964886] [drm] number of cap sets: 01830worker2 # [ 1.919903] systemd[1]: Starting Coldplug All udev Devices...1831lb # [ 1.911391] [drm] number of cap sets: 01832worker1 # [ 1.914149] Console: switching to colour frame buffer device 160x501833worker1 # [ 1.922638] systemd[1]: Starting Rule-based Manager for Device Events and Files...1834client # [ 1.987567] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1835client # [ 1.987599] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01836lb # [ 1.930545] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1837lb # [ 1.930573] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01838worker2 # [ 1.946843] systemd[1]: Finished Create List of Static Device Nodes.1839worker1 # [ 1.945774] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1840worker2 # [ 1.947942] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1841worker2 # [ 1.948258] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1842worker2 # [ 1.948498] systemd[1]: Mounted Kernel Configuration File System.1843client # [ 2.007953] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844worker2 # [ 1.959245] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1845client # [ 2.015528] systemd[1]: Starting Create Static Device Nodes in /dev...1846lb # [ 1.958926] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847worker1 # [ 1.963569] systemd[1]: Finished Load Kernel Modules.1848worker2 # [ 1.968099] systemd-journald[73]: Collecting audit messages is disabled.1849lb # [ 1.962021] systemd-journald[73]: Collecting audit messages is disabled.1850lb # [ 1.970275] systemd[1]: Starting Create Static Device Nodes in /dev...1851client # [ 2.023715] Console: switching to colour frame buffer device 160x501852client # [ 2.030321] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1853worker2 # [ 1.982165] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1854worker1 # [ 1.977436] systemd[1]: Starting Apply Kernel Variables...1855lb # [ 1.974237] Console: switching to colour frame buffer device 160x501856lb # [ 1.981000] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1857worker2 # [ 1.994681] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1858worker2 # [ 1.999415] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01859worker2 # [ 1.999651] [drm] features: -virgl +edid -resource_blob -host_visible1860worker2 # [ 1.999660] [drm] features: -context_init1861worker2 # [ 2.000433] [drm] number of scanouts: 11862worker2 # [ 2.000451] [drm] number of cap sets: 01863client # [ 2.056217] systemd[1]: Finished Load Kernel Modules.1864client # [ 2.063861] systemd[1]: Starting Apply Kernel Variables...1865lb # [ 2.006872] systemd[1]: Finished Load Kernel Modules.1866lb # [ 2.009178] systemd[1]: Starting Apply Kernel Variables...1867worker2 # [ 2.017561] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1868worker2 # [ 2.017583] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01869worker2 # [ 2.023375] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1870worker2 # [ 2.026024] systemd[1]: Starting Create Static Device Nodes in /dev...1871worker1 # [ 2.022128] systemd[1]: Started Journal Service.1872worker1 # [ 2.016459] systemd-modules-load[74]: Inserted module 'dm_mod'1873worker1 # [ 2.017813] systemd-modules-load[74]: Module 'virtio_balloon' is built in1874worker1 # [ 2.018938] systemd-modules-load[74]: Module 'virtio_console' is built in1875client # [ 2.087457] systemd[1]: Finished Create Static Device Nodes in /dev.1876client # [ 2.089649] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 2.089692] systemd[1]: Reached target Local File Systems.1878lb # [ 2.038252] systemd[1]: Finished Create Static Device Nodes in /dev.1879lb # [ 2.038506] systemd[1]: Reached target Preparation for Local File Systems.1880lb # [ 2.038542] systemd[1]: Reached target Local File Systems.1881worker1 # [ 2.019993] systemd-modules-load[74]: Inserted module 'virtio_gpu'1882worker1 # [ 2.036177] systemd-modules-load[74]: Module 'virtio_rng' is built in1883lb # [ 2.046375] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884client # [ 2.103522] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885worker1 # [ 2.044502] systemd[1]: Starting Create System Files and Directories...1886worker1 # [ 2.047983] systemd-udevd[79]: Using default interface naming scheme 'v261'.1887worker1 # [ 2.053602] systemd[1]: Finished Apply Kernel Variables.1888worker2 # [ 2.063089] Console: switching to colour frame buffer device 160x501889worker2 # [ 2.069616] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1890client # [ 2.127544] systemd[1]: Finished Apply Kernel Variables.1891worker1 # [ 2.061256] systemd[1]: Finished Create System Files and Directories.1892worker2 # [ 2.079357] systemd[1]: Finished Create Static Device Nodes in /dev.1893worker2 # [ 2.081441] systemd[1]: Reached target Preparation for Local File Systems.1894lb # [ 2.078255] systemd[1]: Finished Apply Kernel Variables.1895worker2 # [ 2.086755] systemd[1]: Reached target Local File Systems.1896worker2 # [ 2.090211] systemd[1]: Starting Rule-based Manager for Device Events and Files...1897worker1 # [ 2.077678] systemd[1]: Started Rule-based Manager for Device Events and Files.1898worker2 # [ 2.103342] systemd[1]: Finished Load Kernel Modules.1899client # [ 2.159360] systemd[1]: Started Journal Service.1900client # [ 2.147706] systemd-modules-load[74]: Inserted module 'dm_mod'1901client # [ 2.152408] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902client # [ 2.154645] systemd-modules-load[74]: Module 'virtio_console' is built in1903client # [ 2.164211] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904lb # [ 2.120681] systemd[1]: Started Journal Service.1905client # [ 2.168384] systemd-modules-load[74]: Module 'virtio_rng' is built in1906client # [ 2.169471] systemd[1]: Starting Create System Files and Directories...1907worker2 # [ 2.138819] systemd[1]: Starting Apply Kernel Variables...1908lb # [ 2.117837] systemd-modules-load[74]: Inserted module 'dm_mod'1909lb # [ 2.119007] systemd-modules-load[74]: Module 'virtio_balloon' is built in1910lb # [ 2.124277] systemd-modules-load[74]: Module 'virtio_console' is built in1911lb # [ 2.128871] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912lb # [ 2.131299] systemd-modules-load[74]: Module 'virtio_rng' is built in1913client # [ 2.192605] systemd-udevd[81]: Using default interface naming scheme 'v261'.1914worker1 # [ 2.144141] systemd[1]: Starting Virtual Console Setup...1915worker2 # [ 2.159331] systemd[1]: Finished Apply Kernel Variables.1916lb # [ 2.140485] systemd[1]: Starting Create System Files and Directories...1917lb # [ 2.141859] systemd-udevd[81]: Using default interface naming scheme 'v261'.1918worker2 # [ 2.165297] systemd[1]: Started Journal Service.1919worker2 # [ 2.160369] systemd-modules-load[74]: Inserted module 'dm_mod'1920worker2 # [ 2.161612] systemd-modules-load[74]: Module 'virtio_balloon' is built in1921worker2 # [ 2.162674] systemd-modules-load[74]: Module 'virtio_console' is built in1922worker2 # [ 2.163702] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923client # [ 2.220300] systemd[1]: Finished Create System Files and Directories.1924worker2 # [ 2.173968] systemd-modules-load[74]: Module 'virtio_rng' is built in1925lb # [ 2.164850] systemd[1]: Finished Create System Files and Directories.1926worker2 # [ 2.176735] systemd[1]: Starting Create System Files and Directories...1927client # [ 2.233373] systemd[1]: Started Rule-based Manager for Device Events and Files.1928lb # [ 2.180823] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker1 # [ 2.188515] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1930worker1 # [ 2.192052] systemd[1]: Finished Virtual Console Setup.1931worker2 # [ 2.198601] systemd-udevd[79]: Using default interface naming scheme 'v261'.1932worker2 # [ 2.213282] systemd[1]: Finished Create System Files and Directories.1933worker2 # [ 2.238650] systemd[1]: Started Rule-based Manager for Device Events and Files.1934lb # [ 2.240337] systemd[1]: Starting Virtual Console Setup...1935client # [ 2.304115] systemd[1]: Starting Virtual Console Setup...1936worker2 # [ 2.296125] systemd[1]: Starting Virtual Console Setup...1937client # [ 2.348582] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938client # [ 2.356184] systemd[1]: Finished Virtual Console Setup.1939lb # [ 2.296529] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940lb # [ 2.300512] systemd[1]: Finished Virtual Console Setup.1941worker2 # [ 2.352630] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker2 # [ 2.356531] systemd[1]: Finished Virtual Console Setup.1943worker1 # [ 2.833592] systemd[1]: Finished Coldplug All udev Devices.1944worker1 # [ 2.834555] systemd[1]: Reached target System Initialization.1945worker1 # [ 2.840161] systemd[1]: Reached target Basic System.1946client # [ 2.993751] systemd[1]: Finished Coldplug All udev Devices.1947client # [ 2.994944] systemd[1]: Reached target System Initialization.1948client # [ 2.996181] systemd[1]: Reached target Basic System.1949lb # [ 2.940785] systemd[1]: Finished Coldplug All udev Devices.1950lb # [ 2.942651] systemd[1]: Reached target System Initialization.1951lb # [ 2.944209] systemd[1]: Reached target Basic System.1952worker1 # [ 2.960370] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1953worker2 # [ 2.985062] systemd[1]: Finished Coldplug All udev Devices.1954worker2 # [ 2.986875] systemd[1]: Reached target System Initialization.1955worker2 # [ 2.992867] systemd[1]: Reached target Basic System.1956worker1 # [ 3.007465] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1957worker1 # [ 3.012301] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1958lb # [ 3.083992] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1959client # [ 3.148336] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker1 # [ 3.097269] systemd[1]: Found device /dev/disk/by-label/nixos.1961worker1 # [ 3.101755] systemd[1]: Reached target Initrd Root Device.1962worker1 # [ 3.106069] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1963lb # [ 3.109585] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1964client # [ 3.185538] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1965lb # [ 3.129369] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1966worker2 # [ 3.139637] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1967client # [ 3.192952] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1968worker2 # [ 3.150382] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line.1969worker1 # [ 3.153420] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1970worker1 # [ 3.161188] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1971worker2 # [ 3.167674] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1972worker1 # [ 3.168379] systemd[1]: Mounting /sysroot...1973worker1 # [ 3.209082] EXT4-fs (vda): mounted filesystem f7c5d3cd-a4c0-4abb-872c-160b098e5d04 r/w with ordered data mode. Quota mode: none.1974lb # [ 3.191746] systemd[1]: Found device /dev/disk/by-label/nixos.1975client # [ 3.253991] systemd[1]: Found device /dev/disk/by-label/nixos.1976client # [ 3.256223] systemd[1]: Reached target Initrd Root Device.1977lb # [ 3.195870] systemd[1]: Reached target Initrd Root Device.1978worker1 # [ 3.204071] systemd[1]: Mounted /sysroot.1979lb # [ 3.198244] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980worker1 # [ 3.206421] systemd[1]: Reached target Initrd Root File System.1981client # [ 3.260149] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker1 # [ 3.209964] systemd[1]: Starting Mountpoints Configured in the Real Root...1983worker1 # [ 3.234330] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1984worker1 # [ 3.239695] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1985worker1 # [ 3.241599] systemd[1]: Reloading...1986worker2 # [ 3.259022] systemd[1]: Found device /dev/disk/by-label/nixos.1987worker2 # [ 3.260956] systemd[1]: Reached target Initrd Root Device.1988lb # [ 3.254548] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1989worker2 # [ 3.264127] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1990client # [ 3.318836] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1991lb # [ 3.262048] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992client # [ 3.325003] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993lb # [ 3.268455] systemd[1]: Mounting /sysroot...1994client # [ 3.332488] systemd[1]: Mounting /sysroot...1995worker2 # [ 3.307310] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.316179] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997worker2 # [ 3.320260] systemd[1]: Mounting /sysroot...1998lb # [ 3.327812] EXT4-fs (vda): mounted filesystem ff69be9e-844c-4a61-a44f-dbd929a9fa08 r/w with ordered data mode. Quota mode: none.1999client # [ 3.384829] EXT4-fs (vda): mounted filesystem 3d9162f0-02a6-4fd9-a510-3cdf3892677d r/w with ordered data mode. Quota mode: none.2000client # [ 3.375592] systemd[1]: Mounted /sysroot.
loading 1,545 hidden lines…
3546lb # [ 21.789308] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23547lb # [ 21.791088] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3548lb # [ 21.794450] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3549lb # [ 21.797199] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3550lb # [ 21.799110] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23551lb # [ 21.802065] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13552lb # [ 21.806868] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3553lb # [ 21.810546] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3554lb # [ 21.814681] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3555lb # [ 21.817746] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3556lb # [ 21.821166] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3557lb # [ 21.823776] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3558lb # [ 21.827727] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3559lb # [ 21.829896] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3560lb # [ 21.833665] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3561lb # [ 21.836064] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3562lb # [ 21.838271] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3563lb # [ 21.841283] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3564lb # [ 21.853138] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3565lb # [ 21.854920] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3566lb # [ 21.857888] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3567lb # [ 21.859711] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3568lb # [ 21.862185] envoy[934]: [2026-09-21 10:42:30.021][934][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3569lb # [ 21.864739] envoy[934]: [2026-09-21 10:42:30.042][934][info][main] [source/server/server.cc:531] HTTP header map info:3570lb # [ 21.868141] envoy[934]: [2026-09-21 10:42:30.044][934][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3571lb # [ 21.885097] envoy[934]: [2026-09-21 10:42:30.044][934][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3572lb # [ 21.889435] envoy[934]: [2026-09-21 10:42:30.044][934][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3573lb # [ 21.899535] envoy[934]: [2026-09-21 10:42:30.044][934][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3574lb # [ 21.906187] envoy[934]: [2026-09-21 10:42:30.298][934][info][main] [source/server/server.cc:968] runtime: {}3575lb # [ 21.907654] envoy[934]: [2026-09-21 10:42:30.311][934][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3576lb # [ 21.909815] envoy[934]: [2026-09-21 10:42:30.311][934][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013577lb # [ 21.911813] envoy[934]: [2026-09-21 10:42:30.311][934][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013578lb # [ 21.915380] envoy[934]: [2026-09-21 10:42:30.317][934][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3579lb # [ 21.917385] envoy[934]: [2026-09-21 10:42:30.317][934][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3580lb # [ 21.919074] envoy[934]: [2026-09-21 10:42:30.317][934][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3581lb # [ 21.934560] envoy[934]: [2026-09-21 10:42:30.521][934][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3582worker2 # [ 21.947613] systemd[1]: Reached target Network is Online.3583worker1 # [ 21.950751] systemd[1]: Starting Nix worker-protocol over gRPC...3584lb # [ 21.945211] envoy[934]: [2026-09-21 10:42:30.532][934][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3585worker2 # [ 21.959508] systemd[1]: Starting scheduler-token.service...3586lb # [ 21.948284] envoy[934]: [2026-09-21 10:42:30.534][934][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3587lb # [ 21.952973] envoy[934]: [2026-09-21 10:42:30.534][934][info][main] [source/server/server.cc:1077] starting main dispatch loop3588worker2 # [ 22.080589] systemd[1]: scheduler-token.service: Deactivated successfully.3589worker2 # [ 22.087716] systemd[1]: Finished scheduler-token.service.3590worker2 # [ 22.089673] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 124ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3591worker2 # [ 22.097036] systemd[1]: Starting Nix worker-protocol over gRPC...3592worker1 # [ 22.198990] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3593worker1 # [ 22.211360] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3594worker1 # [ 22.213719] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13595worker1 # [ 22.255353] nix-grpc-daemon[775]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3596worker1 # [ 22.260057] nix-grpc-daemon[775]: E0000 00:00:1789987350.841715 775 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3597worker1 # [ 22.274177] nix-grpc-daemon[775]: E0000 00:00:1789987350.860522 775 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3598worker1 # [ 22.280629] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3599worker1 # [ 22.282517] systemd[1]: Started Nix worker-protocol over gRPC.3600worker2 # [ 22.315719] nix-grpc-daemon[779]: ts=2026-09-21T10:42:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3601lb # [ 22.313050] envoy[934]: [2026-09-21 10:42:30.899][934][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3602worker1 # [ 22.323422] nix-grpc-daemon[775]: E0000 00:00:1789987350.908686 816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3603worker2 # [ 22.327725] nix-grpc-daemon[779]: ts=2026-09-21T10:42:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3604worker2 # [ 22.330361] nix-grpc-daemon[779]: ts=2026-09-21T10:42:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13605lb # [ 22.318457] envoy[934]: [2026-09-21 10:42:30.899][934][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3606lb # [ 22.325343] envoy[934]: [2026-09-21 10:42:30.899][934][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3607worker1 # [ 22.343945] nix-grpc-daemon[775]: E0000 00:00:1789987350.930462 816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3608lb # [ 22.335380] envoy[934]: [2026-09-21 10:42:30.922][934][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3609worker1 # [ 22.348052] nix-grpc-daemon[775]: E0000 00:00:1789987350.933997 816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3610worker2 # [ 22.366532] nix-grpc-daemon[779]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3611worker2 # [ 22.368374] nix-grpc-daemon[779]: E0000 00:00:1789987350.961424 779 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3612worker1 # [ 22.367032] nix-grpc-daemon[775]: E0000 00:00:1789987350.952469 816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3613worker2 # [ 22.381866] nix-grpc-daemon[779]: E0000 00:00:1789987350.976812 779 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3614worker1 # [ 22.381810] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=scheduler_connected addr=lb:500513615worker2 # [ 22.385580] nix-grpc-daemon[779]: ts=2026-09-21T10:42:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3616worker2 # [ 22.387288] systemd[1]: Started Nix worker-protocol over gRPC.3617worker1 # [ 22.385595] nix-grpc-daemon[775]: ts=2026-09-21T10:42:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3618worker2 # [ 22.423069] nix-grpc-daemon[779]: E0000 00:00:1789987351.016894 820 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3619worker2 # [ 22.442084] nix-grpc-daemon[779]: E0000 00:00:1789987351.036988 820 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3620worker2 # [ 22.448179] nix-grpc-daemon[779]: E0000 00:00:1789987351.040111 820 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3621worker2 # [ 22.465162] nix-grpc-daemon[779]: E0000 00:00:1789987351.059303 820 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3622worker2 # [ 22.479211] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 22.481011] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker1 # [ 22.587052] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513625worker1 # [ 22.592837] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker2 # [ 22.682550] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513627worker2 # [ 22.687783] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 22.793609] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513629worker1 # [ 22.798691] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker2 # [ 22.888843] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 22.893895] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker1 # [ 22.999623] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 23.004813] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker2 # [ 23.095002] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 23.100157] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 23.207448] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 23.211597] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 23.301153] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 23.305810] nix-grpc-daemon[779]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 23.408059] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 23.412985] nix-grpc-daemon[775]: ts=2026-09-21T10:42:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 23.508768] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 23.512790] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 23.613679] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 23.618743] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 23.709394] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 23.714384] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 23.819643] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 23.824773] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 23.915612] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 23.920700] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 24.025555] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 24.030612] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 24.121881] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 24.126890] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 24.232757] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 24.235449] nix-grpc-daemon[775]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 24.327963] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 24.332633] nix-grpc-daemon[779]: ts=2026-09-21T10:42:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 24.433658] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 24.438795] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 24.535325] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 24.539541] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 24.639645] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 24.644843] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 24.736477] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 24.741475] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 24.845648] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 24.850681] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 24.942638] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 24.947717] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 25.051601] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 25.056831] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 25.148943] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 25.153910] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 25.259505] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 25.263817] nix-grpc-daemon[775]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 25.355201] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 25.360555] nix-grpc-daemon[779]: ts=2026-09-21T10:42:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 25.460242] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 25.465523] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 25.563529] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 25.567757] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 25.666268] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 25.669564] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 25.764356] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 25.769386] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 25.870566] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 25.875794] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 25.970870] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 25.976106] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 26.076673] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 26.081732] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 26.177135] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 26.182232] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 26.284515] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 26.289310] nix-grpc-daemon[775]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 26.383440] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 26.389219] nix-grpc-daemon[779]: ts=2026-09-21T10:42:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 26.484664] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 26.489752] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 26.592252] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 26.596347] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 26.690657] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 26.695609] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 26.793031] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 26.798451] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 26.896492] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 26.901206] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 26.999703] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 27.005142] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 27.102121] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 27.107321] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 27.206674] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 27.212164] nix-grpc-daemon[779]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 27.309706] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 27.313659] nix-grpc-daemon[775]: ts=2026-09-21T10:42:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 27.413175] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 27.418292] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 27.510253] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 27.515419] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 27.621144] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 27.625974] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 27.715919] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 27.720953] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 27.826937] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 27.832116] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 27.921770] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 27.926791] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 28.033510] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 28.038449] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 28.127590] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 28.132705] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 28.239596] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 28.244632] nix-grpc-daemon[779]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 28.334983] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 28.339090] nix-grpc-daemon[775]: ts=2026-09-21T10:42:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 28.445755] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 28.450436] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 28.535356] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 28.540627] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 28.653328] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 28.657714] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 28.741400] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 28.746580] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 28.858953] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 28.864305] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 28.947424] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 28.953656] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 29.065316] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 29.070443] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 29.153744] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 29.159061] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754lb # [ 29.227267] rustfs-bucket-start[1016]: mb s3://farm3755lb # [ 29.241005] systemd[1]: Finished rustfs-bucket.service.3756lb # [ 29.252696] systemd[1]: Starting niks3 server...3757worker2 # [ 29.271253] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513758worker2 # [ 29.274578] nix-grpc-daemon[779]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker1 # [ 29.361399] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_connected addr=lb:500513760worker1 # [ 29.365564] nix-grpc-daemon[775]: ts=2026-09-21T10:42:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker2 # [ 29.475658] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513762worker2 # [ 29.480993] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763lb # [ 29.508414] postgres[1032]: [1032] ERROR: relation "goose_db_version" does not exist at character 363764lb # [ 29.509704] postgres[1032]: [1032] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3765lb # [ 29.542641] niks3-server[1025]: 2026/09/21 10:42:38 OK 20241026095416_initial_model.sql (19.76ms)3766lb # [ 29.545948] niks3-server[1025]: 2026/09/21 10:42:38 OK 20251210153512_drop_unused_gin_index.sql (3.61ms)3767lb # [ 29.551506] niks3-server[1025]: 2026/09/21 10:42:38 OK 20251218171726_add_pins.sql (5.57ms)3768worker1 # [ 29.561716] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513769lb # [ 29.557255] niks3-server[1025]: 2026/09/21 10:42:38 OK 20260628120000_add_object_size_and_stats.sql (4.38ms)3770lb # [ 29.562264] niks3-server[1025]: 2026/09/21 10:42:38 OK 20260905000000_add_claims.sql (4.98ms)3771worker1 # [ 29.566881] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772lb # [ 29.566846] niks3-server[1025]: 2026/09/21 10:42:38 OK 20260920000000_drop_claims.sql (4.41ms)3773lb # [ 29.568698] niks3-server[1025]: 2026/09/21 10:42:38 goose: successfully migrated database to version: 202609200000003774lb # [ 29.573274] niks3-server[1025]: 2026/09/21 10:42:38 OK 1_commit_pending_closure.sql (6.43ms)3775lb # [ 29.575909] niks3-server[1025]: 2026/09/21 10:42:38 OK 2_object_stats_trigger.sql (2.51ms)3776lb # [ 29.577463] niks3-server[1025]: 2026/09/21 10:42:38 goose: up to current file version: 23777lb # [ 29.581258] niks3-server[1025]: 2026/09/21 10:42:38 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3778lb # [ 29.629217] niks3-server[1025]: 2026/09/21 10:42:38 INFO Read proxy enabled — serving cache objects from S33779lb # [ 29.630781] niks3-server[1025]: 2026/09/21 10:42:38 INFO Using socket-activated listener address=0.0.0.0:57513780lb # [ 29.634774] niks3-server[1025]: 2026/09/21 10:42:38 INFO systemd watchdog enabled interval=15s3781lb # [ 29.635946] niks3-server[1025]: 2026/09/21 10:42:38 INFO Starting HTTP server address=0.0.0.0:57513782lb # [ 29.640203] systemd[1]: Started niks3 server.3783lb # [ 29.641372] systemd[1]: Reached target Multi-User System.3784lb # [ 29.642632] systemd[1]: Startup finished in 986ms (kernel) + 4.690s (initrd) + 23.962s (userspace) = 29.639s.3785lb # [ 29.646568] niks3-server[1025]: 2026/09/21 10:42:38 INFO lead: acquired remote=192.168.1.4:604883786worker2 # [ 29.661407] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_take_over3787worker2 # [ 29.682036] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513788worker2 # [ 29.687013] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3789worker1 # [ 29.767816] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 29.772877] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791lb: (finished: waiting for unit niks3.service, in 14.83 seconds)3792lb: waiting for TCP port 5751 on localhost3793lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3794lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)3795lb: waiting for unit envoy.service3796worker2 # [ 29.888300] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513797worker2 # [ 29.893278] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3798lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3799lb: waiting for TCP port 50051 on localhost3800lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3801lb # [ 29.945520] envoy[934]: conn peer=[::1]:49966 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=43802lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)3803worker1 # [ 29.974785] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513804worker1 # [ 29.980795] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3805worker2 # [ 30.094560] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513806worker2 # [ 30.101235] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807worker2 # [ 30.152901] systemd[1]: Starting scheduler-token.service...3808worker1 # [ 30.175846] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 30.181113] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 30.249577] systemd[1]: scheduler-token.service: Deactivated successfully.3811worker2 # [ 30.254745] systemd[1]: Finished scheduler-token.service.3812worker2 # [ 30.255883] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 100ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3813lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3814lb: must succeed: curl -sf localhost:9901/clusters3815worker2 # [ 30.300770] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513816worker2 # [ 30.304496] nix-grpc-daemon[779]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3818lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3819subtest: one node schedules and both builders hold a WorkerSession on it3820lb: must succeed: curl -sf localhost:9901/clusters3821worker1 # [ 30.383453] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_connected addr=lb:500513822worker1 # [ 30.387658] nix-grpc-daemon[775]: ts=2026-09-21T10:42:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3824worker2 # [ 30.505731] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=scheduler_connected addr=lb:500513825worker2 # [ 30.527235] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=oidc_jwks provider=mock keys=13826worker2 # [ 30.531658] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:477163827worker2 # [ 30.537004] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03828worker1 # [ 30.583738] nix-grpc-daemon[775]: ts=2026-09-21T10:42:39Z level=info event=scheduler_connected addr=lb:500513829worker2 # [ 30.586957] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:477163830worker2 # [ 30.591907] nix-grpc-daemon[779]: ts=2026-09-21T10:42:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03831lb: must succeed: curl -sf localhost:9901/clusters3832lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3833worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3834worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3835lb: must succeed: curl -sf localhost:9901/clusters3836lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3837lb: must succeed: curl -sf localhost:9901/clusters3838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3839lb: must succeed: curl -sf localhost:9901/clusters3840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3841lb: must succeed: curl -sf localhost:9901/clusters3842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3843worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3844worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3845worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3846worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3847lb: must succeed: curl -sf localhost:9901/clusters3848lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3849lb: must succeed: curl -sf localhost:9901/clusters3850lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3851worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3852worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3853worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3854worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3855worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3856worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3857worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3858worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3859(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.95 seconds)3860subtest: balancer auth: client cert, bearer token, nothing, unknown CN3861worker1 # [ 32.657667] nix-grpc-daemon[775]: ts=2026-09-21T10:42:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379503862worker1 # [ 32.669947] systemd[1]: Started Nix Daemon.3863worker1 # [ 32.946082] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3864worker1 # [ 32.970018] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)3865worker1 # [ 33.072760] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 2/5)3866worker1 # [ 33.262271] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 368 ms (attempt 3/5)3867worker1 # [ 33.630703] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 289 ms (attempt 4/5)3868worker1 # [ 33.930465] nix-grpc-daemon[775]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3869worker1 # [ 33.937575] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3870worker1 # [ 33.961745] nix-grpc-daemon[775]: ts=2026-09-21T10:42:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:379503871worker1 # [ 33.976275] nix-grpc-daemon[775]: ts=2026-09-21T10:42:42Z level=debug event=path_info_miss path=llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test error="error: path '/nix/store/llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test' is required, but there is no substituter that can build it"3872worker2 # [ 34.004930] nix-grpc-daemon[779]: ts=2026-09-21T10:42:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566863873worker2 # [ 34.016998] systemd[1]: Started Nix Daemon.3874worker2 # [ 34.311970] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3875worker2 # [ 34.333777] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)3876worker2 # [ 34.418291] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5)3877worker2 # [ 34.510849] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 203 ms (attempt 3/5)3878worker2 # [ 34.721295] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 4/5)3879worker2 # [ 34.955749] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3880worker2 # [ 34.961400] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3881worker2 # [ 34.974037] nix-grpc-daemon[779]: ts=2026-09-21T10:42:43Z level=debug event=path_info_miss path=llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test error="error: path '/nix/store/llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test' is required, but there is no substituter that can build it"3882client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3883client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3884client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3885client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3886worker1 # [ 35.433507] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=info event=oidc_jwks provider=mock keys=13887worker1 # [ 35.438208] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:379503888worker1 # [ 35.452323] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:379503889worker1 # [ 35.463427] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=path_info_miss path=llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test error="error: path '/nix/store/llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test' is required, but there is no substituter that can build it"3890worker1 # [ 35.477636] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:379503891worker1 # [ 35.485604] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=path_info_miss path=llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test error="error: path '/nix/store/llyf0bg5i97i0wydcrmwr522c4lqspsl-nixos-system-client-test' is required, but there is no substituter that can build it"3892client: must succeed: rm -r /var/lib/nix-grpc-store3893client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)3894worker1 # [ 35.834789] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=info event=denied method=QueryMissing cn=- role=none3895worker2 # [ 36.099371] nix-grpc-daemon[779]: ts=2026-09-21T10:42:44Z level=info event=denied method=QueryMissing cn=stranger role=none3896(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.82 seconds)3897subtest: DAG build is scheduled across workers and lands in the cache3898client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23899worker1 # [ 36.359524] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379503900client # don't know how to build these paths:3901client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3902worker1 # [ 36.372281] nix-grpc-daemon[775]: ts=2026-09-21T10:42:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379503903worker2 # [ 36.410597] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03904worker2 # [ 36.417803] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13905worker2 # [ 36.469412] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566863906worker2 # [ 36.491549] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566863907client # copying 1 paths to 'lb:50051'...3908worker2 # [ 36.506885] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566863909client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3910worker2 # [ 36.533124] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3911worker2 # [ 36.567628] nix-grpc-daemon[779]: ts=2026-09-21T10:42:45Z level=info event=niks3_push_started pid=8783912worker2 # [ 36.862756] nix-grpc-daemon[878]: time=2026-09-21T10:42:45.456Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""3913worker2 # [ 37.052795] nix-daemon[863]: accepted connection from pid 891, user nix-grpc-daemon (trusted)3914worker2 # [ 37.073586] nix-daemon[863]: reaped child process 894, status = succeeded3915worker2 # [ 37.283114] nix-daemon[863]: accepted connection from pid 896, user nix-grpc-daemon (trusted)3916worker2 # [ 37.303648] nix-daemon[863]: reaped child process 899, status = succeeded3917lb # [ 37.310031] niks3-server[1025]: 2026/09/21 10:42:45 INFO Received uploads request method=POST path=/api/pending_closures3918worker2 # [ 37.355978] nix-grpc-daemon[878]: time=2026-09-21T10:42:45.950Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3919worker2 # [ 37.359384] nix-grpc-daemon[878]: time=2026-09-21T10:42:45.951Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3920lb # [ 37.406142] niks3-server[1025]: 2026/09/21 10:42:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3921lb # [ 37.415720] niks3-server[1025]: 2026/09/21 10:42:46 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3922lb # [ 37.445313] niks3-server[1025]: 2026/09/21 10:42:46 INFO Signed narinfos id=1 count=13923worker2 # [ 37.458494] nix-grpc-daemon[878]: time=2026-09-21T10:42:46.052Z level=INFO msg="Uploading 1 narinfos"3924lb # [ 37.468376] niks3-server[1025]: 2026/09/21 10:42:46 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3925lb # [ 37.476907] niks3-server[1025]: 2026/09/21 10:42:46 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3926lb # [ 37.485480] niks3-server[1025]: 2026/09/21 10:42:46 INFO Completed upload id=13927worker2 # [ 37.495457] nix-grpc-daemon[878]: time=2026-09-21T10:42:46.090Z level=INFO msg="Upload complete. (624ms)"3928lb # [ 37.494134] niks3-server[1025]: 2026/09/21 10:42:46 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3929worker2 # [ 37.513316] nix-grpc-daemon[779]: ts=2026-09-21T10:42:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=5043930worker2 # [ 37.532612] nix-grpc-daemon[779]: ts=2026-09-21T10:42:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566863931worker2 # [ 37.550429] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3932worker2 # [ 37.561906] nix-daemon[863]: reaped child process 876, status = succeeded3933client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3934client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3935worker2 # [ 37.580577] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3936client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds3937client # farm-a> LOG-farm-a-t13938worker2 # [ 38.841560] nix-daemon[863]: reaped child process 903, status = succeeded3939worker2 # [ 39.076101] nix-daemon[863]: accepted connection from pid 919, user nix-grpc-daemon (trusted)3940worker2 # [ 39.095481] nix-daemon[863]: reaped child process 922, status = succeeded3941lb # [ 39.102756] niks3-server[1025]: 2026/09/21 10:42:47 INFO Received uploads request method=POST path=/api/pending_closures3942worker2 # [ 39.139136] nix-grpc-daemon[878]: time=2026-09-21T10:42:47.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3943worker2 # [ 39.143008] nix-grpc-daemon[878]: time=2026-09-21T10:42:47.736Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3944lb # [ 39.193035] niks3-server[1025]: 2026/09/21 10:42:47 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3945lb # [ 39.204494] niks3-server[1025]: 2026/09/21 10:42:47 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3946lb # [ 39.215778] niks3-server[1025]: 2026/09/21 10:42:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3947worker2 # [ 39.230755] nix-grpc-daemon[878]: time=2026-09-21T10:42:47.825Z level=INFO msg="Uploading 1 narinfos"3948lb # [ 39.222699] niks3-server[1025]: 2026/09/21 10:42:47 INFO Signed narinfos id=2 count=13949lb # [ 39.227520] niks3-server[1025]: 2026/09/21 10:42:47 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3950lb # [ 39.242634] niks3-server[1025]: 2026/09/21 10:42:47 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3951worker2 # [ 39.262873] nix-grpc-daemon[878]: time=2026-09-21T10:42:47.857Z level=INFO msg="Upload complete. (461ms)"3952lb # [ 39.253397] niks3-server[1025]: 2026/09/21 10:42:47 INFO Completed upload id=23953lb # [ 39.259317] niks3-server[1025]: 2026/09/21 10:42:47 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3954worker2 # [ 39.271010] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13955worker2 # [ 39.284972] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03956worker2 # [ 39.287378] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13957worker2 # [ 39.289849] nix-daemon[863]: reaped child process 901, status = succeeded3958worker2 # [ 39.334007] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566863959worker2 # [ 39.351801] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566863960worker2 # [ 39.369419] nix-grpc-daemon[878]: time=2026-09-21T10:42:47.964Z level=INFO msg="All 1 paths already cached"3961client # copying 1 paths to 'lb:50051'...3962client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3963worker2 # [ 39.382653] nix-grpc-daemon[779]: ts=2026-09-21T10:42:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566863964worker2 # [ 39.411772] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3965worker2 # [ 39.653597] nix-daemon[863]: accepted connection from pid 929, user nix-grpc-daemon (trusted)3966worker2 # [ 39.675340] nix-daemon[863]: reaped child process 933, status = succeeded3967worker2 # [ 39.873229] nix-daemon[863]: accepted connection from pid 935, user nix-grpc-daemon (trusted)3968worker2 # [ 39.892276] nix-daemon[863]: reaped child process 938, status = succeeded3969lb # [ 39.896239] niks3-server[1025]: 2026/09/21 10:42:48 INFO Received uploads request method=POST path=/api/pending_closures3970worker2 # [ 39.937381] nix-grpc-daemon[878]: time=2026-09-21T10:42:48.531Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3971worker2 # [ 39.939532] nix-grpc-daemon[878]: time=2026-09-21T10:42:48.531Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3972lb # [ 39.990928] niks3-server[1025]: 2026/09/21 10:42:48 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3973lb # [ 40.001826] niks3-server[1025]: 2026/09/21 10:42:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3974worker2 # [ 40.018470] nix-grpc-daemon[878]: time=2026-09-21T10:42:48.612Z level=INFO msg="Uploading 1 narinfos"3975lb # [ 40.012743] niks3-server[1025]: 2026/09/21 10:42:48 INFO Signed narinfos id=3 count=13976lb # [ 40.023601] niks3-server[1025]: 2026/09/21 10:42:48 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3977lb # [ 40.044098] niks3-server[1025]: 2026/09/21 10:42:48 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3978lb # [ 40.052632] niks3-server[1025]: 2026/09/21 10:42:48 INFO Completed upload id=33979worker2 # [ 40.062505] nix-grpc-daemon[878]: time=2026-09-21T10:42:48.657Z level=INFO msg="Upload complete. (624ms)"3980lb # [ 40.054935] niks3-server[1025]: 2026/09/21 10:42:48 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3981worker2 # [ 40.075203] nix-grpc-daemon[779]: ts=2026-09-21T10:42:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=6243982worker2 # [ 40.089551] nix-grpc-daemon[779]: ts=2026-09-21T10:42:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566863983worker2 # [ 40.108160] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3984worker2 # [ 40.124447] nix-daemon[863]: reaped child process 927, status = succeeded3985client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3986worker2 # [ 40.133908] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3987client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3988client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds3989client # farm-b> LOG-farm-b-t13990worker2 # [ 41.189811] nix-daemon[863]: reaped child process 942, status = succeeded3991worker2 # [ 41.379873] nix-daemon[863]: accepted connection from pid 958, user nix-grpc-daemon (trusted)3992worker2 # [ 41.396124] nix-daemon[863]: reaped child process 961, status = succeeded3993lb # [ 41.399965] niks3-server[1025]: 2026/09/21 10:42:49 INFO Received uploads request method=POST path=/api/pending_closures3994worker2 # [ 41.423746] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3995worker2 # [ 41.426655] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.017Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3996lb # [ 41.441826] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3997lb # [ 41.455737] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3998worker2 # [ 41.475320] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.069Z level=INFO msg="Uploading 1 narinfos"3999lb # [ 41.465304] niks3-server[1025]: 2026/09/21 10:42:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4000lb # [ 41.473246] niks3-server[1025]: 2026/09/21 10:42:50 INFO Signed narinfos id=4 count=14001lb # [ 41.475948] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4002lb # [ 41.484094] niks3-server[1025]: 2026/09/21 10:42:50 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4003lb # [ 41.493431] niks3-server[1025]: 2026/09/21 10:42:50 INFO Completed upload id=44004worker2 # [ 41.503105] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.098Z level=INFO msg="Upload complete. (349ms)"4005lb # [ 41.496419] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4006worker2 # [ 41.524912] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14007worker2 # [ 41.536414] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04008worker2 # [ 41.541312] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14009worker2 # [ 41.545374] nix-daemon[863]: reaped child process 940, status = succeeded4010worker2 # [ 41.566022] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864011worker2 # [ 41.581842] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864012worker2 # [ 41.596772] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.191Z level=INFO msg="All 2 paths already cached"4013client # copying 1 paths to 'lb:50051'...4014client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4015worker2 # [ 41.606838] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864016worker2 # [ 41.625304] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4017worker2 # [ 41.820199] nix-daemon[863]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4018worker2 # [ 41.838830] nix-daemon[863]: reaped child process 970, status = succeeded4019lb # [ 41.942179] envoy[934]: conn peer=[2001:db8:1::1]:39018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=23734020lb # [ 41.947495] envoy[934]: conn peer=[2001:db8:1::1]:39024 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=387 ms=814021lb # [ 41.952958] envoy[934]: conn peer=[2001:db8:1::1]:39028 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114022lb # [ 41.959071] envoy[934]: conn peer=[2001:db8:1::1]:39034 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=184023worker2 # [ 42.006788] nix-daemon[863]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4024worker2 # [ 42.024263] nix-daemon[863]: reaped child process 975, status = succeeded4025lb # [ 42.026554] niks3-server[1025]: 2026/09/21 10:42:50 INFO Received uploads request method=POST path=/api/pending_closures4026worker2 # [ 42.045354] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.640Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4027worker2 # [ 42.047266] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.642Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4028lb # [ 42.058913] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4029lb # [ 42.071082] niks3-server[1025]: 2026/09/21 10:42:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4030worker2 # [ 42.086998] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.681Z level=INFO msg="Uploading 1 narinfos"4031lb # [ 42.081822] niks3-server[1025]: 2026/09/21 10:42:50 INFO Signed narinfos id=5 count=14032lb # [ 42.087687] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4033lb # [ 42.098446] niks3-server[1025]: 2026/09/21 10:42:50 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4034worker2 # [ 42.112001] nix-grpc-daemon[878]: time=2026-09-21T10:42:50.707Z level=INFO msg="Upload complete. (469ms)"4035lb # [ 42.105746] niks3-server[1025]: 2026/09/21 10:42:50 INFO Completed upload id=54036lb # [ 42.108609] niks3-server[1025]: 2026/09/21 10:42:50 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4037worker2 # [ 42.124484] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=7524038worker2 # [ 42.133332] nix-grpc-daemon[779]: ts=2026-09-21T10:42:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864039worker2 # [ 42.140779] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4040worker2 # [ 42.143269] nix-daemon[863]: reaped child process 965, status = succeeded4041client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4042client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4043worker2 # [ 42.155907] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4044client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4045client # farm-top> LOG-farm-top-t14046worker2 # [ 43.395611] nix-daemon[863]: reaped child process 979, status = succeeded4047worker2 # [ 43.575929] nix-daemon[863]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4048worker2 # [ 43.594113] nix-daemon[863]: reaped child process 999, status = succeeded4049lb # [ 43.598016] niks3-server[1025]: 2026/09/21 10:42:52 INFO Received uploads request method=POST path=/api/pending_closures4050worker2 # [ 43.619031] nix-grpc-daemon[878]: time=2026-09-21T10:42:52.212Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4051worker2 # [ 43.620679] nix-grpc-daemon[878]: time=2026-09-21T10:42:52.212Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4052lb # [ 43.635812] niks3-server[1025]: 2026/09/21 10:42:52 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4053lb # [ 43.651281] niks3-server[1025]: 2026/09/21 10:42:52 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4054worker2 # [ 43.675794] nix-grpc-daemon[878]: time=2026-09-21T10:42:52.270Z level=INFO msg="Uploading 1 narinfos"4055lb # [ 43.661115] niks3-server[1025]: 2026/09/21 10:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4056lb # [ 43.669792] niks3-server[1025]: 2026/09/21 10:42:52 INFO Signed narinfos id=6 count=14057lb # [ 43.674724] niks3-server[1025]: 2026/09/21 10:42:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4058lb # [ 43.684296] niks3-server[1025]: 2026/09/21 10:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4059lb # [ 43.690017] niks3-server[1025]: 2026/09/21 10:42:52 INFO Completed upload id=64060worker2 # [ 43.699758] nix-grpc-daemon[878]: time=2026-09-21T10:42:52.294Z level=INFO msg="Upload complete. (330ms)"4061lb # [ 43.692749] niks3-server[1025]: 2026/09/21 10:42:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4062worker2 # [ 43.713163] nix-grpc-daemon[779]: ts=2026-09-21T10:42:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14063worker2 # [ 43.730498] nix-daemon[863]: reaped child process 977, status = succeeded4064client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.64 seconds)4065client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4066client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.25 seconds)4067client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4068client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4069client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4070client # copying 3 paths...4071client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4072client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4073client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4074client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4075(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.33 seconds)4076subtest: repeat is answered Cached by the scheduler without building4077worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4078worker1 # finding garbage collector roots...4079worker1 # deleting specified paths...4080worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4081worker1 # deleting unused links...4082worker1 # note: hard linking is currently saving 0.0 KiB4083worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4084worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4085worker2 # finding garbage collector roots...4086worker2 # deleting specified paths...4087worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4088worker2 # deleting unused links...4089worker2 # note: hard linking is currently saving 0.0 KiB4090worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4091client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24092worker2 # [ 45.336779] nix-grpc-daemon[779]: ts=2026-09-21T10:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566864093client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4094client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4095worker2 # [ 45.356877] nix-grpc-daemon[779]: ts=2026-09-21T10:42:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864096worker2 # [ 45.367917] nix-grpc-daemon[878]: time=2026-09-21T10:42:53.962Z level=INFO msg="All 2 paths already cached"4097worker2 # [ 45.386239] nix-grpc-daemon[779]: ts=2026-09-21T10:42:53Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14098client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.37 seconds)4099client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4100client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4101lb: must succeed: curl -sf localhost:9901/clusters4102lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4103lb: must succeed: curl -sf localhost:9901/clusters4104lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4105worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4106worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4107(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.49 seconds)4108subtest: inputs already in the cache are not sent to the scheduler4109lb: must succeed: curl -sf localhost:9901/clusters4110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4111lb: must succeed: curl -sf localhost:9901/clusters4112lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4113worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4114worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4115client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24116client # don't know how to build these paths:4117client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4118worker1 # [ 46.413964] nix-grpc-daemon[775]: ts=2026-09-21T10:42:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379504119worker1 # [ 46.421524] nix-grpc-daemon[775]: ts=2026-09-21T10:42:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504120worker2 # [ 46.453356] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04121worker2 # [ 46.460854] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14122worker2 # [ 46.510551] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864123worker2 # [ 46.526400] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864124worker2 # [ 46.542274] nix-grpc-daemon[878]: time=2026-09-21T10:42:55.136Z level=INFO msg="All 2 paths already cached"4125client # copying 1 paths to 'lb:50051'...4126client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4127worker2 # [ 46.552433] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864128worker2 # [ 46.570565] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4129worker2 # [ 46.778062] nix-daemon[863]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4130worker2 # [ 46.796995] nix-daemon[863]: reaped child process 1032, status = succeeded4131worker2 # [ 46.967051] nix-daemon[863]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)4132worker2 # [ 46.983828] nix-daemon[863]: reaped child process 1037, status = succeeded4133lb # [ 46.990010] niks3-server[1025]: 2026/09/21 10:42:55 INFO Received uploads request method=POST path=/api/pending_closures4134worker2 # [ 47.009167] nix-grpc-daemon[878]: time=2026-09-21T10:42:55.603Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4135worker2 # [ 47.010636] nix-grpc-daemon[878]: time=2026-09-21T10:42:55.603Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4136lb # [ 47.017270] niks3-server[1025]: 2026/09/21 10:42:55 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4137worker2 # [ 47.040138] nix-grpc-daemon[878]: time=2026-09-21T10:42:55.634Z level=INFO msg="Uploading 1 narinfos"4138lb # [ 47.031296] niks3-server[1025]: 2026/09/21 10:42:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4139lb # [ 47.042543] niks3-server[1025]: 2026/09/21 10:42:55 INFO Signed narinfos id=7 count=14140lb # [ 47.046799] niks3-server[1025]: 2026/09/21 10:42:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4141lb # [ 47.056288] niks3-server[1025]: 2026/09/21 10:42:55 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4142lb # [ 47.063939] niks3-server[1025]: 2026/09/21 10:42:55 INFO Completed upload id=74143worker2 # [ 47.073555] nix-grpc-daemon[878]: time=2026-09-21T10:42:55.668Z level=INFO msg="Upload complete. (483ms)"4144lb # [ 47.067191] niks3-server[1025]: 2026/09/21 10:42:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4145worker2 # [ 47.080125] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=7524146worker2 # [ 47.092276] nix-grpc-daemon[779]: ts=2026-09-21T10:42:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864147worker2 # [ 47.100221] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4148worker2 # [ 47.102984] nix-daemon[863]: reaped child process 1027, status = succeeded4149client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4150client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4151worker2 # [ 47.114222] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4152client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4153client # farm-top> LOG-farm-top-t1b4154worker2 # [ 48.361512] nix-daemon[863]: reaped child process 1041, status = succeeded4155worker2 # [ 48.539148] nix-daemon[863]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4156worker2 # [ 48.555438] nix-daemon[863]: reaped child process 1060, status = succeeded4157lb # [ 48.561525] niks3-server[1025]: 2026/09/21 10:42:57 INFO Received uploads request method=POST path=/api/pending_closures4158worker2 # [ 48.584046] nix-grpc-daemon[878]: time=2026-09-21T10:42:57.176Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4159worker2 # [ 48.585500] nix-grpc-daemon[878]: time=2026-09-21T10:42:57.176Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4160lb # [ 48.602658] niks3-server[1025]: 2026/09/21 10:42:57 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4161worker2 # [ 48.626406] nix-grpc-daemon[878]: time=2026-09-21T10:42:57.221Z level=INFO msg="Uploading 1 narinfos"4162lb # [ 48.620169] niks3-server[1025]: 2026/09/21 10:42:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4163lb # [ 48.629000] niks3-server[1025]: 2026/09/21 10:42:57 INFO Signed narinfos id=8 count=14164lb # [ 48.631286] niks3-server[1025]: 2026/09/21 10:42:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4165lb # [ 48.636234] niks3-server[1025]: 2026/09/21 10:42:57 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4166lb # [ 48.646540] niks3-server[1025]: 2026/09/21 10:42:57 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4167lb # [ 48.649732] niks3-server[1025]: 2026/09/21 10:42:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4168worker2 # [ 48.659275] nix-grpc-daemon[878]: time=2026-09-21T10:42:57.254Z level=INFO msg="Upload complete. (332ms)"4169lb # [ 48.652331] niks3-server[1025]: 2026/09/21 10:42:57 INFO Completed upload id=84170worker2 # [ 48.672980] nix-grpc-daemon[779]: ts=2026-09-21T10:42:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14171worker2 # [ 48.686358] nix-daemon[863]: reaped child process 1039, status = succeeded4172client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.58 seconds)4173client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4174client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4175lb: must succeed: curl -sf localhost:9901/clusters4176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4177lb: must succeed: curl -sf localhost:9901/clusters4178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4179worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4180worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4181(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.22 seconds)4182subtest: two clients wanting the same drv build it once4183client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4184client # [ 49.301767] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4185client # Running as unit: dedup1.service; invocation ID: 3adf71e90b7c496884639d26143ec71f4186client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4187client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4188client # [ 49.571123] nix[879]: don't know how to build these paths:4189client # [ 49.572246] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4190worker1 # [ 49.517976] nix-grpc-daemon[775]: ts=2026-09-21T10:42:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379504191worker2 # [ 49.531778] nix-grpc-daemon[779]: ts=2026-09-21T10:42:58Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04192worker2 # [ 49.538320] nix-grpc-daemon[779]: ts=2026-09-21T10:42:58Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14193worker1 # [ 49.584879] nix-grpc-daemon[775]: ts=2026-09-21T10:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504194worker1 # [ 49.604935] nix-grpc-daemon[775]: ts=2026-09-21T10:42:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504195client # [ 49.672264] nix[879]: copying 1 paths to 'lb:50051'...4196worker1 # [ 49.619533] nix-grpc-daemon[775]: ts=2026-09-21T10:42:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:379504197client # [ 49.674781] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4198worker1 # [ 49.635938] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4199worker1 # [ 49.668339] nix-grpc-daemon[775]: ts=2026-09-21T10:42:58Z level=info event=niks3_push_started pid=8684200worker1 # [ 49.934226] nix-grpc-daemon[868]: time=2026-09-21T10:42:58.518Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4201worker1 # [ 50.125548] nix-daemon[839]: accepted connection from pid 881, user nix-grpc-daemon (trusted)4202worker1 # [ 50.144735] nix-daemon[839]: reaped child process 884, status = succeeded4203worker1 # [ 50.321454] nix-daemon[839]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4204worker1 # [ 50.338599] nix-daemon[839]: reaped child process 889, status = succeeded4205lb # [ 50.343829] niks3-server[1025]: 2026/09/21 10:42:58 INFO Received uploads request method=POST path=/api/pending_closures4206worker1 # [ 50.362102] nix-grpc-daemon[868]: time=2026-09-21T10:42:58.948Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4207worker1 # [ 50.366143] nix-grpc-daemon[868]: time=2026-09-21T10:42:58.950Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4208client # [ 50.419032] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4209client # Running as unit: dedup2.service; invocation ID: 5d52c945ce854b598c9d8ca60f95c5b74210lb # [ 50.384911] niks3-server[1025]: 2026/09/21 10:42:58 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4211client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4212lb # [ 50.398445] niks3-server[1025]: 2026/09/21 10:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4213worker1 # [ 50.413392] nix-grpc-daemon[868]: time=2026-09-21T10:42:58.999Z level=INFO msg="Uploading 1 narinfos"4214lb # [ 50.409306] niks3-server[1025]: 2026/09/21 10:42:58 INFO Signed narinfos id=9 count=14215lb # [ 50.417695] niks3-server[1025]: 2026/09/21 10:42:59 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4216lb # [ 50.428355] niks3-server[1025]: 2026/09/21 10:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4217worker1 # [ 50.444710] nix-grpc-daemon[868]: time=2026-09-21T10:42:59.031Z level=INFO msg="Upload complete. (502ms)"4218lb # [ 50.437428] niks3-server[1025]: 2026/09/21 10:42:59 INFO Completed upload id=94219worker1 # [ 50.452222] nix-grpc-daemon[775]: ts=2026-09-21T10:42:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_in=4324220lb # [ 50.441344] niks3-server[1025]: 2026/09/21 10:42:59 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4221worker1 # [ 50.455631] nix-grpc-daemon[775]: ts=2026-09-21T10:42:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504222worker1 # [ 50.465052] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4223worker1 # [ 50.468825] nix-daemon[839]: reaped child process 866, status = succeeded4224client # [ 50.529696] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4225worker1 # [ 50.479395] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4226lb # [ 51.942155] envoy[934]: conn peer=[2001:db8:1::1]:39038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4100 ms=73794227lb # [ 51.947203] envoy[934]: conn peer=[2001:db8:1::1]:38862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=476 ms=734228lb # [ 51.952054] envoy[934]: conn peer=[2001:db8:1::1]:38876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1825 ms=22704229worker1 # [ 52.114273] nix-grpc-daemon[775]: ts=2026-09-21T10:43:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379504230client # [ 52.179282] nix[895]: this derivation will be built:4231worker2 # [ 52.134188] nix-grpc-daemon[779]: ts=2026-09-21T10:43:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864232worker2 # [ 52.148732] nix-grpc-daemon[779]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4233client # [ 52.222584] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4234worker2 # [ 52.185994] nix-grpc-daemon[779]: ts=2026-09-21T10:43:00Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04235worker1 # [ 52.192762] nix-grpc-daemon[775]: ts=2026-09-21T10:43:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504236client # [ 52.250453] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4237client: must succeed: sleep 24238client: (finished: must succeed: sleep 2, in 2.07 seconds)4239worker1 # [ 55.109327] nix-daemon[839]: reaped child process 897, status = succeeded4240client: waiting for success: ! systemctl is-active dedup1 dedup24241worker1 # [ 55.267143] nix-daemon[839]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4242worker1 # [ 55.284477] nix-daemon[839]: reaped child process 930, status = succeeded4243lb # [ 55.289260] niks3-server[1025]: 2026/09/21 10:43:03 INFO Received uploads request method=POST path=/api/pending_closures4244worker1 # [ 55.308393] nix-grpc-daemon[868]: time=2026-09-21T10:43:03.894Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4245worker1 # [ 55.310657] nix-grpc-daemon[868]: time=2026-09-21T10:43:03.894Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4246lb # [ 55.336238] niks3-server[1025]: 2026/09/21 10:43:03 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4247lb # [ 55.346869] niks3-server[1025]: 2026/09/21 10:43:03 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4248lb # [ 55.357559] niks3-server[1025]: 2026/09/21 10:43:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4249lb # [ 55.363418] niks3-server[1025]: 2026/09/21 10:43:03 INFO Signed narinfos id=10 count=14250worker1 # [ 55.372174] nix-grpc-daemon[868]: time=2026-09-21T10:43:03.957Z level=INFO msg="Uploading 1 narinfos"4251lb # [ 55.366769] niks3-server[1025]: 2026/09/21 10:43:03 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4252lb # [ 55.380307] niks3-server[1025]: 2026/09/21 10:43:03 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4253lb # [ 55.389345] niks3-server[1025]: 2026/09/21 10:43:03 INFO Completed upload id=104254worker1 # [ 55.397464] nix-grpc-daemon[868]: time=2026-09-21T10:43:03.983Z level=INFO msg="Upload complete. (300ms)"4255lb # [ 55.392433] niks3-server[1025]: 2026/09/21 10:43:03 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4256worker1 # [ 55.408988] nix-grpc-daemon[775]: ts=2026-09-21T10:43:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14257worker1 # [ 55.422460] nix-grpc-daemon[775]: ts=2026-09-21T10:43:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14258worker1 # [ 55.428482] nix-daemon[839]: reaped child process 895, status = succeeded4259client # [ 55.525723] systemd[1]: dedup2.service: Deactivated successfully.4260client # [ 55.529986] systemd[1]: dedup2.service: Consumed 242ms CPU time over 5.110s wall clock time, 33.2M memory peak, 6.3K incoming IP traffic, 5.8K outgoing IP traffic.4261client # [ 55.540854] systemd[1]: dedup1.service: Deactivated successfully.4262client # [ 55.542893] systemd[1]: dedup1.service: Consumed 176ms CPU time over 6.238s wall clock time, 41.3M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4263client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4264client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4265client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4266worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4267worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4268worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4269worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4270worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4271worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4272worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4273worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4274(finished: subtest: two clients wanting the same drv build it once, in 7.49 seconds)4275subtest: build hook: nix-daemon with builders = grpc://lb4276client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4277client # [ 56.941359] systemd[1]: Started Nix Daemon.4278client # [ 57.129052] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4279worker1 # [ 57.294457] nix-grpc-daemon[775]: ts=2026-09-21T10:43:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:379504280worker1 # [ 57.301659] nix-grpc-daemon[775]: ts=2026-09-21T10:43:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504281worker2 # [ 57.318596] nix-grpc-daemon[779]: ts=2026-09-21T10:43:05Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04282worker2 # [ 57.324078] nix-grpc-daemon[779]: ts=2026-09-21T10:43:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14283worker2 # [ 57.362408] nix-grpc-daemon[779]: ts=2026-09-21T10:43:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864284worker2 # [ 57.377729] nix-grpc-daemon[779]: ts=2026-09-21T10:43:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864285worker2 # [ 57.393578] nix-grpc-daemon[779]: ts=2026-09-21T10:43:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864286worker2 # [ 57.409136] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4287worker2 # [ 57.618377] nix-daemon[863]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)4288worker2 # [ 57.637085] nix-daemon[863]: reaped child process 1104, status = succeeded4289worker2 # [ 57.810549] nix-daemon[863]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4290worker2 # [ 57.826023] nix-daemon[863]: reaped child process 1109, status = succeeded4291lb # [ 57.829261] niks3-server[1025]: 2026/09/21 10:43:06 INFO Received uploads request method=POST path=/api/pending_closures4292worker2 # [ 57.848670] nix-grpc-daemon[878]: time=2026-09-21T10:43:06.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4293worker2 # [ 57.850101] nix-grpc-daemon[878]: time=2026-09-21T10:43:06.443Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4294lb # [ 57.859362] niks3-server[1025]: 2026/09/21 10:43:06 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4295lb # [ 57.874750] niks3-server[1025]: 2026/09/21 10:43:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4296worker2 # [ 57.890438] nix-grpc-daemon[878]: time=2026-09-21T10:43:06.485Z level=INFO msg="Uploading 1 narinfos"4297lb # [ 57.886110] niks3-server[1025]: 2026/09/21 10:43:06 INFO Signed narinfos id=11 count=14298lb # [ 57.891237] niks3-server[1025]: 2026/09/21 10:43:06 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4299lb # [ 57.899683] niks3-server[1025]: 2026/09/21 10:43:06 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4300lb # [ 57.906070] niks3-server[1025]: 2026/09/21 10:43:06 INFO Completed upload id=114301worker2 # [ 57.915502] nix-grpc-daemon[878]: time=2026-09-21T10:43:06.510Z level=INFO msg="Upload complete. (489ms)"4302lb # [ 57.908907] niks3-server[1025]: 2026/09/21 10:43:06 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4303worker2 # [ 57.924247] nix-grpc-daemon[779]: ts=2026-09-21T10:43:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=5124304worker2 # [ 57.932305] nix-grpc-daemon[779]: ts=2026-09-21T10:43:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864305worker2 # [ 57.944167] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4306worker2 # [ 57.947425] nix-daemon[863]: reaped child process 1099, status = succeeded4307worker2 # [ 57.958921] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4308worker2 # [ 59.385299] nix-daemon[863]: reaped child process 1113, status = succeeded4309worker2 # [ 59.546765] nix-daemon[863]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)4310worker2 # [ 59.563337] nix-daemon[863]: reaped child process 1132, status = succeeded4311lb # [ 59.569666] niks3-server[1025]: 2026/09/21 10:43:08 INFO Received uploads request method=POST path=/api/pending_closures4312worker2 # [ 59.591651] nix-grpc-daemon[878]: time=2026-09-21T10:43:08.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4313worker2 # [ 59.593324] nix-grpc-daemon[878]: time=2026-09-21T10:43:08.183Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4314lb # [ 59.602792] niks3-server[1025]: 2026/09/21 10:43:08 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4315lb # [ 59.620300] niks3-server[1025]: 2026/09/21 10:43:08 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4316worker2 # [ 59.640867] nix-grpc-daemon[878]: time=2026-09-21T10:43:08.235Z level=INFO msg="Uploading 1 narinfos"4317lb # [ 59.631145] niks3-server[1025]: 2026/09/21 10:43:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4318lb # [ 59.637912] niks3-server[1025]: 2026/09/21 10:43:08 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4319lb # [ 59.646585] niks3-server[1025]: 2026/09/21 10:43:08 INFO Signed narinfos id=12 count=14320lb # [ 59.652960] niks3-server[1025]: 2026/09/21 10:43:08 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4321lb # [ 59.659067] niks3-server[1025]: 2026/09/21 10:43:08 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4322lb # [ 59.664482] niks3-server[1025]: 2026/09/21 10:43:08 INFO Completed upload id=124323worker2 # [ 59.674180] nix-grpc-daemon[878]: time=2026-09-21T10:43:08.269Z level=INFO msg="Upload complete. (302ms)"4324worker2 # [ 59.685102] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14325worker2 # [ 59.703549] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864326worker2 # [ 59.707895] nix-daemon[863]: reaped child process 1111, status = succeeded4327worker2 # [ 59.723128] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566864328worker2 # [ 59.729355] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_out=1284329worker2 # [ 60.046479] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:566864330worker1 # [ 60.051330] nix-grpc-daemon[775]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504331worker2 # [ 60.080227] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04332worker2 # [ 60.088984] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14333worker2 # [ 60.095528] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864334worker2 # [ 60.106222] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864335worker2 # [ 60.119662] nix-grpc-daemon[878]: time=2026-09-21T10:43:08.714Z level=INFO msg="All 1 paths already cached"4336worker2 # [ 60.128976] nix-grpc-daemon[779]: ts=2026-09-21T10:43:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864337worker2 # [ 60.144436] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4338worker2 # [ 60.352878] nix-daemon[863]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)4339worker2 # [ 60.371230] nix-daemon[863]: reaped child process 1141, status = succeeded4340worker2 # [ 60.543357] nix-daemon[863]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4341worker2 # [ 60.559620] nix-daemon[863]: reaped child process 1146, status = succeeded4342lb # [ 60.561672] niks3-server[1025]: 2026/09/21 10:43:09 INFO Received uploads request method=POST path=/api/pending_closures4343worker2 # [ 60.580806] nix-grpc-daemon[878]: time=2026-09-21T10:43:09.175Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4344worker2 # [ 60.582240] nix-grpc-daemon[878]: time=2026-09-21T10:43:09.175Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4345lb # [ 60.591846] niks3-server[1025]: 2026/09/21 10:43:09 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4346worker2 # [ 60.620900] nix-grpc-daemon[878]: time=2026-09-21T10:43:09.214Z level=INFO msg="Uploading 1 narinfos"4347lb # [ 60.603309] niks3-server[1025]: 2026/09/21 10:43:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4348lb # [ 60.615722] niks3-server[1025]: 2026/09/21 10:43:09 INFO Signed narinfos id=13 count=14349lb # [ 60.622297] niks3-server[1025]: 2026/09/21 10:43:09 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4350lb # [ 60.630615] niks3-server[1025]: 2026/09/21 10:43:09 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4351lb # [ 60.635693] niks3-server[1025]: 2026/09/21 10:43:09 INFO Completed upload id=134352worker2 # [ 60.644510] nix-grpc-daemon[878]: time=2026-09-21T10:43:09.239Z level=INFO msg="Upload complete. (482ms)"4353lb # [ 60.638672] niks3-server[1025]: 2026/09/21 10:43:09 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4354worker2 # [ 60.652791] nix-grpc-daemon[779]: ts=2026-09-21T10:43:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=6244355worker2 # [ 60.666483] nix-grpc-daemon[779]: ts=2026-09-21T10:43:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864356worker2 # [ 60.674468] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4357worker2 # [ 60.676705] nix-daemon[863]: reaped child process 1136, status = succeeded4358worker2 # [ 60.688947] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4359lb # [ 61.943723] envoy[934]: conn peer=[2001:db8:1::1]:38892 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1426 ms=59114360lb # [ 61.949243] envoy[934]: conn peer=[2001:db8:1::1]:44232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1487 ms=33194361lb # [ 61.958822] envoy[934]: conn peer=[2001:db8:1::1]:44234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1741 ms=24964362lb # [ 61.964558] envoy[934]: conn peer=[2001:db8:1::1]:44244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=734363worker2 # [ 62.319547] nix-daemon[863]: reaped child process 1150, status = succeeded4364worker2 # [ 62.487953] nix-daemon[863]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)4365worker2 # [ 62.505796] nix-daemon[863]: reaped child process 1169, status = succeeded4366lb # [ 62.512599] niks3-server[1025]: 2026/09/21 10:43:11 INFO Received uploads request method=POST path=/api/pending_closures4367worker2 # [ 62.532092] nix-grpc-daemon[878]: time=2026-09-21T10:43:11.125Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4368worker2 # [ 62.533647] nix-grpc-daemon[878]: time=2026-09-21T10:43:11.125Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4369lb # [ 62.552566] niks3-server[1025]: 2026/09/21 10:43:11 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4370worker2 # [ 62.572864] nix-grpc-daemon[878]: time=2026-09-21T10:43:11.167Z level=INFO msg="Uploading 1 narinfos"4371lb # [ 62.569049] niks3-server[1025]: 2026/09/21 10:43:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4372lb # [ 62.577335] niks3-server[1025]: 2026/09/21 10:43:11 INFO Signed narinfos id=14 count=14373lb # [ 62.583843] niks3-server[1025]: 2026/09/21 10:43:11 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4374lb # [ 62.587404] niks3-server[1025]: 2026/09/21 10:43:11 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4375worker2 # [ 62.600561] nix-grpc-daemon[878]: time=2026-09-21T10:43:11.195Z level=INFO msg="Upload complete. (301ms)"4376lb # [ 62.591464] niks3-server[1025]: 2026/09/21 10:43:11 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4377lb # [ 62.596858] niks3-server[1025]: 2026/09/21 10:43:11 INFO Completed upload id=144378lb # [ 62.599284] niks3-server[1025]: 2026/09/21 10:43:11 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4379worker2 # [ 62.609919] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14380worker1 # [ 62.616801] nix-grpc-daemon[775]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:379504381worker2 # [ 62.624941] nix-daemon[863]: reaped child process 1148, status = succeeded4382worker1 # [ 62.632687] nix-grpc-daemon[775]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4383worker1 # [ 62.651018] nix-grpc-daemon[775]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4384worker1 # [ 62.691878] nix-grpc-daemon[775]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:379504385worker1 # [ 62.699771] nix-grpc-daemon[775]: ts=2026-09-21T10:43:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_out=1764386worker1 # [ 63.021472] nix-grpc-daemon[775]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:379504387worker1 # [ 63.029084] nix-grpc-daemon[775]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504388worker1 # [ 63.036646] nix-grpc-daemon[868]: time=2026-09-21T10:43:11.621Z level=INFO msg="All 2 paths already cached"4389worker2 # [ 63.049294] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04390worker2 # [ 63.056416] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14391worker2 # [ 63.065639] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864392worker2 # [ 63.076333] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864393worker2 # [ 63.090373] nix-grpc-daemon[878]: time=2026-09-21T10:43:11.685Z level=INFO msg="All 2 paths already cached"4394worker2 # [ 63.099258] nix-grpc-daemon[779]: ts=2026-09-21T10:43:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864395worker2 # [ 63.113519] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4396worker2 # [ 63.316162] nix-daemon[863]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4397worker2 # [ 63.333731] nix-daemon[863]: reaped child process 1178, status = succeeded4398worker2 # [ 63.508075] nix-daemon[863]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)4399worker2 # [ 63.523857] nix-daemon[863]: reaped child process 1183, status = succeeded4400lb # [ 63.531200] niks3-server[1025]: 2026/09/21 10:43:12 INFO Received uploads request method=POST path=/api/pending_closures4401worker2 # [ 63.553330] nix-grpc-daemon[878]: time=2026-09-21T10:43:12.147Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4402worker2 # [ 63.554811] nix-grpc-daemon[878]: time=2026-09-21T10:43:12.147Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4403lb # [ 63.563165] niks3-server[1025]: 2026/09/21 10:43:12 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4404worker2 # [ 63.594351] nix-grpc-daemon[878]: time=2026-09-21T10:43:12.189Z level=INFO msg="Uploading 1 narinfos"4405lb # [ 63.585191] niks3-server[1025]: 2026/09/21 10:43:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4406lb # [ 63.594972] niks3-server[1025]: 2026/09/21 10:43:12 INFO Signed narinfos id=15 count=14407lb # [ 63.600393] niks3-server[1025]: 2026/09/21 10:43:12 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4408lb # [ 63.610725] niks3-server[1025]: 2026/09/21 10:43:12 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4409lb # [ 63.617689] niks3-server[1025]: 2026/09/21 10:43:12 INFO Completed upload id=154410worker2 # [ 63.626130] nix-grpc-daemon[878]: time=2026-09-21T10:43:12.221Z level=INFO msg="Upload complete. (495ms)"4411lb # [ 63.624890] niks3-server[1025]: 2026/09/21 10:43:12 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4412worker2 # [ 63.635761] nix-grpc-daemon[779]: ts=2026-09-21T10:43:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_in=7524413worker2 # [ 63.649838] nix-grpc-daemon[779]: ts=2026-09-21T10:43:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864414worker2 # [ 63.657724] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4415worker2 # [ 63.661338] nix-daemon[863]: reaped child process 1173, status = succeeded4416worker2 # [ 63.673408] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4417worker2 # [ 64.507743] nix-daemon[863]: reaped child process 1187, status = succeeded4418worker2 # [ 64.677307] nix-daemon[863]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)4419worker2 # [ 64.696707] nix-daemon[863]: reaped child process 1206, status = succeeded4420lb # [ 64.702340] niks3-server[1025]: 2026/09/21 10:43:13 INFO Received uploads request method=POST path=/api/pending_closures4421worker2 # [ 64.736472] nix-grpc-daemon[878]: time=2026-09-21T10:43:13.330Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4422worker2 # [ 64.737990] nix-grpc-daemon[878]: time=2026-09-21T10:43:13.330Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4423lb # [ 64.760393] niks3-server[1025]: 2026/09/21 10:43:13 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4424lb # [ 64.771947] niks3-server[1025]: 2026/09/21 10:43:13 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4425worker2 # [ 64.797578] nix-grpc-daemon[878]: time=2026-09-21T10:43:13.392Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 64.781787] niks3-server[1025]: 2026/09/21 10:43:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4427lb # [ 64.792922] niks3-server[1025]: 2026/09/21 10:43:13 INFO Signed narinfos id=16 count=14428lb # [ 64.799561] niks3-server[1025]: 2026/09/21 10:43:13 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4429lb # [ 64.808616] niks3-server[1025]: 2026/09/21 10:43:13 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4430lb # [ 64.816119] niks3-server[1025]: 2026/09/21 10:43:13 INFO Completed upload id=164431worker2 # [ 64.825226] nix-grpc-daemon[878]: time=2026-09-21T10:43:13.420Z level=INFO msg="Upload complete. (342ms)"4432lb # [ 64.819356] niks3-server[1025]: 2026/09/21 10:43:13 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4433worker2 # [ 64.837323] nix-grpc-daemon[779]: ts=2026-09-21T10:43:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14434worker2 # [ 64.855665] nix-daemon[863]: reaped child process 1185, status = succeeded4435worker2 # [ 64.881666] nix-grpc-daemon[779]: ts=2026-09-21T10:43:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864436worker2 # [ 64.906826] nix-grpc-daemon[779]: ts=2026-09-21T10:43:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566864437worker2 # [ 64.912164] nix-grpc-daemon[779]: ts=2026-09-21T10:43:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_out=2324438client # [ 65.097777] nix-daemon[937]: reaped child process 939, status = succeeded4439client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.40 seconds)4440client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4441client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4442client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4443client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4444client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4445client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4446(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.54 seconds)4447subtest: an input only one worker has still reaches the builder4448client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4449client # this derivation will be built:4450client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4451client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4452client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.45 seconds)4453client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4454client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4455worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4456worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4457worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4458worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4459client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24460client # [ 66.404262] nix-daemon[937]: accepted connection from pid 1020, user root (trusted)4461client # this derivation will be built:4462client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4463client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4464worker2 # [ 66.571633] nix-grpc-daemon[779]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:566864465worker2 # [ 66.579328] nix-grpc-daemon[779]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864466client # copying 1 paths...4467client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4468worker1 # [ 66.589778] nix-grpc-daemon[775]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:379504469worker1 # [ 66.605109] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4470worker1 # [ 66.818550] nix-daemon[839]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4471worker1 # [ 66.836562] nix-daemon[839]: reaped child process 953, status = succeeded4472lb # [ 66.841288] niks3-server[1025]: 2026/09/21 10:43:15 INFO Received uploads request method=POST path=/api/pending_closures4473worker1 # [ 66.858909] nix-grpc-daemon[868]: time=2026-09-21T10:43:15.444Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4474worker1 # [ 66.860459] nix-grpc-daemon[868]: time=2026-09-21T10:43:15.444Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4475lb # [ 66.866546] niks3-server[1025]: 2026/09/21 10:43:15 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4476lb # [ 66.881996] niks3-server[1025]: 2026/09/21 10:43:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4477worker1 # [ 66.895666] nix-grpc-daemon[868]: time=2026-09-21T10:43:15.481Z level=INFO msg="Uploading 1 narinfos"4478lb # [ 66.894058] niks3-server[1025]: 2026/09/21 10:43:15 INFO Signed narinfos id=17 count=14479lb # [ 66.903244] niks3-server[1025]: 2026/09/21 10:43:15 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4480worker1 # [ 66.917765] nix-grpc-daemon[868]: time=2026-09-21T10:43:15.504Z level=INFO msg="Upload complete. (298ms)"4481lb # [ 66.910250] niks3-server[1025]: 2026/09/21 10:43:15 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4482lb # [ 66.914884] niks3-server[1025]: 2026/09/21 10:43:15 INFO Completed upload id=174483lb # [ 66.917009] niks3-server[1025]: 2026/09/21 10:43:15 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4484worker1 # [ 66.924225] nix-grpc-daemon[775]: ts=2026-09-21T10:43:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_in=1204485worker1 # [ 66.936672] nix-daemon[839]: reaped child process 948, status = succeeded4486worker2 # [ 66.940924] nix-grpc-daemon[779]: ts=2026-09-21T10:43:15Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04487worker2 # [ 66.947241] nix-grpc-daemon[779]: ts=2026-09-21T10:43:15Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14488worker1 # [ 66.983803] nix-grpc-daemon[775]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504489worker1 # [ 66.999528] nix-grpc-daemon[775]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504490client # copying 2 paths to 'lb:50051'...4491client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4492worker1 # [ 67.017328] nix-grpc-daemon[775]: ts=2026-09-21T10:43:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:379504493client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4494worker1 # [ 67.031954] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4495worker1 # [ 67.235068] nix-daemon[839]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4496worker1 # [ 67.252387] nix-daemon[839]: reaped child process 960, status = succeeded4497worker1 # [ 67.422912] nix-daemon[839]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4498worker1 # [ 67.438997] nix-daemon[839]: reaped child process 965, status = succeeded4499lb # [ 67.442487] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received uploads request method=POST path=/api/pending_closures4500lb # [ 67.452584] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received uploads request method=POST path=/api/pending_closures4501worker1 # [ 67.469061] nix-grpc-daemon[868]: time=2026-09-21T10:43:16.055Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4502worker1 # [ 67.470522] nix-grpc-daemon[868]: time=2026-09-21T10:43:16.055Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4503worker1 # [ 67.472264] nix-grpc-daemon[868]: time=2026-09-21T10:43:16.055Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4504lb # [ 67.486230] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4505lb # [ 67.498685] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4506lb # [ 67.511740] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4507lb # [ 67.518333] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4508lb # [ 67.523547] niks3-server[1025]: 2026/09/21 10:43:16 INFO Signed narinfos id=18 count=14509worker1 # [ 67.532154] nix-grpc-daemon[868]: time=2026-09-21T10:43:16.117Z level=INFO msg="Uploading 3 narinfos"4510lb # [ 67.527930] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4511lb # [ 67.535033] niks3-server[1025]: 2026/09/21 10:43:16 INFO Signed narinfos id=19 count=24512lb # [ 67.539055] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4513lb # [ 67.553810] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4514lb # [ 67.564104] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4515lb # [ 67.567091] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4516lb # [ 67.570502] niks3-server[1025]: 2026/09/21 10:43:16 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4517lb # [ 67.573725] niks3-server[1025]: 2026/09/21 10:43:16 INFO Completed upload id=184518lb # [ 67.576701] niks3-server[1025]: 2026/09/21 10:43:16 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4519lb # [ 67.579837] niks3-server[1025]: 2026/09/21 10:43:16 INFO Completed upload id=194520worker1 # [ 67.587914] nix-grpc-daemon[868]: time=2026-09-21T10:43:16.174Z level=INFO msg="Upload complete. (538ms)"4521worker1 # [ 67.601063] nix-grpc-daemon[775]: ts=2026-09-21T10:43:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=2 nar_bytes_in=9844522worker1 # [ 67.616636] nix-grpc-daemon[775]: ts=2026-09-21T10:43:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504523worker1 # [ 67.629546] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4524worker1 # [ 67.633851] nix-daemon[839]: reaped child process 955, status = succeeded4525client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4526worker1 # [ 67.647710] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4527client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4528client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4529worker1 # [ 68.268585] nix-daemon[839]: reaped child process 969, status = succeeded4530worker1 # [ 68.433859] nix-daemon[839]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4531worker1 # [ 68.450083] nix-daemon[839]: reaped child process 988, status = succeeded4532lb # [ 68.454472] niks3-server[1025]: 2026/09/21 10:43:17 INFO Received uploads request method=POST path=/api/pending_closures4533worker1 # [ 68.473989] nix-grpc-daemon[868]: time=2026-09-21T10:43:17.059Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4534worker1 # [ 68.503087] nix-grpc-daemon[868]: time=2026-09-21T10:43:17.089Z level=INFO msg="Uploading 1 narinfos"4535lb # [ 68.485025] niks3-server[1025]: 2026/09/21 10:43:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4536lb # [ 68.501489] niks3-server[1025]: 2026/09/21 10:43:17 INFO Signed narinfos id=20 count=14537lb # [ 68.507787] niks3-server[1025]: 2026/09/21 10:43:17 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4538lb # [ 68.514895] niks3-server[1025]: 2026/09/21 10:43:17 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4539worker1 # [ 68.531682] nix-grpc-daemon[868]: time=2026-09-21T10:43:17.118Z level=INFO msg="Upload complete. (274ms)"4540lb # [ 68.523794] niks3-server[1025]: 2026/09/21 10:43:17 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4541lb # [ 68.528948] niks3-server[1025]: 2026/09/21 10:43:17 INFO Completed upload id=204542lb # [ 68.531838] niks3-server[1025]: 2026/09/21 10:43:17 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4543worker1 # [ 68.540237] nix-grpc-daemon[775]: ts=2026-09-21T10:43:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14544worker1 # [ 68.551247] nix-daemon[839]: reaped child process 967, status = succeeded4545client # copying 1 paths...4546worker2 # [ 68.591886] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864547worker2 # [ 68.606768] nix-grpc-daemon[779]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4548client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4549worker1 # [ 68.650436] nix-grpc-daemon[775]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:379504550worker1 # [ 68.656941] nix-grpc-daemon[775]: ts=2026-09-21T10:43:17Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_out=1204551client # [ 68.794996] nix-daemon[937]: reaped child process 1023, status = succeeded4552client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.63 seconds)4553client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24554client # [ 69.011190] nix-daemon[937]: accepted connection from pid 1042, user root (trusted)4555client # this derivation will be built:4556client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4557worker1 # [ 69.172117] nix-grpc-daemon[775]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:379504558client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4559worker1 # [ 69.179636] nix-grpc-daemon[775]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504560worker1 # [ 69.186301] nix-grpc-daemon[868]: time=2026-09-21T10:43:17.771Z level=INFO msg="All 1 paths already cached"4561worker2 # [ 69.199952] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04562worker2 # [ 69.210179] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14563worker2 # [ 69.219091] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864564worker2 # [ 69.227555] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864565client # copying 1 paths to 'lb:50051'...4566client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4567worker2 # [ 69.244914] nix-grpc-daemon[779]: ts=2026-09-21T10:43:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566864568worker2 # [ 69.259739] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4569worker2 # [ 69.282557] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)4570worker2 # [ 69.378256] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)4571worker2 # [ 69.447761] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 3/5)4572worker2 # [ 69.638406] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 4/5)4573worker2 # [ 69.782799] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4574worker2 # [ 69.788816] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4575worker2 # [ 69.795846] nix-grpc-daemon[779]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4576worker2 # [ 70.019336] nix-daemon[863]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)4577worker2 # [ 70.037687] nix-daemon[863]: reaped child process 1226, status = succeeded4578worker2 # [ 70.214292] nix-daemon[863]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)4579worker2 # [ 70.230477] nix-daemon[863]: reaped child process 1231, status = succeeded4580lb # [ 70.233658] niks3-server[1025]: 2026/09/21 10:43:18 INFO Received uploads request method=POST path=/api/pending_closures4581worker2 # [ 70.253010] nix-grpc-daemon[878]: time=2026-09-21T10:43:18.847Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4582worker2 # [ 70.254451] nix-grpc-daemon[878]: time=2026-09-21T10:43:18.847Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4583lb # [ 70.263219] niks3-server[1025]: 2026/09/21 10:43:18 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4584worker2 # [ 70.284377] nix-grpc-daemon[878]: time=2026-09-21T10:43:18.879Z level=INFO msg="Uploading 1 narinfos"4585lb # [ 70.277101] niks3-server[1025]: 2026/09/21 10:43:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4586lb # [ 70.287847] niks3-server[1025]: 2026/09/21 10:43:18 INFO Signed narinfos id=21 count=14587lb # [ 70.294042] niks3-server[1025]: 2026/09/21 10:43:18 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4588lb # [ 70.305397] niks3-server[1025]: 2026/09/21 10:43:18 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4589worker2 # [ 70.321784] nix-grpc-daemon[878]: time=2026-09-21T10:43:18.916Z level=INFO msg="Upload complete. (504ms)"4590lb # [ 70.312614] niks3-server[1025]: 2026/09/21 10:43:18 INFO Completed upload id=214591lb # [ 70.316116] niks3-server[1025]: 2026/09/21 10:43:18 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4592worker2 # [ 70.331818] nix-grpc-daemon[779]: ts=2026-09-21T10:43:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 paths=1 nar_bytes_in=5684593worker2 # [ 70.339938] nix-grpc-daemon[779]: ts=2026-09-21T10:43:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566864594worker2 # [ 70.352282] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4595worker2 # [ 70.356919] nix-daemon[863]: reaped child process 1216, status = succeeded4596worker2 # [ 70.371778] nix-grpc-daemon[779]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4597client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4598worker2 # [ 70.388437] nix-daemon[863]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4599client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4600client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4601worker2 # [ 71.635818] nix-daemon[863]: reaped child process 1236, status = succeeded4602worker2 # [ 71.821856] nix-daemon[863]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)4603worker2 # [ 71.840605] nix-daemon[863]: reaped child process 1255, status = succeeded4604lb # [ 71.846626] niks3-server[1025]: 2026/09/21 10:43:20 INFO Received uploads request method=POST path=/api/pending_closures4605worker2 # [ 71.867341] nix-grpc-daemon[878]: time=2026-09-21T10:43:20.462Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4606lb # [ 71.879796] niks3-server[1025]: 2026/09/21 10:43:20 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4607worker2 # [ 71.902454] nix-grpc-daemon[878]: time=2026-09-21T10:43:20.497Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 71.894816] niks3-server[1025]: 2026/09/21 10:43:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4609lb # [ 71.905479] niks3-server[1025]: 2026/09/21 10:43:20 INFO Signed narinfos id=22 count=14610lb # [ 71.910173] niks3-server[1025]: 2026/09/21 10:43:20 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4611lb # [ 71.918782] niks3-server[1025]: 2026/09/21 10:43:20 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4612lb # [ 71.923533] niks3-server[1025]: 2026/09/21 10:43:20 INFO Completed upload id=224613worker2 # [ 71.933052] nix-grpc-daemon[878]: time=2026-09-21T10:43:20.528Z level=INFO msg="Upload complete. (330ms)"4614lb # [ 71.926497] niks3-server[1025]: 2026/09/21 10:43:20 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4615worker2 # [ 71.945067] nix-grpc-daemon[779]: ts=2026-09-21T10:43:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14616lb # [ 71.944608] envoy[934]: conn peer=[2001:db8:1::1]:44248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3887 tx=2070 ms=27114617worker2 # [ 71.959964] nix-daemon[863]: reaped child process 1233, status = succeeded4618lb # [ 71.949602] envoy[934]: conn peer=[2001:db8:1::1]:54080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=704619lb # [ 71.954732] envoy[934]: conn peer=[2001:db8:1::1]:54094 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2189 ms=19434620lb # [ 71.959487] envoy[934]: conn peer=[2001:db8:1::1]:54100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=654621lb # [ 71.964707] envoy[934]: conn peer=[2001:db8:1::1]:54112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4234 tx=1960 ms=21044622lb # [ 71.969008] envoy[934]: conn peer=[2001:db8:1::1]:54114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=284623client # copying 1 paths...4624worker2 # [ 71.991380] nix-grpc-daemon[779]: ts=2026-09-21T10:43:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864625client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4626worker2 # [ 72.027611] nix-grpc-daemon[779]: ts=2026-09-21T10:43:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566864627worker2 # [ 72.033129] nix-grpc-daemon[779]: ts=2026-09-21T10:43:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56686 duration_s=0 paths=1 nar_bytes_out=1204628client # [ 72.211023] nix-daemon[937]: reaped child process 1045, status = succeeded4629client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.42 seconds)4630client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24631client # [ 72.436950] nix-daemon[937]: accepted connection from pid 1064, user root (trusted)4632client # this derivation will be built:4633client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4634client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4635worker1 # [ 72.610746] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:379504636worker2 # [ 72.614707] nix-grpc-daemon[779]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566864637worker2 # [ 72.627926] nix-grpc-daemon[878]: time=2026-09-21T10:43:21.222Z level=INFO msg="All 1 paths already cached"4638worker2 # [ 72.644872] nix-grpc-daemon[779]: ts=2026-09-21T10:43:21Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04639worker2 # [ 72.651129] nix-grpc-daemon[779]: ts=2026-09-21T10:43:21Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14640worker1 # [ 72.652463] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504641worker1 # [ 72.667090] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:379504642worker1 # [ 72.687247] nix-grpc-daemon[868]: time=2026-09-21T10:43:21.272Z level=INFO msg="All 1 paths already cached"4643client # copying 1 paths to 'lb:50051'...4644client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4645worker1 # [ 72.697341] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:379504646worker1 # [ 72.712216] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4647worker1 # [ 72.917886] nix-daemon[839]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4648worker1 # [ 72.934913] nix-daemon[839]: reaped child process 995, status = succeeded4649worker1 # [ 73.111914] nix-daemon[839]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4650worker1 # [ 73.130405] nix-daemon[839]: reaped child process 1000, status = succeeded4651lb # [ 73.136709] niks3-server[1025]: 2026/09/21 10:43:21 INFO Received uploads request method=POST path=/api/pending_closures4652worker1 # [ 73.155829] nix-grpc-daemon[868]: time=2026-09-21T10:43:21.742Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4653worker1 # [ 73.157539] nix-grpc-daemon[868]: time=2026-09-21T10:43:21.742Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4654lb # [ 73.166378] niks3-server[1025]: 2026/09/21 10:43:21 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4655worker1 # [ 73.195131] nix-grpc-daemon[868]: time=2026-09-21T10:43:21.781Z level=INFO msg="Uploading 1 narinfos"4656lb # [ 73.179810] niks3-server[1025]: 2026/09/21 10:43:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4657lb # [ 73.191888] niks3-server[1025]: 2026/09/21 10:43:21 INFO Signed narinfos id=23 count=14658lb # [ 73.198729] niks3-server[1025]: 2026/09/21 10:43:21 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4659worker1 # [ 73.219688] nix-grpc-daemon[868]: time=2026-09-21T10:43:21.806Z level=INFO msg="Upload complete. (489ms)"4660lb # [ 73.211811] niks3-server[1025]: 2026/09/21 10:43:21 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4661lb # [ 73.217860] niks3-server[1025]: 2026/09/21 10:43:21 INFO Completed upload id=234662worker1 # [ 73.228238] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_in=5684663lb # [ 73.221786] niks3-server[1025]: 2026/09/21 10:43:21 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4664worker1 # [ 73.240193] nix-grpc-daemon[775]: ts=2026-09-21T10:43:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:379504665worker1 # [ 73.247504] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4666worker1 # [ 73.252823] nix-daemon[839]: reaped child process 990, status = succeeded4667client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4668worker1 # [ 73.266428] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4669client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4670client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4671worker1 # [ 74.489333] nix-daemon[839]: reaped child process 1004, status = succeeded4672worker1 # [ 74.658174] nix-daemon[839]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4673worker1 # [ 74.676959] nix-daemon[839]: reaped child process 1023, status = succeeded4674lb # [ 74.681148] niks3-server[1025]: 2026/09/21 10:43:23 INFO Received uploads request method=POST path=/api/pending_closures4675worker1 # [ 74.700412] nix-grpc-daemon[868]: time=2026-09-21T10:43:23.285Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4676lb # [ 74.716100] niks3-server[1025]: 2026/09/21 10:43:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4677worker1 # [ 74.737037] nix-grpc-daemon[868]: time=2026-09-21T10:43:23.323Z level=INFO msg="Uploading 1 narinfos"4678lb # [ 74.734249] niks3-server[1025]: 2026/09/21 10:43:23 INFO Signed narinfos id=24 count=14679lb # [ 74.736941] niks3-server[1025]: 2026/09/21 10:43:23 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4680lb # [ 74.746135] niks3-server[1025]: 2026/09/21 10:43:23 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4681lb # [ 74.758455] niks3-server[1025]: 2026/09/21 10:43:23 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4682lb # [ 74.768151] niks3-server[1025]: 2026/09/21 10:43:23 INFO Completed upload id=244683worker1 # [ 74.775619] nix-grpc-daemon[868]: time=2026-09-21T10:43:23.362Z level=INFO msg="Upload complete. (299ms)"4684lb # [ 74.770466] niks3-server[1025]: 2026/09/21 10:43:23 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4685worker1 # [ 74.785054] nix-grpc-daemon[775]: ts=2026-09-21T10:43:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14686worker1 # [ 74.801912] nix-daemon[839]: reaped child process 1002, status = succeeded4687client # copying 1 paths...4688worker2 # [ 74.838072] nix-grpc-daemon[779]: ts=2026-09-21T10:43:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566864689worker2 # [ 74.853047] nix-grpc-daemon[779]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4690client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4691worker1 # [ 74.901437] nix-grpc-daemon[775]: ts=2026-09-21T10:43:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:379504692worker1 # [ 74.908851] nix-grpc-daemon[775]: ts=2026-09-21T10:43:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37950 duration_s=0 paths=1 nar_bytes_out=1204693client # [ 75.057360] nix-daemon[937]: reaped child process 1067, status = succeeded4694client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.84 seconds)4695(finished: subtest: an input only one worker has still reaches the builder, in 9.82 seconds)4696subtest: upload whose reference the worker lacks is completed from the cache4697client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4698client # this derivation will be built:4699client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4700client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4701client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.51 seconds)4702client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4703client # this derivation will be built:4704client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4705client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4706client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.45 seconds)4707client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4708client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.22 seconds)4709worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4710worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.26 seconds)4711worker2 # [ 76.561016] systemd[1]: Stopping Nix worker-protocol over gRPC...4712worker2 # [ 77.419872] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=signal_exit4713worker2 # [ 77.663743] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_yield4714worker1 # [ 77.667481] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_restarting addr=lb:500514715worker2 # [ 77.669226] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477164716worker2 # [ 77.675561] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=worker_gone addr=192.168.1.3:500514717lb # [ 77.659984] niks3-server[1025]: 2026/09/21 10:43:26 INFO lead: released remote=192.168.1.4:604884718worker2 # [ 77.681805] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_restarting addr=lb:500514719worker2 # [ 77.685894] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477164720worker2 # [ 77.690315] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=worker_gone addr=192.168.1.4:500514721worker1 # [ 77.873526] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514722worker2 # [ 77.877164] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514723worker1 # [ 77.879140] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4724worker2 # [ 77.882377] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4725worker1 # [ 78.079889] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514726worker2 # [ 78.083588] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514727worker1 # [ 78.085250] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4728worker2 # [ 78.088807] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4729worker1 # [ 78.285980] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514730worker2 # [ 78.289827] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_connected addr=lb:500514731worker1 # [ 78.291250] nix-grpc-daemon[775]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4732worker2 # [ 78.294896] nix-grpc-daemon[779]: ts=2026-09-21T10:43:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4733worker1 # [ 78.491854] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514734worker2 # [ 78.497772] nix-grpc-daemon[779]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514735worker1 # [ 78.497036] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4736worker2 # [ 78.501741] nix-grpc-daemon[779]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4737worker2 # [ 78.669540] nix-grpc-daemon[779]: ts=2026-09-21T10:43:27Z level=info event=scheduler_restarting4738worker1 # [ 78.699563] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514739worker1 # [ 78.703752] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4740worker2 # [ 78.724710] nix-grpc-daemon[779]: ts=2026-09-21T10:43:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4741worker2 # [ 78.736767] nix-daemon[863]: reaped child process 865, status = succeeded4742worker1 # [ 78.900235] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514743worker1 # [ 78.905371] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4744worker1 # [ 79.106160] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514745worker1 # [ 79.111255] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4746worker1 # [ 79.312066] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_connected addr=lb:500514747worker1 # [ 79.317084] nix-grpc-daemon[775]: ts=2026-09-21T10:43:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4748worker1 # [ 79.517610] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_connected addr=lb:500514749worker1 # [ 79.522785] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4750worker2 # [ 79.531232] nix-grpc-daemon[779]: E0000 00:00:1789987408.124710 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4751worker2 # [ 79.537890] nix-grpc-daemon[779]: E0000 00:00:1789987408.125161 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4752worker1 # [ 79.652367] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_take_over4753lb # [ 79.645188] niks3-server[1025]: 2026/09/21 10:43:28 INFO lead: acquired remote=192.168.1.3:342984754worker1 # [ 79.725283] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_connected addr=lb:500514755worker1 # [ 79.729351] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4756worker1 # [ 79.926130] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_connected addr=lb:500514757worker1 # [ 79.931217] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4758worker1 # [ 80.132097] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_connected addr=lb:500514759worker1 # [ 80.137153] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4760worker1 # [ 80.337948] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_connected addr=lb:500514761worker1 # [ 80.343027] nix-grpc-daemon[775]: ts=2026-09-21T10:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4762worker1 # [ 80.543701] nix-grpc-daemon[775]: ts=2026-09-21T10:43:29Z level=info event=scheduler_connected addr=lb:500514763worker1 # [ 80.567582] nix-grpc-daemon[775]: ts=2026-09-21T10:43:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:524704764worker1 # [ 80.572924] nix-grpc-daemon[775]: ts=2026-09-21T10:43:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04765worker2 # [ 80.722231] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4766worker2 # [ 80.729326] systemd[1]: Stopped Nix worker-protocol over gRPC.4767worker2 # [ 80.731890] systemd[1]: nix-grpc-daemon.service: Consumed 3.647s CPU time over 58.624s wall clock time, 65.5M memory peak, 89.9K incoming IP traffic, 217.2K outgoing IP traffic.4768worker2 # [ 80.748532] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4769worker2 # [ 80.755884] systemd[1]: Closed Nix worker-protocol over gRPC.4770worker2 # [ 80.758615] systemd[1]: nix-grpc-daemon.socket: Consumed 588us CPU time over 1min 9.821s wall clock time, 256K memory peak, 90.5K incoming IP traffic, 74.6K outgoing IP traffic.4771lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4772lb: must succeed: curl -sf localhost:9901/clusters4773lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4774lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4775client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24776worker1 # [ 81.115513] nix-grpc-daemon[775]: ts=2026-09-21T10:43:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:379504777client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4778worker1 # [ 81.123027] nix-grpc-daemon[775]: ts=2026-09-21T10:43:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:379504779client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4780worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4781worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4782worker2 # [ 81.266798] systemd[1]: Listening on Nix worker-protocol over gRPC.4783worker1 # [ 81.345198] systemd[1]: Stopping Nix worker-protocol over gRPC...4784worker2 # [ 81.712687] systemd[1]: Starting scheduler-token.service...4785worker2 # [ 81.816002] systemd[1]: scheduler-token.service: Deactivated successfully.4786worker2 # [ 81.825120] systemd[1]: Finished scheduler-token.service.4787worker2 # [ 81.826769] systemd[1]: scheduler-token.service: Consumed 51ms CPU time over 105ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4788worker2 # [ 81.834550] systemd[1]: Starting Nix worker-protocol over gRPC...4789lb # [ 81.945394] envoy[934]: conn peer=[2001:db8:1::1]:54122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3766 tx=1902 ms=29124790lb # [ 81.945714] envoy[934]: conn peer=[2001:db8:1::1]:39140 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=644791lb # [ 81.945839] envoy[934]: conn peer=[2001:db8:1::1]:39150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=1988 ms=23164792lb # [ 81.945952] envoy[934]: conn peer=[2001:db8:1::1]:39164 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=304793lb # [ 81.946084] envoy[934]: conn peer=[2001:db8:1::4]:51854 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=35401 tx=2299 ms=562304794lb # [ 81.946197] envoy[934]: conn peer=[2001:db8:1::1]:50062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=224795worker2 # [ 82.020790] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4796worker2 # [ 82.023501] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4797worker2 # [ 82.027770] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14798worker2 # [ 82.035413] nix-grpc-daemon[1282]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4799worker2 # [ 82.047438] nix-grpc-daemon[1282]: E0000 00:00:1789987410.629822 1282 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4800worker2 # [ 82.055278] nix-grpc-daemon[1282]: E0000 00:00:1789987410.633792 1282 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4801worker2 # [ 82.058468] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4802worker2 # [ 82.061287] systemd[1]: Started Nix worker-protocol over gRPC.4803worker2 # [ 82.064059] nix-grpc-daemon[1282]: E0000 00:00:1789987410.658831 1324 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4804worker2 # [ 82.092063] nix-grpc-daemon[1282]: E0000 00:00:1789987410.676569 1324 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4805worker2 # [ 82.099790] nix-grpc-daemon[1282]: E0000 00:00:1789987410.676723 1324 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4806worker2 # [ 82.102834] nix-grpc-daemon[1282]: E0000 00:00:1789987410.694000 1324 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4807worker2 # [ 82.112252] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:30Z level=info event=scheduler_connected addr=lb:500514808worker1 # [ 82.113422] nix-grpc-daemon[775]: ts=2026-09-21T10:43:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:524704809worker1 # [ 82.120943] nix-grpc-daemon[775]: ts=2026-09-21T10:43:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04810worker1 # [ 82.323723] nix-grpc-daemon[775]: ts=2026-09-21T10:43:30Z level=info event=signal_exit4811worker1 # [ 82.581914] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_yield4812lb # [ 82.577280] niks3-server[1025]: 2026/09/21 10:43:31 INFO lead: released remote=192.168.1.3:342984813worker2 # [ 82.592247] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_restarting addr=lb:500514814worker1 # [ 82.593510] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:524704815worker1 # [ 82.600686] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=worker_gone addr=192.168.1.4:500514816worker1 # [ 82.604906] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_restarting addr=lb:500514817worker1 # [ 82.608859] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:524704818worker1 # [ 82.613200] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=worker_gone addr=192.168.1.3:500514819worker2 # [ 82.793769] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514820worker2 # [ 82.799141] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4821worker1 # [ 82.800000] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514822worker1 # [ 82.805093] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4823worker2 # [ 83.000846] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514824worker2 # [ 83.006135] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4825worker1 # [ 83.005717] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514826worker1 # [ 83.010958] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4827worker2 # [ 83.208886] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514828worker2 # [ 83.213020] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4829worker1 # [ 83.213168] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_connected addr=lb:500514830worker1 # [ 83.218149] nix-grpc-daemon[775]: ts=2026-09-21T10:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4831worker2 # [ 83.409728] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514832worker2 # [ 83.414944] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4833worker1 # [ 83.419237] nix-grpc-daemon[775]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514834worker1 # [ 83.424637] nix-grpc-daemon[775]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4835worker1 # [ 83.587436] nix-grpc-daemon[775]: ts=2026-09-21T10:43:32Z level=info event=scheduler_restarting4836worker2 # [ 83.616086] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514837worker2 # [ 83.621148] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4838worker1 # [ 83.622298] nix-daemon[839]: reaped child process 841, status = succeeded4839worker1 # [ 83.647282] nix-grpc-daemon[775]: ts=2026-09-21T10:43:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4840worker2 # [ 83.822296] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514841worker2 # [ 83.827948] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4842worker2 # [ 84.028963] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514843worker2 # [ 84.034077] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4844worker2 # [ 84.236862] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_connected addr=lb:500514845worker2 # [ 84.241247] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4846worker2 # [ 84.437684] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_connected addr=lb:500514847worker1 # [ 84.436673] nix-grpc-daemon[775]: E0000 00:00:1789987413.021769 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4848worker2 # [ 84.442746] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4849worker1 # [ 84.468602] nix-grpc-daemon[775]: E0000 00:00:1789987413.054998 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4850worker2 # [ 84.643770] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_connected addr=lb:500514851worker1 # [ 84.645631] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4852worker2 # [ 84.648975] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4853worker1 # [ 84.656872] systemd[1]: Stopped Nix worker-protocol over gRPC.4854worker1 # [ 84.659441] systemd[1]: nix-grpc-daemon.service: Consumed 1.854s CPU time over 1min 2.695s wall clock time, 51.8M memory peak, 54.1K incoming IP traffic, 95.3K outgoing IP traffic.4855worker1 # [ 84.673753] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4856worker1 # [ 84.676010] systemd[1]: Closed Nix worker-protocol over gRPC.4857worker1 # [ 84.682666] systemd[1]: nix-grpc-daemon.socket: Consumed 628us CPU time over 1min 13.355s wall clock time, 256K memory peak, 69.5K incoming IP traffic, 60.8K outgoing IP traffic.4858lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4859lb: must succeed: curl -sf localhost:9901/clusters4860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4861lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4862client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24863worker2 # [ 84.849981] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_connected addr=lb:500514864worker2 # [ 84.855063] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker2 # [ 85.038628] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541644866worker2 # [ 85.057455] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)4867worker2 # [ 85.068254] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_connected addr=lb:500514868worker2 # [ 85.075534] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4869worker2 # [ 85.094740] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)4870worker2 # [ 85.176868] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 2/5)4871worker2 # [ 85.260095] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_connected addr=lb:500514872worker2 # [ 85.264080] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4873worker2 # [ 85.383422] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 3/5)4874worker2 # [ 85.397651] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 4/5)4875worker2 # [ 85.460891] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_connected addr=lb:500514876worker2 # [ 85.465930] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4877worker2 # [ 85.667098] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_connected addr=lb:500514878worker2 # [ 85.672234] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4879worker2 # [ 85.701455] nix-grpc-daemon[1282]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4880worker2 # [ 85.707062] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4881client # copying 2 paths...4882client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4883worker2 # [ 85.731251] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541644884client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4885worker2 # [ 85.747103] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)4886worker2 # [ 85.780722] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=niks3_push_started pid=13424887worker2 # [ 85.873152] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_connected addr=lb:500514888worker2 # [ 85.875702] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4889worker2 # [ 85.997956] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.592Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4890worker2 # [ 86.075779] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_connected addr=lb:500514891worker2 # [ 86.078450] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4892worker2 # [ 86.181198] nix-daemon[863]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)4893worker2 # [ 86.198612] nix-daemon[863]: reaped child process 1358, status = succeeded4894lb # [ 86.202728] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received uploads request method=POST path=/api/pending_closures4895lb # [ 86.212596] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received uploads request method=POST path=/api/pending_closures4896worker2 # [ 86.232093] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.826Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4897worker2 # [ 86.233609] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.826Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4898worker2 # [ 86.235316] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.826Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4899lb # [ 86.262875] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4900worker2 # [ 86.279797] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_connected addr=lb:500514901worker2 # [ 86.283363] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4902lb # [ 86.275852] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4903lb # [ 86.287686] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4904lb # [ 86.303415] niks3-server[1025]: 2026/09/21 10:43:34 INFO Signed narinfos id=26 count=24905lb # [ 86.308903] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4906lb # [ 86.316626] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4907lb # [ 86.321254] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4908worker2 # [ 86.332113] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.926Z level=INFO msg="Uploading 3 narinfos"4909lb # [ 86.324415] niks3-server[1025]: 2026/09/21 10:43:34 INFO Signed narinfos id=25 count=14910lb # [ 86.352344] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4911lb # [ 86.361312] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4912lb # [ 86.368060] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4913lb # [ 86.372410] niks3-server[1025]: 2026/09/21 10:43:34 INFO Completed upload id=254914lb # [ 86.376885] niks3-server[1025]: 2026/09/21 10:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4915worker2 # [ 86.386273] nix-grpc-daemon[1342]: time=2026-09-21T10:43:34.981Z level=INFO msg="Upload complete. (384ms)"4916lb # [ 86.379519] niks3-server[1025]: 2026/09/21 10:43:34 INFO Completed upload id=264917lb # [ 86.385471] niks3-server[1025]: 2026/09/21 10:43:34 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4918worker2 # [ 86.396458] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 paths=2 nar_bytes_in=3124919worker2 # [ 86.414931] nix-daemon[863]: reaped child process 1340, status = succeeded4920client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.66 seconds)4921worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4922worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4923worker2 # [ 86.484752] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:35Z level=info event=scheduler_connected addr=lb:500514924worker2 # [ 86.489551] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4925worker1 # [ 86.558710] systemd[1]: Listening on Nix worker-protocol over gRPC.4926worker1 # [ 86.568411] systemd[1]: Starting Nix worker-protocol over gRPC...4927worker2 # [ 86.690689] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:35Z level=info event=scheduler_connected addr=lb:500514928worker2 # [ 86.696119] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4929worker1 # [ 86.768137] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4930worker1 # [ 86.771714] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4931worker1 # [ 86.776301] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14932worker1 # [ 86.784073] nix-grpc-daemon[1046]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4933worker1 # [ 86.785550] nix-grpc-daemon[1046]: E0000 00:00:1789987415.369502 1046 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4934worker1 # [ 86.799210] nix-grpc-daemon[1046]: E0000 00:00:1789987415.376062 1046 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4935worker1 # [ 86.807878] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4936lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4937lb: must succeed: curl -sf localhost:9901/clusters4938worker1 # [ 86.811779] systemd[1]: Started Nix worker-protocol over gRPC.4939lb # [ 86.801921] niks3-server[1025]: 2026/09/21 10:43:35 INFO lead: acquired remote=192.168.1.3:601364940worker1 # [ 86.812906] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=scheduler_take_over4941worker1 # [ 86.816636] nix-grpc-daemon[1046]: E0000 00:00:1789987415.402925 1088 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4942worker1 # [ 86.844059] nix-grpc-daemon[1046]: E0000 00:00:1789987415.420794 1088 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4943worker1 # [ 86.846438] nix-grpc-daemon[1046]: E0000 00:00:1789987415.420959 1088 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4944worker1 # [ 86.855072] nix-grpc-daemon[1046]: E0000 00:00:1789987415.440616 1088 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4945worker1 # [ 86.871223] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=scheduler_connected addr=lb:500514946worker1 # [ 86.874990] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:571544947worker1 # [ 86.876824] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04948worker2 # [ 86.897174] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:35Z level=info event=scheduler_connected addr=lb:500514949worker1 # [ 86.902473] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=oidc_jwks provider=mock keys=14950worker1 # [ 86.903821] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:571544951worker1 # [ 86.905878] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04952lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4953lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.11 seconds)4954(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.90 seconds)4955subtest: scheduler down: the other node takes the lock and keeps it4956lb: must succeed: curl -sf localhost:9901/clusters4957lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4958lb: must succeed: curl -sf localhost:9901/clusters4959lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4960lb: must succeed: curl -sf localhost:9901/clusters4961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4962lb: must succeed: curl -sf localhost:9901/clusters4963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4964worker1 # [ 87.172218] systemd[1]: Stopping Nix worker-protocol over gRPC...4965worker1 # [ 87.793672] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=signal_exit4966worker1 # [ 88.010412] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=scheduler_yield4967lb # [ 88.005334] niks3-server[1025]: 2026/09/21 10:43:36 INFO lead: released remote=192.168.1.3:601364968worker2 # [ 88.018041] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:36Z level=info event=scheduler_restarting addr=lb:500514969worker1 # [ 88.025100] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:571544970worker1 # [ 88.032429] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=worker_gone addr=192.168.1.4:500514971worker1 # [ 88.036616] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=scheduler_restarting addr=lb:500514972worker1 # [ 88.040583] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:571544973worker1 # [ 88.044959] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=worker_gone addr=192.168.1.3:500514974worker2 # [ 88.218162] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:36Z level=info event=scheduler_connected addr=lb:500514975worker2 # [ 88.223397] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4976worker1 # [ 88.231903] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=scheduler_connected addr=lb:500514977worker1 # [ 88.237187] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4978worker2 # [ 88.424399] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514979worker2 # [ 88.429461] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4980worker1 # [ 88.437703] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514981worker1 # [ 88.442887] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4982worker2 # [ 88.630478] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514983worker2 # [ 88.635574] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4984worker1 # [ 88.643800] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514985worker1 # [ 88.649525] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4986worker2 # [ 88.836766] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514987worker2 # [ 88.841712] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4988worker1 # [ 88.851850] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514989worker1 # [ 88.856388] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4990worker1 # [ 89.015740] nix-grpc-daemon[1046]: ts=2026-09-21T10:43:37Z level=info event=scheduler_restarting4991worker2 # [ 89.044478] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514992worker2 # [ 89.049405] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4993worker2 # [ 89.245260] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_connected addr=lb:500514994worker2 # [ 89.250300] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4995worker2 # [ 89.451310] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_connected addr=lb:500514996worker2 # [ 89.456454] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4997worker2 # [ 89.657486] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_connected addr=lb:500514998worker2 # [ 89.662221] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker1 # [ 89.808240] nix-grpc-daemon[1046]: E0000 00:00:1789987418.394481 1091 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5000worker1 # [ 89.825261] nix-grpc-daemon[1046]: E0000 00:00:1789987418.411597 1091 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5001worker2 # [ 89.863189] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_connected addr=lb:500515002worker2 # [ 89.868253] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker2 # [ 90.070792] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_connected addr=lb:500515004worker2 # [ 90.074883] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2 # [ 90.271711] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 90.276942] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker2 # [ 90.477870] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_connected addr=lb:500515008worker2 # [ 90.482894] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5009worker2 # [ 90.683927] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_connected addr=lb:500515010worker2 # [ 90.689055] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker1 # [ 90.822949] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5012worker1 # [ 90.826228] systemd[1]: Stopped Nix worker-protocol over gRPC.5013worker1 # [ 90.831448] systemd[1]: nix-grpc-daemon.service: Consumed 207ms CPU time over 4.250s wall clock time, 13.3M memory peak, 9.5K incoming IP traffic, 7.9K outgoing IP traffic.5014worker1 # [ 90.847214] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5015worker1 # [ 90.850445] systemd[1]: Closed Nix worker-protocol over gRPC.5016worker1 # [ 90.853846] systemd[1]: nix-grpc-daemon.socket: Consumed 991us CPU time over 4.300s wall clock time, 256K memory peak, 13.6K incoming IP traffic, 10.7K outgoing IP traffic.5017worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5018worker2 # [ 90.891564] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5019worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5020worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5021worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5022worker2 # [ 91.294636] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_connected addr=lb:500515023worker2 # [ 91.298833] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5024worker2 # [ 91.495597] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_connected addr=lb:500515025worker2 # [ 91.500798] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker2 # [ 91.701854] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_connected addr=lb:500515027worker2 # [ 91.706927] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5028worker2 # [ 91.907902] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_connected addr=lb:500515029worker2 # [ 91.913025] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5030lb # [ 91.948204] envoy[934]: conn peer=[2001:db8:1::3]:39792 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9002 tx=2455 ms=612565031lb # [ 91.948532] envoy[934]: conn peer=[2001:db8:1::1]:50070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=13745032lb # [ 91.948663] envoy[934]: conn peer=[2001:db8:1::3]:35812 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1028 tx=349 ms=21945033worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5034worker2 # [ 92.059597] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_take_over5035lb # [ 92.050938] niks3-server[1025]: 2026/09/21 10:43:40 INFO lead: acquired remote=192.168.1.4:385025036worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5037worker2 # [ 92.113951] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_connected addr=lb:500515038worker2 # [ 92.119071] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5039worker2 # [ 92.322136] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_connected addr=lb:500515040worker2 # [ 92.326431] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker2 # [ 92.523001] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_connected addr=lb:500515042worker2 # [ 92.528400] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker2 # [ 92.729334] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 92.734513] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker2 # [ 92.935538] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 92.940743] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5048worker2 # [ 93.141800] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=scheduler_connected addr=lb:500515049worker2 # [ 93.167065] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=oidc_jwks provider=mock keys=15050worker2 # [ 93.172958] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:382345051worker2 # [ 93.176815] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05052worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds)5053client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25054client # don't know how to build these paths:5055client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5056worker2 # [ 93.443778] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541645057worker2 # [ 93.452768] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541645058worker2 # [ 93.474179] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05059worker2 # [ 93.480735] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15060worker2 # [ 93.521321] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645061worker2 # [ 93.536994] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541645062client # copying 1 paths to 'lb:50051'...5063client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5064worker2 # [ 93.552879] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541645065worker2 # [ 93.568663] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5066worker2 # [ 93.775663] nix-daemon[863]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5067worker2 # [ 93.796863] nix-daemon[863]: reaped child process 1385, status = succeeded5068worker2 # [ 93.972959] nix-daemon[863]: accepted connection from pid 1387, user nix-grpc-daemon (trusted)5069worker2 # [ 93.990471] nix-daemon[863]: reaped child process 1390, status = succeeded5070lb # [ 93.993310] niks3-server[1025]: 2026/09/21 10:43:42 INFO Received uploads request method=POST path=/api/pending_closures5071worker2 # [ 94.012443] nix-grpc-daemon[1342]: time=2026-09-21T10:43:42.606Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5072worker2 # [ 94.013926] nix-grpc-daemon[1342]: time=2026-09-21T10:43:42.606Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5073lb # [ 94.025375] niks3-server[1025]: 2026/09/21 10:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5074worker2 # [ 94.046233] nix-grpc-daemon[1342]: time=2026-09-21T10:43:42.641Z level=INFO msg="Uploading 1 narinfos"5075lb # [ 94.040103] niks3-server[1025]: 2026/09/21 10:43:42 INFO Signed narinfos id=27 count=15076lb # [ 94.043248] niks3-server[1025]: 2026/09/21 10:43:42 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5077lb # [ 94.056873] niks3-server[1025]: 2026/09/21 10:43:42 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5078worker2 # [ 94.073728] nix-grpc-daemon[1342]: time=2026-09-21T10:43:42.668Z level=INFO msg="Upload complete. (488ms)"5079lb # [ 94.065646] niks3-server[1025]: 2026/09/21 10:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5080lb # [ 94.071468] niks3-server[1025]: 2026/09/21 10:43:42 INFO Completed upload id=275081lb # [ 94.075399] niks3-server[1025]: 2026/09/21 10:43:42 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5082worker2 # [ 94.080622] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 paths=1 nar_bytes_in=5205083worker2 # [ 94.095542] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645084worker2 # [ 94.102476] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5085worker2 # [ 94.107552] nix-daemon[863]: reaped child process 1380, status = succeeded5086client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5087client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5088worker2 # [ 94.121719] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5089client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5090client # farm-a> LOG-farm-a-on-standby5091worker2 # [ 95.560866] nix-daemon[863]: reaped child process 1394, status = succeeded5092worker2 # [ 95.734571] nix-daemon[863]: accepted connection from pid 1410, user nix-grpc-daemon (trusted)5093worker2 # [ 95.752120] nix-daemon[863]: reaped child process 1413, status = succeeded5094lb # [ 95.757639] niks3-server[1025]: 2026/09/21 10:43:44 INFO Received uploads request method=POST path=/api/pending_closures5095worker2 # [ 95.777028] nix-grpc-daemon[1342]: time=2026-09-21T10:43:44.371Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5096worker2 # [ 95.780901] nix-grpc-daemon[1342]: time=2026-09-21T10:43:44.373Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5097lb # [ 95.791085] niks3-server[1025]: 2026/09/21 10:43:44 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5098lb # [ 95.811340] niks3-server[1025]: 2026/09/21 10:43:44 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5099lb # [ 95.820379] niks3-server[1025]: 2026/09/21 10:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5100lb # [ 95.826608] niks3-server[1025]: 2026/09/21 10:43:44 INFO Signed narinfos id=28 count=15101worker2 # [ 95.837113] nix-grpc-daemon[1342]: time=2026-09-21T10:43:44.430Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 95.830159] niks3-server[1025]: 2026/09/21 10:43:44 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5103lb # [ 95.843462] niks3-server[1025]: 2026/09/21 10:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5104lb # [ 95.849869] niks3-server[1025]: 2026/09/21 10:43:44 INFO Completed upload id=285105worker2 # [ 95.859547] nix-grpc-daemon[1342]: time=2026-09-21T10:43:44.454Z level=INFO msg="Upload complete. (327ms)"5106lb # [ 95.853001] niks3-server[1025]: 2026/09/21 10:43:44 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5107worker2 # [ 95.879017] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15108worker2 # [ 95.895552] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05109worker2 # [ 95.899716] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15110worker2 # [ 95.904702] nix-daemon[863]: reaped child process 1392, status = succeeded5111worker2 # [ 95.921858] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645112worker2 # [ 95.936800] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541645113worker2 # [ 95.949563] nix-grpc-daemon[1342]: time=2026-09-21T10:43:44.544Z level=INFO msg="All 1 paths already cached"5114client # copying 1 paths to 'lb:50051'...5115client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5116worker2 # [ 95.959182] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541645117worker2 # [ 95.975016] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5118worker2 # [ 96.175657] nix-daemon[863]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5119worker2 # [ 96.194154] nix-daemon[863]: reaped child process 1422, status = succeeded5120worker2 # [ 96.366989] nix-daemon[863]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)5121worker2 # [ 96.384142] nix-daemon[863]: reaped child process 1427, status = succeeded5122lb # [ 96.386946] niks3-server[1025]: 2026/09/21 10:43:44 INFO Received uploads request method=POST path=/api/pending_closures5123worker2 # [ 96.405919] nix-grpc-daemon[1342]: time=2026-09-21T10:43:45.000Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5124worker2 # [ 96.407440] nix-grpc-daemon[1342]: time=2026-09-21T10:43:45.000Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5125lb # [ 96.413411] niks3-server[1025]: 2026/09/21 10:43:45 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5126worker2 # [ 96.438333] nix-grpc-daemon[1342]: time=2026-09-21T10:43:45.033Z level=INFO msg="Uploading 1 narinfos"5127lb # [ 96.431794] niks3-server[1025]: 2026/09/21 10:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5128lb # [ 96.442905] niks3-server[1025]: 2026/09/21 10:43:45 INFO Signed narinfos id=29 count=15129lb # [ 96.447711] niks3-server[1025]: 2026/09/21 10:43:45 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5130lb # [ 96.456267] niks3-server[1025]: 2026/09/21 10:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5131worker2 # [ 96.470324] nix-grpc-daemon[1342]: time=2026-09-21T10:43:45.065Z level=INFO msg="Upload complete. (479ms)"5132lb # [ 96.461396] niks3-server[1025]: 2026/09/21 10:43:45 INFO Completed upload id=295133lb # [ 96.465274] niks3-server[1025]: 2026/09/21 10:43:45 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5134worker2 # [ 96.480773] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 paths=1 nar_bytes_in=6405135worker2 # [ 96.489574] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645136worker2 # [ 96.495910] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5137worker2 # [ 96.501262] nix-daemon[863]: reaped child process 1417, status = succeeded5138client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5139worker2 # [ 96.513686] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5140client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5141client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5142client # farm-b> LOG-farm-b-on-standby5143worker2 # [ 97.557512] nix-daemon[863]: reaped child process 1431, status = succeeded5144worker2 # [ 97.726743] nix-daemon[863]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5145worker2 # [ 97.745103] nix-daemon[863]: reaped child process 1450, status = succeeded5146lb # [ 97.747791] niks3-server[1025]: 2026/09/21 10:43:46 INFO Received uploads request method=POST path=/api/pending_closures5147worker2 # [ 97.768719] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.363Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5148worker2 # [ 97.770906] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.365Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5149lb # [ 97.784181] niks3-server[1025]: 2026/09/21 10:43:46 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5150worker2 # [ 97.810347] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.405Z level=INFO msg="Uploading 1 narinfos"5151lb # [ 97.802385] niks3-server[1025]: 2026/09/21 10:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5152lb # [ 97.813572] niks3-server[1025]: 2026/09/21 10:43:46 INFO Signed narinfos id=30 count=15153lb # [ 97.819255] niks3-server[1025]: 2026/09/21 10:43:46 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5154lb # [ 97.824435] niks3-server[1025]: 2026/09/21 10:43:46 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5155lb # [ 97.831509] niks3-server[1025]: 2026/09/21 10:43:46 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5156lb # [ 97.835875] niks3-server[1025]: 2026/09/21 10:43:46 INFO Completed upload id=305157worker2 # [ 97.845586] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.440Z level=INFO msg="Upload complete. (322ms)"5158lb # [ 97.838357] niks3-server[1025]: 2026/09/21 10:43:46 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5159worker2 # [ 97.864266] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15160worker2 # [ 97.877823] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05161worker2 # [ 97.881479] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15162worker2 # [ 97.885319] nix-daemon[863]: reaped child process 1429, status = succeeded5163worker2 # [ 97.905805] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645164worker2 # [ 97.920977] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541645165client # copying 1 paths to 'lb:50051'...5166worker2 # [ 97.934803] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.529Z level=INFO msg="All 2 paths already cached"5167client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5168worker2 # [ 97.941544] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541645169worker2 # [ 97.956269] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5170worker2 # [ 98.157233] nix-daemon[863]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5171worker2 # [ 98.174445] nix-daemon[863]: reaped child process 1459, status = succeeded5172worker2 # [ 98.346151] nix-daemon[863]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)5173worker2 # [ 98.363539] nix-daemon[863]: reaped child process 1464, status = succeeded5174lb # [ 98.365282] niks3-server[1025]: 2026/09/21 10:43:46 INFO Received uploads request method=POST path=/api/pending_closures5175worker2 # [ 98.384129] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.978Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5176worker2 # [ 98.385588] nix-grpc-daemon[1342]: time=2026-09-21T10:43:46.978Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5177lb # [ 98.394636] niks3-server[1025]: 2026/09/21 10:43:46 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5178worker2 # [ 98.413247] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.008Z level=INFO msg="Uploading 1 narinfos"5179lb # [ 98.406813] niks3-server[1025]: 2026/09/21 10:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5180lb # [ 98.417885] niks3-server[1025]: 2026/09/21 10:43:46 INFO Signed narinfos id=31 count=15181lb # [ 98.426021] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5182worker2 # [ 98.440739] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.035Z level=INFO msg="Upload complete. (468ms)"5183lb # [ 98.432182] niks3-server[1025]: 2026/09/21 10:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5184lb # [ 98.438235] niks3-server[1025]: 2026/09/21 10:43:47 INFO Completed upload id=315185worker2 # [ 98.448125] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 paths=1 nar_bytes_in=7685186lb # [ 98.441293] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5187worker2 # [ 98.456745] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645188worker2 # [ 98.466322] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5189worker2 # [ 98.468503] nix-daemon[863]: reaped child process 1454, status = succeeded5190client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5191worker2 # [ 98.480350] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5192client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5193client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5194client # farm-top> LOG-farm-top-on-standby5195worker2 # [ 99.113236] nix-daemon[863]: reaped child process 1468, status = succeeded5196worker2 # [ 99.274161] nix-daemon[863]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5197worker2 # [ 99.291143] nix-daemon[863]: reaped child process 1487, status = succeeded5198lb # [ 99.297008] niks3-server[1025]: 2026/09/21 10:43:47 INFO Received uploads request method=POST path=/api/pending_closures5199worker2 # [ 99.318502] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.913Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5200worker2 # [ 99.321767] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.915Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5201lb # [ 99.338931] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5202lb # [ 99.350931] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5203lb # [ 99.359955] niks3-server[1025]: 2026/09/21 10:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5204worker2 # [ 99.376336] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.970Z level=INFO msg="Uploading 1 narinfos"5205lb # [ 99.368910] niks3-server[1025]: 2026/09/21 10:43:47 INFO Signed narinfos id=32 count=15206lb # [ 99.371701] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5207lb # [ 99.382960] niks3-server[1025]: 2026/09/21 10:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5208lb # [ 99.390709] niks3-server[1025]: 2026/09/21 10:43:47 INFO Completed upload id=325209lb # [ 99.393782] niks3-server[1025]: 2026/09/21 10:43:47 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5210worker2 # [ 99.400392] nix-grpc-daemon[1342]: time=2026-09-21T10:43:47.995Z level=INFO msg="Upload complete. (312ms)"5211worker2 # [ 99.413293] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15212worker2 # [ 99.431816] nix-daemon[863]: reaped child process 1466, status = succeeded5213client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.28 seconds)5214client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5215client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5216worker1 # [ 99.780914] systemd[1]: Listening on Nix worker-protocol over gRPC.5217worker1 # [ 99.788650] systemd[1]: Starting Nix worker-protocol over gRPC...5218worker1 # [ 99.986121] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5219worker1 # [ 99.988777] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5220worker1 # [ 99.992867] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15221worker1 # [ 100.000067] nix-grpc-daemon[1113]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5222worker1 # [ 100.016646] nix-grpc-daemon[1113]: E0000 00:00:1789987428.586082 1113 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5223worker1 # [ 100.019462] nix-grpc-daemon[1113]: E0000 00:00:1789987428.589674 1113 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5224worker1 # [ 100.024968] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5225worker1 # [ 100.027162] systemd[1]: Started Nix worker-protocol over gRPC.5226worker1 # [ 100.032182] nix-grpc-daemon[1113]: E0000 00:00:1789987428.616513 1155 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5227lb: must succeed: curl -sf localhost:9901/clusters5228worker1 # [ 100.056154] nix-grpc-daemon[1113]: E0000 00:00:1789987428.634326 1155 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5229worker1 # [ 100.058429] nix-grpc-daemon[1113]: E0000 00:00:1789987428.634475 1155 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5230worker1 # [ 100.068568] nix-grpc-daemon[1113]: E0000 00:00:1789987428.653952 1155 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5231worker1 # [ 100.087841] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:48Z level=info event=scheduler_connected addr=lb:500515232worker2 # [ 100.089718] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:382345233worker2 # [ 100.096076] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5235worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5236worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5237lb: must succeed: curl -sf localhost:9901/clusters5238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5239lb: must succeed: curl -sf localhost:9901/clusters5240lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5241worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5242worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5243(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.51 seconds)5244subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5245client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5246client # [ 100.548006] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5247client # Running as unit: rs.service; invocation ID: 558da3cc9dd2458d94860d3336af72ce5248client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5249worker1 # [ 100.744787] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514545250worker1 # [ 100.760504] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5251client # [ 100.830198] nix[1168]: don't know how to build these paths:5252client # [ 100.831722] nix[1168]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5253worker2 # [ 100.793339] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05254worker2 # [ 100.800051] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15255worker2 # [ 100.846133] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645256worker2 # [ 100.861058] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541645257client # [ 100.924118] nix[1168]: copying 1 paths to 'lb:50051'...5258worker2 # [ 100.874547] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541645259client # [ 100.931218] nix[1168]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5260worker2 # [ 100.889246] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5261worker2 # [ 101.095380] nix-daemon[863]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)5262worker2 # [ 101.114706] nix-daemon[863]: reaped child process 1503, status = succeeded5263worker2 # [ 101.288495] nix-daemon[863]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)5264worker2 # [ 101.303673] nix-daemon[863]: reaped child process 1508, status = succeeded5265lb # [ 101.309267] niks3-server[1025]: 2026/09/21 10:43:49 INFO Received uploads request method=POST path=/api/pending_closures5266worker2 # [ 101.328424] nix-grpc-daemon[1342]: time=2026-09-21T10:43:49.923Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5267worker2 # [ 101.329912] nix-grpc-daemon[1342]: time=2026-09-21T10:43:49.923Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5268lb # [ 101.346722] niks3-server[1025]: 2026/09/21 10:43:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5269worker2 # [ 101.366149] nix-grpc-daemon[1342]: time=2026-09-21T10:43:49.961Z level=INFO msg="Uploading 1 narinfos"5270lb # [ 101.361537] niks3-server[1025]: 2026/09/21 10:43:49 INFO Signed narinfos id=33 count=15271lb # [ 101.368949] niks3-server[1025]: 2026/09/21 10:43:49 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5272lb # [ 101.375959] niks3-server[1025]: 2026/09/21 10:43:49 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5273worker2 # [ 101.393506] nix-grpc-daemon[1342]: time=2026-09-21T10:43:49.988Z level=INFO msg="Upload complete. (487ms)"5274lb # [ 101.384527] niks3-server[1025]: 2026/09/21 10:43:49 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5275lb # [ 101.389645] niks3-server[1025]: 2026/09/21 10:43:49 INFO Completed upload id=335276lb # [ 101.393421] niks3-server[1025]: 2026/09/21 10:43:49 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5277worker2 # [ 101.400244] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=0 paths=1 nar_bytes_in=4165278worker2 # [ 101.413315] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541645279worker2 # [ 101.423577] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5280worker2 # [ 101.426170] nix-daemon[863]: reaped child process 1498, status = succeeded5281client # [ 101.486938] nix[1168]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5282worker2 # [ 101.439595] nix-daemon[863]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5283lb # [ 101.946278] envoy[934]: conn peer=[2001:db8:1::1]:43808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=4071 ms=59845284lb: must succeed: curl -sf localhost:9901/clusters5285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5286lb: must succeed: curl -sf localhost:9901/clusters5287lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5288lb: must succeed: curl -sf localhost:9901/clusters5289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5290lb: must succeed: curl -sf localhost:9901/clusters5291lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5292worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5293worker1 # [ 103.442305] systemd-journald[274]: Received client request to rotate journal, rotating.5294worker1 # [ 103.469590] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9c51fb1631aa4c9f95abf019cc10d679.5295worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5296worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5297worker2 # [ 103.526459] systemd[1]: Stopping Nix worker-protocol over gRPC...5298worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5299client: must succeed: sleep 5.05300worker2 # [ 104.055873] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:52Z level=info event=draining5301client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5302worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5303worker2 # [ 109.094416] nix-daemon[863]: reaped child process 1512, status = succeeded5304worker2 # [ 109.279513] nix-daemon[863]: accepted connection from pid 1553, user nix-grpc-daemon (trusted)5305worker2 # [ 109.295644] nix-daemon[863]: reaped child process 1556, status = succeeded5306lb # [ 109.301656] niks3-server[1025]: 2026/09/21 10:43:57 INFO Received uploads request method=POST path=/api/pending_closures5307worker2 # [ 109.323585] nix-grpc-daemon[1342]: time=2026-09-21T10:43:57.916Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5308worker2 # [ 109.325162] nix-grpc-daemon[1342]: time=2026-09-21T10:43:57.916Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5309lb # [ 109.342192] niks3-server[1025]: 2026/09/21 10:43:57 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5310lb # [ 109.354814] niks3-server[1025]: 2026/09/21 10:43:57 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5311worker2 # [ 109.371944] nix-grpc-daemon[1342]: time=2026-09-21T10:43:57.966Z level=INFO msg="Uploading 1 narinfos"5312lb # [ 109.365003] niks3-server[1025]: 2026/09/21 10:43:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5313lb # [ 109.373071] niks3-server[1025]: 2026/09/21 10:43:57 INFO Signed narinfos id=34 count=15314lb # [ 109.377353] niks3-server[1025]: 2026/09/21 10:43:57 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5315lb # [ 109.383999] niks3-server[1025]: 2026/09/21 10:43:57 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5316lb # [ 109.389842] niks3-server[1025]: 2026/09/21 10:43:57 INFO Completed upload id=345317worker2 # [ 109.399045] nix-grpc-daemon[1342]: time=2026-09-21T10:43:57.994Z level=INFO msg="Upload complete. (343ms)"5318lb # [ 109.393028] niks3-server[1025]: 2026/09/21 10:43:57 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5319worker2 # [ 109.412990] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54164 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15320worker2 # [ 109.427312] nix-daemon[863]: reaped child process 1510, status = succeeded5321client # [ 109.504942] systemd[1]: rs.service: Deactivated successfully.5322client # [ 109.508891] systemd[1]: rs.service: Consumed 175ms CPU time over 8.960s wall clock time, 39.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5323worker2 # [ 110.058748] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=signal_exit5324worker2 # [ 110.282831] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=scheduler_yield5325lb # [ 110.278158] niks3-server[1025]: 2026/09/21 10:43:58 INFO lead: released remote=192.168.1.4:385025326worker2 # [ 110.288417] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=scheduler_restarting addr=lb:500515327worker1 # [ 110.287288] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:58Z level=info event=scheduler_restarting addr=lb:500515328worker2 # [ 110.294802] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382345329worker2 # [ 110.299506] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=worker_gone addr=192.168.1.3:500515330worker2 # [ 110.303645] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382345331worker2 # [ 110.308178] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:58Z level=info event=worker_gone addr=192.168.1.4:500515332worker1 # [ 110.487358] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515333worker2 # [ 110.491778] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515334worker1 # [ 110.493076] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5335worker2 # [ 110.496102] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker2 # [ 110.692254] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 110.693876] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515338worker2 # [ 110.697339] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5339worker1 # [ 110.699153] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker2 # [ 110.898378] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515341worker1 # [ 110.900259] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515342worker2 # [ 110.903621] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5343worker1 # [ 110.905606] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker2 # [ 111.104738] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 111.107319] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 111.109917] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 111.113992] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.23 seconds)5349client: waiting for success: ! systemctl is-active rs5350client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5351client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5352client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5353client: must fail: journalctl -u rs | grep -q 'reconnecting'5354worker2 # [ 111.288271] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=scheduler_restarting5355client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5356lb: must succeed: curl -sf localhost:9901/clusters5357worker1 # [ 111.310496] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_connected addr=lb:500515358worker1 # [ 111.314923] nix-grpc-daemon[1113]: ts=2026-09-21T10:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker2 # [ 111.342976] nix-grpc-daemon[1282]: ts=2026-09-21T10:43:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5360worker2 # [ 111.348554] nix-daemon[863]: reaped child process 1334, status = succeeded5361lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5362worker1 # [ 111.511322] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 111.516715] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker1 # [ 111.717692] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 111.723164] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker1 # [ 111.924956] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_connected addr=lb:500515367worker1 # [ 111.930804] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5368lb # [ 111.946652] envoy[934]: conn peer=[2001:db8:1::1]:43816 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1407 ms=86805369lb # [ 111.952076] envoy[934]: conn peer=[2001:db8:1::4]:41960 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39170 tx=2258 ms=292125370worker2 # [ 112.104675] nix-grpc-daemon[1282]: E0000 00:00:1789987440.697620 1327 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5371worker1 # [ 112.136807] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_connected addr=lb:500515372worker2 # [ 112.136877] nix-grpc-daemon[1282]: E0000 00:00:1789987440.731646 1327 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5373worker1 # [ 112.141238] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker2 # [ 112.283944] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5375worker2 # [ 112.287074] systemd[1]: Stopped Nix worker-protocol over gRPC.5376worker2 # [ 112.304794] systemd[1]: nix-grpc-daemon.service: Consumed 2.022s CPU time over 30.445s wall clock time, 36.7M memory peak, 63.4K incoming IP traffic, 165K outgoing IP traffic.5377worker2 # [ 112.336916] systemd[1]: Starting scheduler-token.service...5378worker1 # [ 112.340331] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_connected addr=lb:500515379worker1 # [ 112.345931] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380lb: must succeed: curl -sf localhost:9901/clusters5381worker2 # [ 112.474966] systemd[1]: scheduler-token.service: Deactivated successfully.5382worker2 # [ 112.484318] systemd[1]: Finished scheduler-token.service.5383worker2 # [ 112.487007] systemd[1]: scheduler-token.service: Consumed 59ms CPU time over 145ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5384worker2 # [ 112.493835] systemd[1]: Starting Nix worker-protocol over gRPC...5385lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.15 seconds)5386worker1 # [ 112.550684] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515387worker1 # [ 112.555284] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5388worker2 # [ 112.729821] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5389worker2 # [ 112.732562] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5390worker2 # [ 112.736875] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15391worker2 # [ 112.744591] nix-grpc-daemon[1566]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5392worker2 # [ 112.746055] nix-grpc-daemon[1566]: E0000 00:00:1789987441.339127 1566 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5393worker2 # [ 112.754605] nix-grpc-daemon[1566]: E0000 00:00:1789987441.345535 1566 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5394worker1 # [ 112.751908] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515395worker2 # [ 112.758588] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5396worker2 # [ 112.761346] systemd[1]: Started Nix worker-protocol over gRPC.5397worker1 # [ 112.757276] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398lb # [ 112.755031] niks3-server[1025]: 2026/09/21 10:44:01 INFO lead: acquired remote=192.168.1.4:372145399worker2 # [ 112.767799] nix-grpc-daemon[1566]: E0000 00:00:1789987441.359922 1607 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5400worker2 # [ 112.796177] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_take_over5401worker2 # [ 112.797340] nix-grpc-daemon[1566]: E0000 00:00:1789987441.378006 1607 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5402worker2 # [ 112.802738] nix-grpc-daemon[1566]: E0000 00:00:1789987441.378190 1607 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5403worker2 # [ 112.805727] nix-grpc-daemon[1566]: E0000 00:00:1789987441.396804 1607 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5404worker2 # [ 112.816287] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515405worker2 # [ 112.817971] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5406worker1 # [ 112.957969] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515407worker1 # [ 112.963131] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5408worker2 # [ 113.018292] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 113.023405] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker1 # [ 113.163927] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 113.169031] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 113.224738] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515413worker2 # [ 113.229871] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker1 # [ 113.370979] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:01Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 113.377280] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:563665416worker2 # [ 113.382566] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05417worker2 # [ 113.430809] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=info event=scheduler_connected addr=lb:500515418worker2 # [ 113.444125] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=info event=oidc_jwks provider=mock keys=15419worker2 # [ 113.448788] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:563665420worker2 # [ 113.454242] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05421lb: must succeed: curl -sf localhost:9901/clusters5422lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5423worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5424worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5425(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.27 seconds)5426subtest: scheduler restart mid-build: build finishes, no double build5427client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5428client # [ 113.825329] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5429client # Running as unit: mid.service; invocation ID: 7d78e334d716407cacd6f5a472a8a0b65430client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5431worker2 # [ 114.058765] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:563685432worker2 # [ 114.073594] nix-daemon[863]: accepted connection from pid 1566, user nix-grpc-daemon (trusted)5433client # [ 114.142997] nix[1204]: don't know how to build these paths:5434client # [ 114.145708] nix[1204]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5435worker2 # [ 114.104134] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05436worker2 # [ 114.110504] nix-grpc-daemon[1566]: ts=2026-09-21T10:44:02Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15437worker1 # [ 114.158973] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514545438worker1 # [ 114.183436] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)5439worker1 # [ 114.252976] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)5440worker1 # [ 114.388241] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 118 ms (attempt 3/5)5441worker1 # [ 114.506557] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 468 ms (attempt 4/5)5442worker1 # [ 114.984666] nix-grpc-daemon[1113]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5443worker1 # [ 114.991085] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5444worker1 # [ 115.007484] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514545445client # [ 115.072545] nix[1204]: copying 1 paths to 'lb:50051'...5446worker1 # [ 115.020629] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514545447client # [ 115.075657] nix[1204]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5448worker1 # [ 115.042355] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5449worker1 # [ 115.072263] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:03Z level=info event=niks3_push_started pid=12315450worker1 # [ 115.282861] nix-grpc-daemon[1231]: time=2026-09-21T10:44:03.868Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5451worker1 # [ 115.456906] nix-daemon[839]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5452worker1 # [ 115.476767] nix-daemon[839]: reaped child process 1247, status = succeeded5453worker1 # [ 115.645983] nix-daemon[839]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5454worker1 # [ 115.663373] nix-daemon[839]: reaped child process 1252, status = succeeded5455lb # [ 115.670394] niks3-server[1025]: 2026/09/21 10:44:04 INFO Received uploads request method=POST path=/api/pending_closures5456worker1 # [ 115.689279] nix-grpc-daemon[1231]: time=2026-09-21T10:44:04.274Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5457worker1 # [ 115.690755] nix-grpc-daemon[1231]: time=2026-09-21T10:44:04.274Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5458lb # [ 115.706978] niks3-server[1025]: 2026/09/21 10:44:04 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5459lb # [ 115.718661] niks3-server[1025]: 2026/09/21 10:44:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5460worker1 # [ 115.732291] nix-grpc-daemon[1231]: time=2026-09-21T10:44:04.318Z level=INFO msg="Uploading 1 narinfos"5461lb # [ 115.729464] niks3-server[1025]: 2026/09/21 10:44:04 INFO Signed narinfos id=35 count=15462lb # [ 115.736186] niks3-server[1025]: 2026/09/21 10:44:04 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5463lb # [ 115.748581] niks3-server[1025]: 2026/09/21 10:44:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5464lb # [ 115.758810] niks3-server[1025]: 2026/09/21 10:44:04 INFO Completed upload id=355465worker1 # [ 115.762954] nix-grpc-daemon[1231]: time=2026-09-21T10:44:04.349Z level=INFO msg="Upload complete. (476ms)"5466worker1 # [ 115.774191] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51454 duration_s=0 paths=1 nar_bytes_in=4245467worker1 # [ 115.783559] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514545468lb # [ 115.779563] niks3-server[1025]: 2026/09/21 10:44:04 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5469worker1 # [ 115.792956] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5470worker1 # [ 115.795965] nix-daemon[839]: reaped child process 1229, status = succeeded5471client # [ 115.858920] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5472worker1 # [ 115.808658] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5473lb: must succeed: curl -sf localhost:9901/clusters5474lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5475lb: must succeed: curl -sf localhost:9901/clusters5476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5477worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5478worker2 # [ 117.833233] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1566 (nix-grpc-daemon) on client request.5479lb # [ 117.842055] niks3-server[1025]: 2026/09/21 10:44:06 INFO lead: released remote=192.168.1.4:372145480worker2 # [ 117.850428] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5481worker1 # [ 117.863058] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5482client # [ 117.916778] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5483worker2 # [ 117.886373] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5484worker2 # [ 117.896900] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5485worker2 # [ 117.904952] nix-daemon[863]: reaped child process 1625, status = succeeded5486worker2 # [ 117.906547] systemd[1]: nix-grpc-daemon.service: Consumed 259ms CPU time over 5.388s wall clock time, 14.4M memory peak, 9.9K incoming IP traffic, 13.8K outgoing IP traffic.5487worker2 # [ 117.912402] systemd[1]: Starting scheduler-token.service...5488worker2 # [ 117.996524] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5489worker2 # [ 118.003433] systemd[1]: scheduler-token.service: Deactivated successfully.5490worker2 # [ 118.006316] systemd[1]: Finished scheduler-token.service.5491worker2 # [ 118.007654] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5492worker2 # [ 118.011125] systemd[1]: Starting Nix worker-protocol over gRPC...5493worker1 # [ 118.063076] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:06Z level=info event=scheduler_connected addr=lb:500515494worker2 # [ 118.185544] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5495worker2 # [ 118.187284] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5496worker2 # [ 118.191338] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15497worker2 # [ 118.199586] nix-grpc-daemon[1656]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5498worker2 # [ 118.211086] nix-grpc-daemon[1656]: E0000 00:00:1789987446.792544 1656 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5499lb # [ 118.203119] niks3-server[1025]: 2026/09/21 10:44:06 INFO lead: acquired remote=192.168.1.4:372265500worker2 # [ 118.215075] nix-grpc-daemon[1656]: E0000 00:00:1789987446.794489 1656 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5501worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.45 seconds)5502worker2 # [ 118.224255] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5503client: must succeed: sleep 5.05504worker2 # [ 118.225827] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=scheduler_take_over5505worker2 # [ 118.226880] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:563725506worker2 # [ 118.256304] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=15507worker2 # [ 118.258445] nix-grpc-daemon[1656]: E0000 00:00:1789987446.816048 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5508worker2 # [ 118.261108] systemd[1]: Started Nix worker-protocol over gRPC.5509worker2 # [ 118.261912] nix-grpc-daemon[1656]: E0000 00:00:1789987446.835751 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5510worker2 # [ 118.266833] nix-grpc-daemon[1656]: E0000 00:00:1789987446.835892 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5511worker2 # [ 118.271717] nix-grpc-daemon[1656]: E0000 00:00:1789987446.850604 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5512worker2 # [ 118.274451] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=scheduler_connected addr=lb:500515513worker2 # [ 118.278607] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=oidc_jwks provider=mock keys=15514worker2 # [ 118.279882] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:563725515worker2 # [ 118.281823] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05516lb # [ 121.943507] envoy[934]: conn peer=[2001:db8:1::4]:57588 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3341 tx=334 ms=50575517client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5518client: waiting for success: ! systemctl is-active mid5519worker1 # [ 123.635786] nix-daemon[839]: reaped child process 1256, status = succeeded5520worker1 # [ 123.801616] nix-daemon[839]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)5521worker1 # [ 123.819233] nix-daemon[839]: reaped child process 1296, status = succeeded5522lb # [ 123.825334] niks3-server[1025]: 2026/09/21 10:44:12 INFO Received uploads request method=POST path=/api/pending_closures5523worker1 # [ 123.844812] nix-grpc-daemon[1231]: time=2026-09-21T10:44:12.428Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5524worker1 # [ 123.846523] nix-grpc-daemon[1231]: time=2026-09-21T10:44:12.428Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5525lb # [ 123.863022] niks3-server[1025]: 2026/09/21 10:44:12 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5526lb # [ 123.875769] niks3-server[1025]: 2026/09/21 10:44:12 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5527lb # [ 123.884080] niks3-server[1025]: 2026/09/21 10:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5528lb # [ 123.890993] niks3-server[1025]: 2026/09/21 10:44:12 INFO Signed narinfos id=36 count=15529worker1 # [ 123.899309] nix-grpc-daemon[1231]: time=2026-09-21T10:44:12.484Z level=INFO msg="Uploading 1 narinfos"5530lb # [ 123.908177] niks3-server[1025]: 2026/09/21 10:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5531worker1 # [ 123.924144] nix-grpc-daemon[1231]: time=2026-09-21T10:44:12.510Z level=INFO msg="Upload complete. (300ms)"5532lb # [ 123.920202] niks3-server[1025]: 2026/09/21 10:44:12 INFO Completed upload id=365533worker1 # [ 123.937063] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51454 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15534lb # [ 123.943649] niks3-server[1025]: 2026/09/21 10:44:12 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5535worker1 # [ 123.952390] nix-grpc-daemon[1231]: time=2026-09-21T10:44:12.538Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5536worker1 # [ 123.958847] nix-daemon[839]: reaped child process 1254, status = succeeded5537client # [ 124.037908] systemd[1]: mid.service: Deactivated successfully.5538client # [ 124.042169] systemd[1]: mid.service: Consumed 205ms CPU time over 10.216s wall clock time, 43.2M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.5539client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5540client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5541client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5542(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.08 seconds)5543subtest: interrupting the client stops the build on the worker5544client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5545client # [ 124.871454] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5546client # Running as unit: intr.service; invocation ID: b0d65aee3c2d47008a28feeb969a1f915547client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)5548worker2 # [ 125.068599] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594265549worker2 # [ 125.083704] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5550client # [ 125.151483] nix[1240]: don't know how to build these paths:5551client # [ 125.153396] nix[1240]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5552worker2 # [ 125.113963] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:13Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05553worker2 # [ 125.121211] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:13Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15554worker2 # [ 125.162021] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265555worker2 # [ 125.181657] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)5556worker2 # [ 125.279146] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 2/5)5557worker2 # [ 125.412944] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 305 ms (attempt 3/5)5558worker2 # [ 125.725679] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 535 ms (attempt 4/5)5559worker2 # [ 126.262383] nix-grpc-daemon[1656]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5560worker2 # [ 126.269248] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5561worker2 # [ 126.288687] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265562client # [ 126.357224] nix[1240]: copying 1 paths to 'lb:50051'...5563worker2 # [ 126.307051] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594265564client # [ 126.357471] nix[1240]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5565worker2 # [ 126.321681] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5566worker2 # [ 126.349995] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:14Z level=info event=niks3_push_started pid=17345567worker2 # [ 126.569678] nix-grpc-daemon[1734]: time=2026-09-21T10:44:15.164Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5568worker2 # [ 126.743980] nix-daemon[863]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)5569worker2 # [ 126.760638] nix-daemon[863]: reaped child process 1750, status = succeeded5570worker2 # [ 126.934323] nix-daemon[863]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)5571worker2 # [ 126.950396] nix-daemon[863]: reaped child process 1755, status = succeeded5572lb # [ 126.953311] niks3-server[1025]: 2026/09/21 10:44:15 INFO Received uploads request method=POST path=/api/pending_closures5573worker2 # [ 126.977939] nix-grpc-daemon[1734]: time=2026-09-21T10:44:15.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5574worker2 # [ 126.979493] nix-grpc-daemon[1734]: time=2026-09-21T10:44:15.571Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5575lb # [ 126.999994] niks3-server[1025]: 2026/09/21 10:44:15 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5576worker2 # [ 127.022622] nix-grpc-daemon[1734]: time=2026-09-21T10:44:15.616Z level=INFO msg="Uploading 1 narinfos"5577lb # [ 127.012805] niks3-server[1025]: 2026/09/21 10:44:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5578lb # [ 127.025342] niks3-server[1025]: 2026/09/21 10:44:15 INFO Signed narinfos id=37 count=15579lb # [ 127.038386] niks3-server[1025]: 2026/09/21 10:44:15 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5580worker2 # [ 127.059416] nix-grpc-daemon[1734]: time=2026-09-21T10:44:15.654Z level=INFO msg="Upload complete. (485ms)"5581lb # [ 127.051500] niks3-server[1025]: 2026/09/21 10:44:15 INFO Completed upload id=375582lb # [ 127.055592] niks3-server[1025]: 2026/09/21 10:44:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5583worker2 # [ 127.068278] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=4245584lb # [ 127.064235] niks3-server[1025]: 2026/09/21 10:44:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5585worker2 # [ 127.084342] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265586worker2 # [ 127.090833] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5587worker2 # [ 127.095766] nix-daemon[863]: reaped child process 1732, status = succeeded5588client # [ 127.156986] nix[1240]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5589worker2 # [ 127.109277] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5590client: must succeed: systemctl kill -s INT intr5591worker2 # [ 128.533343] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:17Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5592client # [ 128.583122] systemd[1]: intr.service: Sent signal SIGINT to main process 1240 (nix) on client request.5593client # [ 128.590887] nix[1240]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5594client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5595client # [ 128.598137] nix[1240]: [mutex.cc : 2506] RAW: thread should hold at least a read lock on Mutex 0xed34f33fc7b85596client # [ 128.616708] systemd-coredump[1254]: Process 1240 (nix) of user 0 terminated abnormally with signal 6/ABRT, processing...5597client # [ 128.639307] systemd[1]: Created slice Slice /system/systemd-coredump.5598client # [ 128.643450] systemd[1]: Started Process Core Dump (PID 1254/UID 0).5599worker2 # [ 128.733988] nix-daemon[863]: reaped child process 1757, status = succeeded5600worker2 # [ 128.737630] nix-daemon[1759]: unexpected Nix daemon error: error: interrupted by the user5601worker2 # [ 128.745258] nix-daemon[863]: reaped child process 1759, status = failed with exit code 15602(finished: subtest: interrupting the client stops the build on the worker, in 5.08 seconds)5603subtest: a deploy mid-build drains: the build finishes, the new generation takes over5604client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5605client # [ 129.972977] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5606client # Running as unit: sw.service; invocation ID: fa34af4836cd4592b1212b085fb6a9955607client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.08 seconds)5608worker1 # [ 130.350541] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514545609client # [ 130.411014] nix[1261]: don't know how to build these paths:5610client # [ 130.414132] nix[1261]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5611worker2 # [ 130.373382] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:18Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05612worker2 # [ 130.379677] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:18Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15613worker1 # [ 130.427785] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514545614worker1 # [ 130.444080] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514545615client # [ 130.512284] nix[1261]: copying 1 paths to 'lb:50051'...5616worker1 # [ 130.458201] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514545617client # [ 130.516519] nix[1261]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5618worker1 # [ 130.476926] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5619worker1 # [ 130.682162] nix-daemon[839]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5620worker1 # [ 130.701114] nix-daemon[839]: reaped child process 1331, status = succeeded5621worker1 # [ 130.880586] nix-daemon[839]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)5622worker1 # [ 130.899626] nix-daemon[839]: reaped child process 1336, status = succeeded5623lb # [ 130.905808] niks3-server[1025]: 2026/09/21 10:44:19 INFO Received uploads request method=POST path=/api/pending_closures5624worker1 # [ 130.927326] nix-grpc-daemon[1231]: time=2026-09-21T10:44:19.513Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5625worker1 # [ 130.929034] nix-grpc-daemon[1231]: time=2026-09-21T10:44:19.513Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5626lb # [ 130.944175] niks3-server[1025]: 2026/09/21 10:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5627worker1 # [ 130.958112] nix-grpc-daemon[1231]: time=2026-09-21T10:44:19.544Z level=INFO msg="Uploading 1 narinfos"5628lb # [ 130.956279] niks3-server[1025]: 2026/09/21 10:44:19 INFO Signed narinfos id=38 count=15629client # [ 131.019688] systemd-coredump[1255]: Cannot store coredump of 1240 (nix): No space left on device5630lb # [ 130.965458] niks3-server[1025]: 2026/09/21 10:44:19 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5631lb # [ 130.972598] niks3-server[1025]: 2026/09/21 10:44:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5632lb # [ 130.984781] niks3-server[1025]: 2026/09/21 10:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5633lb # [ 130.992125] niks3-server[1025]: 2026/09/21 10:44:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5634lb # [ 130.997453] niks3-server[1025]: 2026/09/21 10:44:19 INFO Completed upload id=385635worker1 # [ 131.005461] nix-grpc-daemon[1231]: time=2026-09-21T10:44:19.591Z level=INFO msg="Upload complete. (511ms)"5636worker1 # [ 131.014400] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51454 duration_s=0 paths=1 nar_bytes_in=4165637worker1 # [ 131.031288] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514545638worker1 # [ 131.038528] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5639worker1 # [ 131.043705] nix-daemon[839]: reaped child process 1326, status = succeeded5640client # [ 131.108462] nix[1261]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5641worker1 # [ 131.059742] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5642client # [ 131.150814] systemd-coredump[1255]: Process 1240 (nix) of user 0 terminated abnormally without generating a coredump.5643client # [ 131.170077] systemd[1]: systemd-coredump@0-1-1254_1255-0.service: Deactivated successfully.5644client # [ 131.171524] systemd[1]: systemd-coredump@0-1-1254_1255-0.service: Consumed 1.905s CPU time over 2.519s wall clock time, 968.7M memory peak, 926.2M written to disk.5645client # [ 131.185965] systemd[1]: intr.service: Main process exited, code=dumped, status=6/ABRT5646client # [ 131.188763] systemd[1]: intr.service: Failed with result 'core-dump'.5647client # [ 131.189971] systemd[1]: intr.service: Consumed 361ms CPU time over 6.316s wall clock time, 41.4M memory peak, 32K written to disk, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5648lb # [ 131.943620] envoy[934]: conn peer=[2001:db8:1::1]:41810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1518 ms=99065649lb # [ 131.948799] envoy[934]: conn peer=[2001:db8:1::1]:48126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=853 ms=60565650worker1: must succeed: systemctl reload nix-grpc-daemon.service5651worker1 # [ 132.591116] systemd[1]: Reloading Nix worker-protocol over gRPC...5652worker1 # [ 132.641313] systemd[1]: Reloaded Nix worker-protocol over gRPC.5653worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)5654worker1: must succeed: pgrep -f 'read -t [9]0 x'5655worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5656client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25657worker1 # [ 132.970705] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514545658client # don't know how to build these paths:5659client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5660worker2 # [ 132.985033] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265661worker2 # [ 133.012129] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05662worker2 # [ 133.018753] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15663worker1 # [ 133.025832] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:21Z level=info event=draining5664worker2 # [ 133.031178] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265665worker2 # [ 133.046137] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265666client # copying 1 paths to 'lb:50051'...5667client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5668worker2 # [ 133.062103] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594265669worker2 # [ 133.077024] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5670worker2 # [ 133.275450] nix-daemon[863]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)5671worker2 # [ 133.294075] nix-daemon[863]: reaped child process 1817, status = succeeded5672worker2 # [ 133.465683] nix-daemon[863]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)5673worker2 # [ 133.483042] nix-daemon[863]: reaped child process 1822, status = succeeded5674lb # [ 133.485907] niks3-server[1025]: 2026/09/21 10:44:22 INFO Received uploads request method=POST path=/api/pending_closures5675worker2 # [ 133.506619] nix-grpc-daemon[1734]: time=2026-09-21T10:44:22.101Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5676worker2 # [ 133.508171] nix-grpc-daemon[1734]: time=2026-09-21T10:44:22.101Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5677lb # [ 133.514688] niks3-server[1025]: 2026/09/21 10:44:22 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5678lb # [ 133.530098] niks3-server[1025]: 2026/09/21 10:44:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5679worker2 # [ 133.544362] nix-grpc-daemon[1734]: time=2026-09-21T10:44:22.139Z level=INFO msg="Uploading 1 narinfos"5680lb # [ 133.537057] niks3-server[1025]: 2026/09/21 10:44:22 INFO Signed narinfos id=39 count=15681lb # [ 133.542230] niks3-server[1025]: 2026/09/21 10:44:22 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5682lb # [ 133.553435] niks3-server[1025]: 2026/09/21 10:44:22 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5683worker2 # [ 133.562799] nix-grpc-daemon[1734]: time=2026-09-21T10:44:22.157Z level=INFO msg="Upload complete. (470ms)"5684lb # [ 133.557489] niks3-server[1025]: 2026/09/21 10:44:22 INFO Completed upload id=395685worker2 # [ 133.568411] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=5285686lb # [ 133.560345] niks3-server[1025]: 2026/09/21 10:44:22 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5687worker2 # [ 133.579997] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265688worker2 # [ 133.583183] nix-daemon[863]: reaped child process 1812, status = succeeded5689worker2 # [ 133.589945] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5690client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5691worker2 # [ 133.600175] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5692client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5693client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5694client # farm-a> LOG-farm-a-during-drain5695worker2 # [ 134.443155] nix-daemon[863]: reaped child process 1826, status = succeeded5696worker2 # [ 134.631618] nix-daemon[863]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)5697worker2 # [ 134.650181] nix-daemon[863]: reaped child process 1845, status = succeeded5698lb # [ 134.652904] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received uploads request method=POST path=/api/pending_closures5699worker2 # [ 134.675728] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.267Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5700worker2 # [ 134.678030] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.267Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5701lb # [ 134.686198] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5702lb # [ 134.702668] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5703lb # [ 134.715023] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5704worker2 # [ 134.734335] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.328Z level=INFO msg="Uploading 1 narinfos"5705lb # [ 134.732298] niks3-server[1025]: 2026/09/21 10:44:23 INFO Signed narinfos id=40 count=15706lb # [ 134.734558] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5707lb # [ 134.746347] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5708lb # [ 134.750298] niks3-server[1025]: 2026/09/21 10:44:23 INFO Completed upload id=405709worker2 # [ 134.759788] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.354Z level=INFO msg="Upload complete. (348ms)"5710lb # [ 134.752513] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5711worker2 # [ 134.777133] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15712worker2 # [ 134.792410] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05713worker2 # [ 134.796003] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15714worker2 # [ 134.801713] nix-daemon[863]: reaped child process 1824, status = succeeded5715worker2 # [ 134.821916] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265716worker2 # [ 134.837200] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265717worker2 # [ 134.851622] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.446Z level=INFO msg="All 1 paths already cached"5718client # copying 1 paths to 'lb:50051'...5719client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5720worker2 # [ 134.861420] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594265721worker2 # [ 134.875164] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5722worker2 # [ 135.083589] nix-daemon[863]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)5723worker2 # [ 135.099953] nix-daemon[863]: reaped child process 1854, status = succeeded5724worker2 # [ 135.272946] nix-daemon[863]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)5725worker2 # [ 135.288979] nix-daemon[863]: reaped child process 1859, status = succeeded5726lb # [ 135.292162] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received uploads request method=POST path=/api/pending_closures5727worker2 # [ 135.310873] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.905Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5728worker2 # [ 135.312490] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.905Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5729lb # [ 135.318650] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5730lb # [ 135.333912] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5731worker2 # [ 135.350020] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.944Z level=INFO msg="Uploading 1 narinfos"5732lb # [ 135.346879] niks3-server[1025]: 2026/09/21 10:44:23 INFO Signed narinfos id=41 count=15733lb # [ 135.355563] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5734worker2 # [ 135.370760] nix-grpc-daemon[1734]: time=2026-09-21T10:44:23.965Z level=INFO msg="Upload complete. (477ms)"5735lb # [ 135.362032] niks3-server[1025]: 2026/09/21 10:44:23 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5736lb # [ 135.367152] niks3-server[1025]: 2026/09/21 10:44:23 INFO Completed upload id=415737worker2 # [ 135.380213] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=6405738lb # [ 135.370655] niks3-server[1025]: 2026/09/21 10:44:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5739worker2 # [ 135.384349] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265740worker2 # [ 135.395498] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5741worker2 # [ 135.397483] nix-daemon[863]: reaped child process 1849, status = succeeded5742client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5743worker2 # [ 135.409800] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5744client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5745client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5746client # farm-b> LOG-farm-b-during-drain5747worker2 # [ 136.833481] nix-daemon[863]: reaped child process 1863, status = succeeded5748worker2 # [ 137.001858] nix-daemon[863]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)5749worker2 # [ 137.018429] nix-daemon[863]: reaped child process 1882, status = succeeded5750lb # [ 137.021350] niks3-server[1025]: 2026/09/21 10:44:25 INFO Received uploads request method=POST path=/api/pending_closures5751worker2 # [ 137.041209] nix-grpc-daemon[1734]: time=2026-09-21T10:44:25.635Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5752worker2 # [ 137.042659] nix-grpc-daemon[1734]: time=2026-09-21T10:44:25.635Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5753lb # [ 137.063767] niks3-server[1025]: 2026/09/21 10:44:25 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5754worker2 # [ 137.085309] nix-grpc-daemon[1734]: time=2026-09-21T10:44:25.680Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 137.078071] niks3-server[1025]: 2026/09/21 10:44:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5756lb # [ 137.087733] niks3-server[1025]: 2026/09/21 10:44:25 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5757lb # [ 137.093643] niks3-server[1025]: 2026/09/21 10:44:25 INFO Signed narinfos id=42 count=15758lb # [ 137.101456] niks3-server[1025]: 2026/09/21 10:44:25 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5759worker2 # [ 137.115174] nix-grpc-daemon[1734]: time=2026-09-21T10:44:25.710Z level=INFO msg="Upload complete. (296ms)"5760lb # [ 137.106153] niks3-server[1025]: 2026/09/21 10:44:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5761lb # [ 137.111662] niks3-server[1025]: 2026/09/21 10:44:25 INFO Completed upload id=425762lb # [ 137.114798] niks3-server[1025]: 2026/09/21 10:44:25 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5763worker2 # [ 137.129297] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15764worker2 # [ 137.144613] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05765worker2 # [ 137.149332] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15766worker2 # [ 137.153458] nix-daemon[863]: reaped child process 1861, status = succeeded5767worker2 # [ 137.185841] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265768worker2 # [ 137.204506] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265769worker2 # [ 137.219760] nix-grpc-daemon[1734]: time=2026-09-21T10:44:25.814Z level=INFO msg="All 2 paths already cached"5770client # copying 1 paths to 'lb:50051'...5771client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5772worker2 # [ 137.229917] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594265773worker2 # [ 137.243188] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5774worker2 # [ 137.439876] nix-daemon[863]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)5775worker2 # [ 137.458840] nix-daemon[863]: reaped child process 1891, status = succeeded5776worker2 # [ 137.627017] nix-daemon[863]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)5777worker2 # [ 137.642850] nix-daemon[863]: reaped child process 1896, status = succeeded5778lb # [ 137.644886] niks3-server[1025]: 2026/09/21 10:44:26 INFO Received uploads request method=POST path=/api/pending_closures5779worker2 # [ 137.665280] nix-grpc-daemon[1734]: time=2026-09-21T10:44:26.260Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5780worker2 # [ 137.666741] nix-grpc-daemon[1734]: time=2026-09-21T10:44:26.260Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5781lb # [ 137.675533] niks3-server[1025]: 2026/09/21 10:44:26 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5782lb # [ 137.688494] niks3-server[1025]: 2026/09/21 10:44:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5783worker2 # [ 137.705752] nix-grpc-daemon[1734]: time=2026-09-21T10:44:26.300Z level=INFO msg="Uploading 1 narinfos"5784lb # [ 137.697876] niks3-server[1025]: 2026/09/21 10:44:26 INFO Signed narinfos id=43 count=15785lb # [ 137.704236] niks3-server[1025]: 2026/09/21 10:44:26 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5786lb # [ 137.714114] niks3-server[1025]: 2026/09/21 10:44:26 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5787lb # [ 137.722138] niks3-server[1025]: 2026/09/21 10:44:26 INFO Completed upload id=435788worker2 # [ 137.731730] nix-grpc-daemon[1734]: time=2026-09-21T10:44:26.326Z level=INFO msg="Upload complete. (472ms)"5789lb # [ 137.725001] niks3-server[1025]: 2026/09/21 10:44:26 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5790worker2 # [ 137.740305] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=7685791worker2 # [ 137.753047] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594265792worker2 # [ 137.760461] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5793worker2 # [ 137.763660] nix-daemon[863]: reaped child process 1886, status = succeeded5794client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5795worker2 # [ 137.773226] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5796client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5797client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5798client # farm-top> LOG-farm-top-during-drain5799worker2 # [ 139.012800] nix-daemon[863]: reaped child process 1900, status = succeeded5800worker2 # [ 139.187661] nix-daemon[863]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)5801worker2 # [ 139.203102] nix-daemon[863]: reaped child process 1919, status = succeeded5802lb # [ 139.210013] niks3-server[1025]: 2026/09/21 10:44:27 INFO Received uploads request method=POST path=/api/pending_closures5803worker2 # [ 139.231648] nix-grpc-daemon[1734]: time=2026-09-21T10:44:27.824Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5804worker2 # [ 139.233755] nix-grpc-daemon[1734]: time=2026-09-21T10:44:27.824Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5805lb # [ 139.247402] niks3-server[1025]: 2026/09/21 10:44:27 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5806lb # [ 139.259613] niks3-server[1025]: 2026/09/21 10:44:27 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5807lb # [ 139.267536] niks3-server[1025]: 2026/09/21 10:44:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5808worker2 # [ 139.281702] nix-grpc-daemon[1734]: time=2026-09-21T10:44:27.876Z level=INFO msg="Uploading 1 narinfos"5809lb # [ 139.276540] niks3-server[1025]: 2026/09/21 10:44:27 INFO Signed narinfos id=44 count=15810lb # [ 139.282156] niks3-server[1025]: 2026/09/21 10:44:27 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5811lb # [ 139.288302] niks3-server[1025]: 2026/09/21 10:44:27 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5812lb # [ 139.296959] niks3-server[1025]: 2026/09/21 10:44:27 INFO Completed upload id=445813worker2 # [ 139.306624] nix-grpc-daemon[1734]: time=2026-09-21T10:44:27.901Z level=INFO msg="Upload complete. (322ms)"5814lb # [ 139.300353] niks3-server[1025]: 2026/09/21 10:44:27 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5815worker2 # [ 139.321170] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=15816worker2 # [ 139.337388] nix-daemon[863]: reaped child process 1898, status = succeeded5817client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.67 seconds)5818client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5819client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5820worker1: must succeed: pkill -f 'read -t [9]0 x'5821worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5822client: waiting for success: ! systemctl is-active sw5823worker1 # [ 139.687271] nix-daemon[839]: reaped child process 1340, status = succeeded5824worker1 # [ 139.844307] nix-daemon[839]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5825worker1 # [ 139.862981] nix-daemon[839]: reaped child process 1385, status = succeeded5826lb # [ 139.867071] niks3-server[1025]: 2026/09/21 10:44:28 INFO Received uploads request method=POST path=/api/pending_closures5827worker1 # [ 139.885649] nix-grpc-daemon[1231]: time=2026-09-21T10:44:28.471Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5828worker1 # [ 139.887135] nix-grpc-daemon[1231]: time=2026-09-21T10:44:28.471Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5829lb # [ 139.897799] niks3-server[1025]: 2026/09/21 10:44:28 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5830lb # [ 139.911249] niks3-server[1025]: 2026/09/21 10:44:28 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5831worker1 # [ 139.930592] nix-grpc-daemon[1231]: time=2026-09-21T10:44:28.516Z level=INFO msg="Uploading 1 narinfos"5832lb # [ 139.923553] niks3-server[1025]: 2026/09/21 10:44:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5833lb # [ 139.932360] niks3-server[1025]: 2026/09/21 10:44:28 INFO Signed narinfos id=45 count=15834lb # [ 139.934396] niks3-server[1025]: 2026/09/21 10:44:28 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5835lb # [ 139.944463] niks3-server[1025]: 2026/09/21 10:44:28 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5836lb # [ 139.948210] niks3-server[1025]: 2026/09/21 10:44:28 INFO Completed upload id=455837worker1 # [ 139.956387] nix-grpc-daemon[1231]: time=2026-09-21T10:44:28.542Z level=INFO msg="Upload complete. (281ms)"5838lb # [ 139.950361] niks3-server[1025]: 2026/09/21 10:44:28 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5839worker1 # [ 139.968794] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51454 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=15840worker1 # [ 139.980499] nix-daemon[839]: reaped child process 1338, status = succeeded5841client # [ 140.059554] systemd[1]: sw.service: Deactivated successfully.5842client # [ 140.063814] systemd[1]: sw.service: Consumed 196ms CPU time over 10.089s wall clock time, 39.3M memory peak, 8K written to disk, 7K incoming IP traffic, 8.2K outgoing IP traffic.5843worker1 # [ 140.026965] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:28Z level=info event=signal_exit5844client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5845client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5846client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5847worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5848worker2 # [ 141.262741] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:563725849worker1 # [ 141.265024] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:29Z level=info event=scheduler_restarting5850worker2 # [ 141.263128] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:29Z level=info event=worker_gone addr=192.168.1.3:500515851worker1 # [ 141.288610] nix-grpc-daemon[1113]: ts=2026-09-21T10:44:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5852worker1 # [ 141.313808] nix-daemon[839]: reaped child process 1172, status = succeeded5853lb # [ 141.944557] envoy[934]: conn peer=[2001:db8:1::1]:48596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4089 ms=63625854worker1 # [ 142.101719] nix-grpc-daemon[1113]: E0000 00:00:1789987470.687991 1158 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5855worker1 # [ 142.139058] nix-grpc-daemon[1113]: E0000 00:00:1789987470.725267 1158 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5856worker1 # [ 142.511415] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5857worker1 # [ 142.518982] systemd[1]: nix-grpc-daemon.service: Consumed 1.125s CPU time over 42.720s wall clock time, 34.1M memory peak, 27.7K incoming IP traffic, 40.7K outgoing IP traffic.5858worker1 # [ 142.538537] systemd[1]: Starting Nix worker-protocol over gRPC...5859lb # [ 141.949822] envoy[934]: conn peer=[2001:db8:1::1]:48136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1429 ms=96435860lb # [ 141.949981] envoy[934]: conn peer=[2001:db8:1::3]:46238 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3107 tx=929 ms=411915861worker1 # [ 142.738976] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5862worker1 # [ 142.741740] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5863worker1 # [ 142.746170] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15864worker1 # [ 142.755327] nix-grpc-daemon[1399]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5865worker1 # [ 142.764410] nix-grpc-daemon[1399]: E0000 00:00:1789987471.339400 1399 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5866worker1 # [ 142.768591] nix-grpc-daemon[1399]: E0000 00:00:1789987471.341712 1399 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5867worker1 # [ 142.771977] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5868worker1 # [ 142.775377] systemd[1]: Started Nix worker-protocol over gRPC.5869worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.95 seconds)5870lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5871lb: must succeed: curl -sf localhost:9901/clusters5872worker1 # [ 142.780124] nix-grpc-daemon[1399]: E0000 00:00:1789987471.364605 1441 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5873worker1 # [ 142.808047] nix-grpc-daemon[1399]: E0000 00:00:1789987471.382622 1441 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5874worker1 # [ 142.810396] nix-grpc-daemon[1399]: E0000 00:00:1789987471.382847 1441 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5875worker1 # [ 142.816776] nix-grpc-daemon[1399]: E0000 00:00:1789987471.402258 1441 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5876worker1 # [ 142.831902] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:31Z level=info event=scheduler_connected addr=lb:500515877worker2 # [ 142.836251] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:563725878worker2 # [ 142.836636] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5882lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5883lb: must succeed: curl -sf localhost:9901/clusters5884lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5885worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5886worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5887(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.23 seconds)5888subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5889client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15890worker2 # [ 144.353925] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594265891worker2 # [ 144.361657] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594265892worker2 # [ 144.403536] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05893worker2 # [ 144.409566] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15894worker2 # [ 144.415503] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05895worker2 # [ 144.420727] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:32Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05896worker1 # [ 144.454983] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449625897worker1 # [ 144.470022] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5898worker1 # [ 144.495064] nix-grpc-daemon[1399]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5899worker1 # [ 144.576558] nix-grpc-daemon[1399]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 2/5)5900worker1 # [ 144.740892] nix-grpc-daemon[1399]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 3/5)5901worker1 # [ 144.751248] nix-grpc-daemon[1399]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 4/5)5902worker1 # [ 144.943843] nix-grpc-daemon[1399]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5903worker1 # [ 144.949322] nix-grpc-daemon[1399]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5904worker1 # [ 144.967331] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449625905worker1 # [ 144.981861] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449625906worker1 # [ 144.996104] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5907worker1 # [ 145.024501] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:33Z level=info event=niks3_push_started pid=14595908worker1 # [ 145.240629] nix-grpc-daemon[1459]: time=2026-09-21T10:44:33.826Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5909worker1 # [ 145.412166] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5910worker1 # [ 145.430077] nix-daemon[839]: reaped child process 1475, status = succeeded5911worker1 # [ 145.602460] nix-daemon[839]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)5912worker1 # [ 145.620210] nix-daemon[839]: reaped child process 1480, status = succeeded5913lb # [ 145.627305] niks3-server[1025]: 2026/09/21 10:44:34 INFO Received uploads request method=POST path=/api/pending_closures5914worker1 # [ 145.647291] nix-grpc-daemon[1459]: time=2026-09-21T10:44:34.232Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5915worker1 # [ 145.649422] nix-grpc-daemon[1459]: time=2026-09-21T10:44:34.232Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5916lb # [ 145.666944] niks3-server[1025]: 2026/09/21 10:44:34 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5917lb # [ 145.679351] niks3-server[1025]: 2026/09/21 10:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5918worker1 # [ 145.694567] nix-grpc-daemon[1459]: time=2026-09-21T10:44:34.280Z level=INFO msg="Uploading 1 narinfos"5919lb # [ 145.693035] niks3-server[1025]: 2026/09/21 10:44:34 INFO Signed narinfos id=46 count=15920lb # [ 145.702940] niks3-server[1025]: 2026/09/21 10:44:34 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5921lb # [ 145.712896] niks3-server[1025]: 2026/09/21 10:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5922lb # [ 145.721747] niks3-server[1025]: 2026/09/21 10:44:34 INFO Completed upload id=465923worker1 # [ 145.729912] nix-grpc-daemon[1459]: time=2026-09-21T10:44:34.316Z level=INFO msg="Upload complete. (485ms)"5924lb # [ 145.724921] niks3-server[1025]: 2026/09/21 10:44:34 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5925worker1 # [ 145.741110] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=5125926worker1 # [ 145.756591] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449625927worker1 # [ 145.766975] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5928worker1 # [ 145.769578] nix-daemon[839]: reaped child process 1457, status = succeeded5929worker1 # [ 145.784751] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5930worker1 # [ 147.217031] nix-daemon[839]: reaped child process 1484, status = succeeded5931worker1 # [ 147.385141] nix-daemon[839]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)5932worker1 # [ 147.404839] nix-daemon[839]: reaped child process 1503, status = succeeded5933lb # [ 147.409296] niks3-server[1025]: 2026/09/21 10:44:35 INFO Received uploads request method=POST path=/api/pending_closures5934worker1 # [ 147.432386] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.015Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5935worker1 # [ 147.433923] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.015Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5936lb # [ 147.455030] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5937lb # [ 147.466093] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5938lb # [ 147.476087] niks3-server[1025]: 2026/09/21 10:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5939lb # [ 147.481347] niks3-server[1025]: 2026/09/21 10:44:36 INFO Signed narinfos id=47 count=15940worker1 # [ 147.489997] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.075Z level=INFO msg="Uploading 1 narinfos"5941lb # [ 147.485229] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5942lb # [ 147.496770] niks3-server[1025]: 2026/09/21 10:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5943lb # [ 147.502163] niks3-server[1025]: 2026/09/21 10:44:36 INFO Completed upload id=475944worker1 # [ 147.510310] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.096Z level=INFO msg="Upload complete. (313ms)"5945lb # [ 147.504848] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5946worker2 # [ 147.520960] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:36Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15947worker1 # [ 147.520929] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=15948worker2 # [ 147.536162] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:36Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05949worker1 # [ 147.538414] nix-daemon[839]: reaped child process 1482, status = succeeded5950worker1 # [ 147.575280] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449625951worker1 # [ 147.595569] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449625952worker1 # [ 147.613014] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449625953worker1 # [ 147.632757] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5954worker1 # [ 147.842962] nix-daemon[839]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)5955worker1 # [ 147.863771] nix-daemon[839]: reaped child process 1510, status = succeeded5956worker1 # [ 148.041694] nix-daemon[839]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)5957worker1 # [ 148.058687] nix-daemon[839]: reaped child process 1515, status = succeeded5958lb # [ 148.061400] niks3-server[1025]: 2026/09/21 10:44:36 INFO Received uploads request method=POST path=/api/pending_closures5959worker1 # [ 148.079208] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.665Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5960worker1 # [ 148.080895] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.665Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5961lb # [ 148.087126] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5962worker1 # [ 148.108898] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.695Z level=INFO msg="Uploading 1 narinfos"5963lb # [ 148.102192] niks3-server[1025]: 2026/09/21 10:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5964lb # [ 148.113108] niks3-server[1025]: 2026/09/21 10:44:36 INFO Signed narinfos id=48 count=15965lb # [ 148.120328] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5966lb # [ 148.128953] niks3-server[1025]: 2026/09/21 10:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5967worker1 # [ 148.141172] nix-grpc-daemon[1459]: time=2026-09-21T10:44:36.727Z level=INFO msg="Upload complete. (490ms)"5968lb # [ 148.133841] niks3-server[1025]: 2026/09/21 10:44:36 INFO Completed upload id=485969lb # [ 148.137627] niks3-server[1025]: 2026/09/21 10:44:36 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5970worker1 # [ 148.148111] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=5125971worker1 # [ 148.158500] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449625972worker1 # [ 148.166868] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5973worker1 # [ 148.169207] nix-daemon[839]: reaped child process 1505, status = succeeded5974worker1 # [ 148.183396] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5975worker1 # [ 149.629061] nix-daemon[839]: reaped child process 1519, status = succeeded5976worker1 # [ 149.811777] nix-daemon[839]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)5977worker1 # [ 149.829970] nix-daemon[839]: reaped child process 1538, status = succeeded5978lb # [ 149.834946] niks3-server[1025]: 2026/09/21 10:44:38 INFO Received uploads request method=POST path=/api/pending_closures5979worker1 # [ 149.853056] nix-grpc-daemon[1459]: time=2026-09-21T10:44:38.438Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5980worker1 # [ 149.854508] nix-grpc-daemon[1459]: time=2026-09-21T10:44:38.438Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5981lb # [ 149.869529] niks3-server[1025]: 2026/09/21 10:44:38 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5982lb # [ 149.881389] niks3-server[1025]: 2026/09/21 10:44:38 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5983worker1 # [ 149.901735] nix-grpc-daemon[1459]: time=2026-09-21T10:44:38.487Z level=INFO msg="Uploading 1 narinfos"5984lb # [ 149.895411] niks3-server[1025]: 2026/09/21 10:44:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5985lb # [ 149.903072] niks3-server[1025]: 2026/09/21 10:44:38 INFO Signed narinfos id=49 count=15986lb # [ 149.906911] niks3-server[1025]: 2026/09/21 10:44:38 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5987lb # [ 149.913058] niks3-server[1025]: 2026/09/21 10:44:38 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5988lb # [ 149.916961] niks3-server[1025]: 2026/09/21 10:44:38 INFO Completed upload id=495989worker1 # [ 149.925100] nix-grpc-daemon[1459]: time=2026-09-21T10:44:38.511Z level=INFO msg="Upload complete. (330ms)"5990lb # [ 149.919304] niks3-server[1025]: 2026/09/21 10:44:38 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5991worker2 # [ 149.934864] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:38Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15992worker1 # [ 149.937165] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=15993worker2 # [ 149.947529] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:38Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05994worker1 # [ 149.956381] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449625995worker1 # [ 149.966562] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449625996worker1 # [ 149.971914] nix-daemon[839]: reaped child process 1517, status = succeeded5997worker1 # [ 149.979034] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449625998worker1 # [ 149.990678] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5999worker1 # [ 150.187161] nix-daemon[839]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6000worker1 # [ 150.203241] nix-daemon[839]: reaped child process 1545, status = succeeded6001worker1 # [ 150.376827] nix-daemon[839]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)6002worker1 # [ 150.392761] nix-daemon[839]: reaped child process 1550, status = succeeded6003lb # [ 150.398044] niks3-server[1025]: 2026/09/21 10:44:38 INFO Received uploads request method=POST path=/api/pending_closures6004worker1 # [ 150.415205] nix-grpc-daemon[1459]: time=2026-09-21T10:44:39.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6005worker1 # [ 150.416912] nix-grpc-daemon[1459]: time=2026-09-21T10:44:39.001Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6006lb # [ 150.428216] niks3-server[1025]: 2026/09/21 10:44:39 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6007lb # [ 150.438559] niks3-server[1025]: 2026/09/21 10:44:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6008worker1 # [ 150.452772] nix-grpc-daemon[1459]: time=2026-09-21T10:44:39.039Z level=INFO msg="Uploading 1 narinfos"6009lb # [ 150.449065] niks3-server[1025]: 2026/09/21 10:44:39 INFO Signed narinfos id=50 count=16010lb # [ 150.451850] niks3-server[1025]: 2026/09/21 10:44:39 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6011lb # [ 150.461636] niks3-server[1025]: 2026/09/21 10:44:39 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6012lb # [ 150.468328] niks3-server[1025]: 2026/09/21 10:44:39 INFO Completed upload id=506013worker1 # [ 150.476355] nix-grpc-daemon[1459]: time=2026-09-21T10:44:39.062Z level=INFO msg="Upload complete. (472ms)"6014lb # [ 150.471172] niks3-server[1025]: 2026/09/21 10:44:39 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6015worker1 # [ 150.484141] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=5126016worker1 # [ 150.495521] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626017worker1 # [ 150.501992] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6018worker1 # [ 150.506524] nix-daemon[839]: reaped child process 1540, status = succeeded6019worker1 # [ 150.517122] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6020worker1 # [ 151.344389] nix-daemon[839]: reaped child process 1554, status = succeeded6021worker1 # [ 151.501859] nix-daemon[839]: accepted connection from pid 1570, user nix-grpc-daemon (trusted)6022worker1 # [ 151.519462] nix-daemon[839]: reaped child process 1573, status = succeeded6023lb # [ 151.524959] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received uploads request method=POST path=/api/pending_closures6024worker1 # [ 151.543587] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.128Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6025worker1 # [ 151.545286] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.128Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6026lb # [ 151.557936] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6027lb # [ 151.568294] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6028worker1 # [ 151.587324] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.173Z level=INFO msg="Uploading 1 narinfos"6029lb # [ 151.583521] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6030lb # [ 151.594495] niks3-server[1025]: 2026/09/21 10:44:40 INFO Signed narinfos id=51 count=16031worker1 # [ 151.606768] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.193Z level=INFO msg="Upload complete. (277ms)"6032lb # [ 151.599284] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6033lb # [ 151.604110] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6034lb # [ 151.608281] niks3-server[1025]: 2026/09/21 10:44:40 INFO Completed upload id=516035worker2 # [ 151.615790] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:40Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16036lb # [ 151.610379] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6037worker1 # [ 151.616803] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16038worker2 # [ 151.627858] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:40Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06039worker1 # [ 151.629848] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626040worker1 # [ 151.634980] nix-daemon[839]: reaped child process 1552, status = succeeded6041worker1 # [ 151.641371] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626042worker1 # [ 151.651622] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.238Z level=INFO msg="All 1 paths already cached"6043worker1 # [ 151.658358] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626044worker1 # [ 151.668106] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6045worker1 # [ 151.852857] nix-daemon[839]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6046worker1 # [ 151.869862] nix-daemon[839]: reaped child process 1580, status = succeeded6047worker1 # [ 152.041091] nix-daemon[839]: accepted connection from pid 1582, user nix-grpc-daemon (trusted)6048worker1 # [ 152.057885] nix-daemon[839]: reaped child process 1585, status = succeeded6049lb # [ 152.061975] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received uploads request method=POST path=/api/pending_closures6050worker1 # [ 152.078508] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.664Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6051worker1 # [ 152.079963] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.664Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6052lb # [ 152.087565] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6053worker1 # [ 152.107610] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.693Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 152.101691] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6055lb # [ 152.111965] niks3-server[1025]: 2026/09/21 10:44:40 INFO Signed narinfos id=52 count=16056lb # [ 152.119146] niks3-server[1025]: 2026/09/21 10:44:40 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6057worker1 # [ 152.136698] nix-grpc-daemon[1459]: time=2026-09-21T10:44:40.723Z level=INFO msg="Upload complete. (457ms)"6058lb # [ 152.129316] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6059lb # [ 152.134739] niks3-server[1025]: 2026/09/21 10:44:40 INFO Completed upload id=526060worker1 # [ 152.142550] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=6246061lb # [ 152.138303] niks3-server[1025]: 2026/09/21 10:44:40 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6062worker1 # [ 152.155065] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626063worker1 # [ 152.160423] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6064worker1 # [ 152.164544] nix-daemon[839]: reaped child process 1575, status = succeeded6065worker1 # [ 152.175135] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6066worker1 # [ 153.819538] nix-daemon[839]: reaped child process 1589, status = succeeded6067worker1 # [ 153.999058] nix-daemon[839]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)6068worker1 # [ 154.016256] nix-daemon[839]: reaped child process 1608, status = succeeded6069lb # [ 154.021988] niks3-server[1025]: 2026/09/21 10:44:42 INFO Received uploads request method=POST path=/api/pending_closures6070worker1 # [ 154.040422] nix-grpc-daemon[1459]: time=2026-09-21T10:44:42.624Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6071worker1 # [ 154.042072] nix-grpc-daemon[1459]: time=2026-09-21T10:44:42.624Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6072lb # [ 154.049592] niks3-server[1025]: 2026/09/21 10:44:42 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6073lb # [ 154.069050] niks3-server[1025]: 2026/09/21 10:44:42 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6074lb # [ 154.079269] niks3-server[1025]: 2026/09/21 10:44:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6075worker1 # [ 154.091974] nix-grpc-daemon[1459]: time=2026-09-21T10:44:42.678Z level=INFO msg="Uploading 1 narinfos"6076lb # [ 154.089000] niks3-server[1025]: 2026/09/21 10:44:42 INFO Signed narinfos id=53 count=16077lb # [ 154.091550] niks3-server[1025]: 2026/09/21 10:44:42 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6078lb # [ 154.100824] niks3-server[1025]: 2026/09/21 10:44:42 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6079lb # [ 154.107745] niks3-server[1025]: 2026/09/21 10:44:42 INFO Completed upload id=536080worker1 # [ 154.115833] nix-grpc-daemon[1459]: time=2026-09-21T10:44:42.702Z level=INFO msg="Upload complete. (331ms)"6081lb # [ 154.110830] niks3-server[1025]: 2026/09/21 10:44:42 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6082worker2 # [ 154.127525] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:42Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16083worker1 # [ 154.129163] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16084worker2 # [ 154.140788] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:42Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06085worker1 # [ 154.146264] nix-daemon[839]: reaped child process 1587, status = succeeded6086worker1 # [ 154.181010] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626087worker1 # [ 154.196146] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626088worker1 # [ 154.209874] nix-grpc-daemon[1459]: time=2026-09-21T10:44:42.796Z level=INFO msg="All 1 paths already cached"6089worker1 # [ 154.218482] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626090worker1 # [ 154.232809] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6091worker1 # [ 154.441344] nix-daemon[839]: accepted connection from pid 1612, user nix-grpc-daemon (trusted)6092worker1 # [ 154.461115] nix-daemon[839]: reaped child process 1615, status = succeeded6093worker1 # [ 154.637333] nix-daemon[839]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6094worker1 # [ 154.653380] nix-daemon[839]: reaped child process 1620, status = succeeded6095lb # [ 154.656315] niks3-server[1025]: 2026/09/21 10:44:43 INFO Received uploads request method=POST path=/api/pending_closures6096worker1 # [ 154.673839] nix-grpc-daemon[1459]: time=2026-09-21T10:44:43.260Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6097worker1 # [ 154.675302] nix-grpc-daemon[1459]: time=2026-09-21T10:44:43.260Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6098lb # [ 154.684653] niks3-server[1025]: 2026/09/21 10:44:43 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6099lb # [ 154.699861] niks3-server[1025]: 2026/09/21 10:44:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6100worker1 # [ 154.718074] nix-grpc-daemon[1459]: time=2026-09-21T10:44:43.304Z level=INFO msg="Uploading 1 narinfos"6101lb # [ 154.710098] niks3-server[1025]: 2026/09/21 10:44:43 INFO Signed narinfos id=54 count=16102lb # [ 154.717497] niks3-server[1025]: 2026/09/21 10:44:43 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6103lb # [ 154.730180] niks3-server[1025]: 2026/09/21 10:44:43 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6104lb # [ 154.738340] niks3-server[1025]: 2026/09/21 10:44:43 INFO Completed upload id=546105worker1 # [ 154.746409] nix-grpc-daemon[1459]: time=2026-09-21T10:44:43.332Z level=INFO msg="Upload complete. (497ms)"6106lb # [ 154.741665] niks3-server[1025]: 2026/09/21 10:44:43 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6107worker1 # [ 154.756291] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=6246108worker1 # [ 154.768754] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626109worker1 # [ 154.776584] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6110worker1 # [ 154.781852] nix-daemon[839]: reaped child process 1610, status = succeeded6111worker1 # [ 154.796564] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6112worker1 # [ 156.042280] nix-daemon[839]: reaped child process 1624, status = succeeded6113worker1 # [ 156.222740] nix-daemon[839]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6114worker1 # [ 156.239222] nix-daemon[839]: reaped child process 1643, status = succeeded6115lb # [ 156.243935] niks3-server[1025]: 2026/09/21 10:44:44 INFO Received uploads request method=POST path=/api/pending_closures6116worker1 # [ 156.263241] nix-grpc-daemon[1459]: time=2026-09-21T10:44:44.847Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6117worker1 # [ 156.264995] nix-grpc-daemon[1459]: time=2026-09-21T10:44:44.848Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6118lb # [ 156.277388] niks3-server[1025]: 2026/09/21 10:44:44 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6119lb # [ 156.296388] niks3-server[1025]: 2026/09/21 10:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6120worker1 # [ 156.311061] nix-grpc-daemon[1459]: time=2026-09-21T10:44:44.897Z level=INFO msg="Uploading 1 narinfos"6121lb # [ 156.306676] niks3-server[1025]: 2026/09/21 10:44:44 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6122lb # [ 156.314844] niks3-server[1025]: 2026/09/21 10:44:44 INFO Signed narinfos id=55 count=16123lb # [ 156.321182] niks3-server[1025]: 2026/09/21 10:44:44 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6124worker1 # [ 156.328843] nix-grpc-daemon[1459]: time=2026-09-21T10:44:44.915Z level=INFO msg="Upload complete. (321ms)"6125lb # [ 156.324526] niks3-server[1025]: 2026/09/21 10:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6126lb # [ 156.328256] niks3-server[1025]: 2026/09/21 10:44:44 INFO Completed upload id=556127lb # [ 156.330161] niks3-server[1025]: 2026/09/21 10:44:44 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6128worker1 # [ 156.341006] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16129worker2 # [ 156.345626] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:44Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16130worker1 # [ 156.357833] nix-daemon[839]: reaped child process 1622, status = succeeded6131worker2 # [ 156.358804] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:44Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06132worker1 # [ 156.396174] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626133worker1 # [ 156.411658] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626134worker1 # [ 156.425497] nix-grpc-daemon[1459]: time=2026-09-21T10:44:45.011Z level=INFO msg="All 1 paths already cached"6135worker1 # [ 156.434121] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626136worker1 # [ 156.447976] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6137worker1 # [ 156.659394] nix-daemon[839]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6138worker1 # [ 156.678031] nix-daemon[839]: reaped child process 1650, status = succeeded6139worker1 # [ 156.850498] nix-daemon[839]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6140worker1 # [ 156.866735] nix-daemon[839]: reaped child process 1655, status = succeeded6141lb # [ 156.869964] niks3-server[1025]: 2026/09/21 10:44:45 INFO Received uploads request method=POST path=/api/pending_closures6142worker1 # [ 156.887167] nix-grpc-daemon[1459]: time=2026-09-21T10:44:45.473Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6143worker1 # [ 156.888728] nix-grpc-daemon[1459]: time=2026-09-21T10:44:45.473Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6144lb # [ 156.894469] niks3-server[1025]: 2026/09/21 10:44:45 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6145lb # [ 156.908697] niks3-server[1025]: 2026/09/21 10:44:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6146worker1 # [ 156.922662] nix-grpc-daemon[1459]: time=2026-09-21T10:44:45.508Z level=INFO msg="Uploading 1 narinfos"6147lb # [ 156.921196] niks3-server[1025]: 2026/09/21 10:44:45 INFO Signed narinfos id=56 count=16148lb # [ 156.927886] niks3-server[1025]: 2026/09/21 10:44:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6149lb # [ 156.936253] niks3-server[1025]: 2026/09/21 10:44:45 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6150lb # [ 156.941584] niks3-server[1025]: 2026/09/21 10:44:45 INFO Completed upload id=566151worker1 # [ 156.949568] nix-grpc-daemon[1459]: time=2026-09-21T10:44:45.536Z level=INFO msg="Upload complete. (484ms)"6152lb # [ 156.944743] niks3-server[1025]: 2026/09/21 10:44:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6153worker1 # [ 156.956151] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=6246154worker1 # [ 156.964709] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626155worker1 # [ 156.978112] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6156worker1 # [ 156.980942] nix-daemon[839]: reaped child process 1645, status = succeeded6157worker1 # [ 156.992209] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6158worker1 # [ 158.437105] nix-daemon[839]: reaped child process 1659, status = succeeded6159worker1 # [ 158.624133] nix-daemon[839]: accepted connection from pid 1675, user nix-grpc-daemon (trusted)6160worker1 # [ 158.643047] nix-daemon[839]: reaped child process 1678, status = succeeded6161lb # [ 158.647441] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received uploads request method=POST path=/api/pending_closures6162worker1 # [ 158.668452] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.253Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6163worker1 # [ 158.669932] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.253Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6164lb # [ 158.678515] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6165lb # [ 158.699783] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6166worker1 # [ 158.720133] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.306Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 158.714750] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6168lb # [ 158.725707] niks3-server[1025]: 2026/09/21 10:44:47 INFO Signed narinfos id=57 count=16169lb # [ 158.730281] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6170lb # [ 158.739993] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6171lb # [ 158.745452] niks3-server[1025]: 2026/09/21 10:44:47 INFO Completed upload id=576172worker1 # [ 158.753654] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.340Z level=INFO msg="Upload complete. (347ms)"6173lb # [ 158.748430] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6174worker2 # [ 158.764795] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:47Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16175worker1 # [ 158.765264] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16176worker2 # [ 158.778106] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:47Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06177worker1 # [ 158.787109] nix-daemon[839]: reaped child process 1657, status = succeeded6178worker1 # [ 158.816772] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626179worker1 # [ 158.832458] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626180worker1 # [ 158.847100] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.433Z level=INFO msg="All 2 paths already cached"6181worker1 # [ 158.856146] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626182worker1 # [ 158.870201] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6183worker1 # [ 159.071806] nix-daemon[839]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6184worker1 # [ 159.090913] nix-daemon[839]: reaped child process 1685, status = succeeded6185worker1 # [ 159.260380] nix-daemon[839]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6186worker1 # [ 159.275185] nix-daemon[839]: reaped child process 1690, status = succeeded6187lb # [ 159.281996] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received uploads request method=POST path=/api/pending_closures6188worker1 # [ 159.299511] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.885Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6189worker1 # [ 159.301084] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.885Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6190lb # [ 159.307010] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6191lb # [ 159.321655] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6192worker1 # [ 159.335303] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.921Z level=INFO msg="Uploading 1 narinfos"6193lb # [ 159.333917] niks3-server[1025]: 2026/09/21 10:44:47 INFO Signed narinfos id=58 count=16194lb # [ 159.339448] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6195lb # [ 159.348519] niks3-server[1025]: 2026/09/21 10:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6196worker1 # [ 159.354507] nix-grpc-daemon[1459]: time=2026-09-21T10:44:47.941Z level=INFO msg="Upload complete. (467ms)"6197lb # [ 159.352999] niks3-server[1025]: 2026/09/21 10:44:47 INFO Completed upload id=586198worker1 # [ 159.360329] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=7526199lb # [ 159.355862] niks3-server[1025]: 2026/09/21 10:44:47 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6200worker1 # [ 159.370785] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626201worker1 # [ 159.378183] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6202worker1 # [ 159.380314] nix-daemon[839]: reaped child process 1680, status = succeeded6203worker1 # [ 159.390442] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6204worker1 # [ 160.833406] nix-daemon[839]: reaped child process 1694, status = succeeded6205worker1 # [ 161.018319] nix-daemon[839]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6206worker1 # [ 161.034725] nix-daemon[839]: reaped child process 1713, status = succeeded6207lb # [ 161.040866] niks3-server[1025]: 2026/09/21 10:44:49 INFO Received uploads request method=POST path=/api/pending_closures6208worker1 # [ 161.060411] nix-grpc-daemon[1459]: time=2026-09-21T10:44:49.645Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6209worker1 # [ 161.061944] nix-grpc-daemon[1459]: time=2026-09-21T10:44:49.645Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6210lb # [ 161.082375] niks3-server[1025]: 2026/09/21 10:44:49 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6211lb # [ 161.093745] niks3-server[1025]: 2026/09/21 10:44:49 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6212worker1 # [ 161.113800] nix-grpc-daemon[1459]: time=2026-09-21T10:44:49.700Z level=INFO msg="Uploading 1 narinfos"6213lb # [ 161.107579] niks3-server[1025]: 2026/09/21 10:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6214lb # [ 161.117339] niks3-server[1025]: 2026/09/21 10:44:49 INFO Signed narinfos id=59 count=16215lb # [ 161.121748] niks3-server[1025]: 2026/09/21 10:44:49 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6216lb # [ 161.130343] niks3-server[1025]: 2026/09/21 10:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6217lb # [ 161.137097] niks3-server[1025]: 2026/09/21 10:44:49 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6218lb # [ 161.142303] niks3-server[1025]: 2026/09/21 10:44:49 INFO Completed upload id=596219worker1 # [ 161.150124] nix-grpc-daemon[1459]: time=2026-09-21T10:44:49.736Z level=INFO msg="Upload complete. (346ms)"6220worker2 # [ 161.159558] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:49Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16221worker1 # [ 161.161197] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16222worker1 # [ 161.181059] nix-daemon[839]: reaped child process 1692, status = succeeded6223worker1 # [ 161.215053] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626224worker1 # [ 161.229681] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626225worker1 # [ 161.244093] nix-grpc-daemon[1459]: time=2026-09-21T10:44:49.830Z level=INFO msg="All 2 paths already cached"6226worker1 # [ 161.253210] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626227worker1 # [ 161.267231] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6228worker1 # [ 161.470584] nix-daemon[839]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6229worker1 # [ 161.490138] nix-daemon[839]: reaped child process 1720, status = succeeded6230worker1 # [ 161.660676] nix-daemon[839]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6231worker1 # [ 161.678200] nix-daemon[839]: reaped child process 1725, status = succeeded6232lb # [ 161.681605] niks3-server[1025]: 2026/09/21 10:44:50 INFO Received uploads request method=POST path=/api/pending_closures6233worker1 # [ 161.698992] nix-grpc-daemon[1459]: time=2026-09-21T10:44:50.285Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6234worker1 # [ 161.700520] nix-grpc-daemon[1459]: time=2026-09-21T10:44:50.285Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6235lb # [ 161.707611] niks3-server[1025]: 2026/09/21 10:44:50 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6236worker1 # [ 161.729502] nix-grpc-daemon[1459]: time=2026-09-21T10:44:50.315Z level=INFO msg="Uploading 1 narinfos"6237lb # [ 161.722308] niks3-server[1025]: 2026/09/21 10:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6238lb # [ 161.733747] niks3-server[1025]: 2026/09/21 10:44:50 INFO Signed narinfos id=60 count=16239lb # [ 161.739924] niks3-server[1025]: 2026/09/21 10:44:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6240lb # [ 161.746546] niks3-server[1025]: 2026/09/21 10:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6241worker1 # [ 161.760512] nix-grpc-daemon[1459]: time=2026-09-21T10:44:50.347Z level=INFO msg="Upload complete. (476ms)"6242lb # [ 161.753429] niks3-server[1025]: 2026/09/21 10:44:50 INFO Completed upload id=606243lb # [ 161.756638] niks3-server[1025]: 2026/09/21 10:44:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6244worker1 # [ 161.768230] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=7526245worker1 # [ 161.779938] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626246worker1 # [ 161.785661] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6247worker1 # [ 161.790301] nix-daemon[839]: reaped child process 1715, status = succeeded6248worker1 # [ 161.803133] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6249worker1 # [ 163.026617] nix-daemon[839]: reaped child process 1729, status = succeeded6250worker1 # [ 163.182232] nix-daemon[839]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6251worker1 # [ 163.198822] nix-daemon[839]: reaped child process 1748, status = succeeded6252lb # [ 163.207573] niks3-server[1025]: 2026/09/21 10:44:51 INFO Received uploads request method=POST path=/api/pending_closures6253worker1 # [ 163.227977] nix-grpc-daemon[1459]: time=2026-09-21T10:44:51.813Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6254worker1 # [ 163.229587] nix-grpc-daemon[1459]: time=2026-09-21T10:44:51.813Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6255lb # [ 163.248598] niks3-server[1025]: 2026/09/21 10:44:51 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6256worker1 # [ 163.266201] nix-grpc-daemon[1459]: time=2026-09-21T10:44:51.852Z level=INFO msg="Uploading 1 narinfos"6257lb # [ 163.263585] niks3-server[1025]: 2026/09/21 10:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6258lb # [ 163.272304] niks3-server[1025]: 2026/09/21 10:44:51 INFO Signed narinfos id=61 count=16259lb # [ 163.275933] niks3-server[1025]: 2026/09/21 10:44:51 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6260worker1 # [ 163.291520] nix-grpc-daemon[1459]: time=2026-09-21T10:44:51.877Z level=INFO msg="Upload complete. (279ms)"6261lb # [ 163.283973] niks3-server[1025]: 2026/09/21 10:44:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6262lb # [ 163.287649] niks3-server[1025]: 2026/09/21 10:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6263lb # [ 163.291392] niks3-server[1025]: 2026/09/21 10:44:51 INFO Completed upload id=616264lb # [ 163.294196] niks3-server[1025]: 2026/09/21 10:44:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6265worker1 # [ 163.300921] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16266worker2 # [ 163.305991] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:51Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16267worker1 # [ 163.316970] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626268worker1 # [ 163.322263] nix-daemon[839]: reaped child process 1727, status = succeeded6269worker1 # [ 163.329055] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626270worker1 # [ 163.340257] nix-grpc-daemon[1459]: time=2026-09-21T10:44:51.926Z level=INFO msg="All 2 paths already cached"6271worker1 # [ 163.347667] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:449626272worker1 # [ 163.358287] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6273worker1 # [ 163.547475] nix-daemon[839]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6274worker1 # [ 163.563758] nix-daemon[839]: reaped child process 1755, status = succeeded6275worker1 # [ 163.738728] nix-daemon[839]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6276worker1 # [ 163.754708] nix-daemon[839]: reaped child process 1760, status = succeeded6277lb # [ 163.757855] niks3-server[1025]: 2026/09/21 10:44:52 INFO Received uploads request method=POST path=/api/pending_closures6278worker1 # [ 163.774312] nix-grpc-daemon[1459]: time=2026-09-21T10:44:52.360Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6279worker1 # [ 163.775789] nix-grpc-daemon[1459]: time=2026-09-21T10:44:52.360Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6280lb # [ 163.782490] niks3-server[1025]: 2026/09/21 10:44:52 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6281worker1 # [ 163.803381] nix-grpc-daemon[1459]: time=2026-09-21T10:44:52.389Z level=INFO msg="Uploading 1 narinfos"6282lb # [ 163.797727] niks3-server[1025]: 2026/09/21 10:44:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6283lb # [ 163.810160] niks3-server[1025]: 2026/09/21 10:44:52 INFO Signed narinfos id=62 count=16284lb # [ 163.821192] niks3-server[1025]: 2026/09/21 10:44:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6285lb # [ 163.825876] niks3-server[1025]: 2026/09/21 10:44:52 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6286worker1 # [ 163.838438] nix-grpc-daemon[1459]: time=2026-09-21T10:44:52.424Z level=INFO msg="Upload complete. (468ms)"6287lb # [ 163.830938] niks3-server[1025]: 2026/09/21 10:44:52 INFO Completed upload id=626288lb # [ 163.834074] niks3-server[1025]: 2026/09/21 10:44:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6289worker1 # [ 163.848277] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=0 paths=1 nar_bytes_in=7526290worker1 # [ 163.856243] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:449626291worker1 # [ 163.864799] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6292worker1 # [ 163.867455] nix-daemon[839]: reaped child process 1750, status = succeeded6293worker1 # [ 163.878154] nix-daemon[839]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6294worker1 # [ 165.104178] nix-daemon[839]: reaped child process 1764, status = succeeded6295worker1 # [ 165.269710] nix-daemon[839]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6296worker1 # [ 165.287077] nix-daemon[839]: reaped child process 1783, status = succeeded6297lb # [ 165.298032] niks3-server[1025]: 2026/09/21 10:44:53 INFO Received uploads request method=POST path=/api/pending_closures6298worker1 # [ 165.316397] nix-grpc-daemon[1459]: time=2026-09-21T10:44:53.900Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6299worker1 # [ 165.317878] nix-grpc-daemon[1459]: time=2026-09-21T10:44:53.900Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6300lb # [ 165.338329] niks3-server[1025]: 2026/09/21 10:44:53 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6301worker1 # [ 165.360168] nix-grpc-daemon[1459]: time=2026-09-21T10:44:53.946Z level=INFO msg="Uploading 1 narinfos"6302lb # [ 165.355210] niks3-server[1025]: 2026/09/21 10:44:53 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6303lb # [ 165.364256] niks3-server[1025]: 2026/09/21 10:44:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6304lb # [ 165.367713] niks3-server[1025]: 2026/09/21 10:44:53 INFO Signed narinfos id=63 count=16305lb # [ 165.370814] niks3-server[1025]: 2026/09/21 10:44:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6306worker1 # [ 165.387334] nix-grpc-daemon[1459]: time=2026-09-21T10:44:53.973Z level=INFO msg="Upload complete. (297ms)"6307lb # [ 165.380298] niks3-server[1025]: 2026/09/21 10:44:53 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6308lb # [ 165.384570] niks3-server[1025]: 2026/09/21 10:44:53 INFO Completed upload id=636309lb # [ 165.387244] niks3-server[1025]: 2026/09/21 10:44:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6310worker1 # [ 165.396871] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44962 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=16311worker1 # [ 165.409901] nix-daemon[839]: reaped child process 1762, status = succeeded6312client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.39 seconds)6313worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6314worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6315client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6316client # [ 165.681473] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6317client # Running as unit: gpu.service; invocation ID: 98d69fb3e13f4d4dbe03878468e57bf76318client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6319lb: must succeed: curl -sf localhost:9901/clusters6320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6321lb: must succeed: curl -sf localhost:9901/clusters6322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6323worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6324worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6325client # [ 165.934085] nix[1345]: don't know how to build these paths:6326client # [ 165.935047] nix[1345]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6327worker1 # [ 165.879916] nix-grpc-daemon[1399]: ts=2026-09-21T10:44:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:449626328worker2 # [ 165.889711] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594266329client # [ 165.966030] nix[1345]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6330worker2 # [ 165.914147] nix-grpc-daemon[1656]: ts=2026-09-21T10:44:54Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06331lb: must succeed: curl -sf localhost:9901/clusters6332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6333lb: must succeed: curl -sf localhost:9901/clusters6334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6335worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6336worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6337client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6338lb # [ 171.947486] envoy[934]: conn peer=[2001:db8:1::1]:34986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27217 tx=11571 ms=210686339client # [ 195.971708] nix[1345]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6340client # [ 195.982506] nix[1345]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6341client # [ 196.031128] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6342client # [ 196.038368] systemd[1]: gpu.service: Failed with result 'exit-code'.6343client # [ 196.042224] systemd[1]: gpu.service: Consumed 183ms CPU time over 30.355s wall clock time, 37.5M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6344client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds)6345client: must succeed: journalctl -u gpu -o cat6346client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds)6347lb: must succeed: curl -sf localhost:9901/clusters6348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6349lb: must succeed: curl -sf localhost:9901/clusters6350lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6351worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6352worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6353(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.50 seconds)6354subtest: low disk drains a worker and builds go to the other6355worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6356worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)6357worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6358worker1 # [ 197.797096] nix-grpc-daemon[1399]: ts=2026-09-21T10:45:26Z level=info event=unhealthy reason=min_free6359worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6360client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26361client # don't know how to build these paths:6362client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6363worker1 # [ 199.090844] nix-grpc-daemon[1399]: ts=2026-09-21T10:45:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:449626364worker1 # [ 199.098439] nix-grpc-daemon[1399]: ts=2026-09-21T10:45:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:449626365worker2 # [ 199.128561] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:27Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06366worker2 # [ 199.129265] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16367worker2 # [ 199.174473] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266368worker2 # [ 199.189065] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594266369client # copying 1 paths to 'lb:50051'...6370client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6371worker2 # [ 199.204665] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594266372worker2 # [ 199.220314] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6373worker2 # [ 199.424122] nix-daemon[863]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6374worker2 # [ 199.443757] nix-daemon[863]: reaped child process 1956, status = succeeded6375worker2 # [ 199.616384] nix-daemon[863]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6376worker2 # [ 199.634312] nix-daemon[863]: reaped child process 1961, status = succeeded6377lb # [ 199.637070] niks3-server[1025]: 2026/09/21 10:45:28 INFO Received uploads request method=POST path=/api/pending_closures6378worker2 # [ 199.659731] nix-grpc-daemon[1734]: time=2026-09-21T10:45:28.254Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6379worker2 # [ 199.661425] nix-grpc-daemon[1734]: time=2026-09-21T10:45:28.254Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6380lb # [ 199.671273] niks3-server[1025]: 2026/09/21 10:45:28 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6381lb # [ 199.685578] niks3-server[1025]: 2026/09/21 10:45:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6382worker2 # [ 199.700689] nix-grpc-daemon[1734]: time=2026-09-21T10:45:28.295Z level=INFO msg="Uploading 1 narinfos"6383lb # [ 199.693680] niks3-server[1025]: 2026/09/21 10:45:28 INFO Signed narinfos id=64 count=16384lb # [ 199.700597] niks3-server[1025]: 2026/09/21 10:45:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6385lb # [ 199.709259] niks3-server[1025]: 2026/09/21 10:45:28 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6386lb # [ 199.715273] niks3-server[1025]: 2026/09/21 10:45:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6387lb # [ 199.719060] niks3-server[1025]: 2026/09/21 10:45:28 INFO Completed upload id=646388worker2 # [ 199.728714] nix-grpc-daemon[1734]: time=2026-09-21T10:45:28.323Z level=INFO msg="Upload complete. (490ms)"6389worker2 # [ 199.736519] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=5126390worker2 # [ 199.751173] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266391worker2 # [ 199.761302] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6392worker2 # [ 199.765423] nix-daemon[863]: reaped child process 1951, status = succeeded6393client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6394worker2 # [ 199.776426] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6395client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6396client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6397client # farm-a> LOG-farm-a-drain6398worker2 # [ 200.816992] nix-daemon[863]: reaped child process 1965, status = succeeded6399worker2 # [ 200.984622] nix-daemon[863]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6400worker2 # [ 201.002355] nix-daemon[863]: reaped child process 1984, status = succeeded6401lb # [ 201.005549] niks3-server[1025]: 2026/09/21 10:45:29 INFO Received uploads request method=POST path=/api/pending_closures6402worker2 # [ 201.032223] nix-grpc-daemon[1734]: time=2026-09-21T10:45:29.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6403worker2 # [ 201.036548] nix-grpc-daemon[1734]: time=2026-09-21T10:45:29.628Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6404lb # [ 201.060466] niks3-server[1025]: 2026/09/21 10:45:29 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6405lb # [ 201.072439] niks3-server[1025]: 2026/09/21 10:45:29 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6406lb # [ 201.083559] niks3-server[1025]: 2026/09/21 10:45:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6407worker2 # [ 201.093626] nix-grpc-daemon[1734]: time=2026-09-21T10:45:29.688Z level=INFO msg="Uploading 1 narinfos"6408lb # [ 201.090869] niks3-server[1025]: 2026/09/21 10:45:29 INFO Signed narinfos id=65 count=16409lb # [ 201.095706] niks3-server[1025]: 2026/09/21 10:45:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6410lb # [ 201.104816] niks3-server[1025]: 2026/09/21 10:45:29 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6411worker2 # [ 201.114265] nix-grpc-daemon[1734]: time=2026-09-21T10:45:29.709Z level=INFO msg="Upload complete. (324ms)"6412lb # [ 201.107627] niks3-server[1025]: 2026/09/21 10:45:29 INFO Completed upload id=656413lb # [ 201.116179] niks3-server[1025]: 2026/09/21 10:45:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6414worker2 # [ 201.130054] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=16415worker2 # [ 201.148506] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06416worker2 # [ 201.154859] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16417worker2 # [ 201.159659] nix-daemon[863]: reaped child process 1963, status = succeeded6418worker2 # [ 201.188184] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266419worker2 # [ 201.202844] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594266420worker2 # [ 201.216595] nix-grpc-daemon[1734]: time=2026-09-21T10:45:29.811Z level=INFO msg="All 1 paths already cached"6421client # copying 1 paths to 'lb:50051'...6422client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6423worker2 # [ 201.226422] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594266424worker2 # [ 201.244458] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6425worker2 # [ 201.448231] nix-daemon[863]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6426worker2 # [ 201.464844] nix-daemon[863]: reaped child process 1993, status = succeeded6427worker2 # [ 201.642331] nix-daemon[863]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6428worker2 # [ 201.658381] nix-daemon[863]: reaped child process 1998, status = succeeded6429lb # [ 201.661055] niks3-server[1025]: 2026/09/21 10:45:30 INFO Received uploads request method=POST path=/api/pending_closures6430worker2 # [ 201.680061] nix-grpc-daemon[1734]: time=2026-09-21T10:45:30.274Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6431worker2 # [ 201.681527] nix-grpc-daemon[1734]: time=2026-09-21T10:45:30.274Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6432lb # [ 201.686537] niks3-server[1025]: 2026/09/21 10:45:30 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6433worker2 # [ 201.709957] nix-grpc-daemon[1734]: time=2026-09-21T10:45:30.304Z level=INFO msg="Uploading 1 narinfos"6434lb # [ 201.701268] niks3-server[1025]: 2026/09/21 10:45:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6435lb # [ 201.710441] niks3-server[1025]: 2026/09/21 10:45:30 INFO Signed narinfos id=66 count=16436lb # [ 201.718356] niks3-server[1025]: 2026/09/21 10:45:30 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6437lb # [ 201.725526] niks3-server[1025]: 2026/09/21 10:45:30 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6438lb # [ 201.731149] niks3-server[1025]: 2026/09/21 10:45:30 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6439lb # [ 201.735848] niks3-server[1025]: 2026/09/21 10:45:30 INFO Completed upload id=666440worker2 # [ 201.745507] nix-grpc-daemon[1734]: time=2026-09-21T10:45:30.340Z level=INFO msg="Upload complete. (484ms)"6441worker2 # [ 201.752358] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=6326442worker2 # [ 201.762880] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266443worker2 # [ 201.771253] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6444worker2 # [ 201.776845] nix-daemon[863]: reaped child process 1988, status = succeeded6445client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6446client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6447worker2 # [ 201.788317] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6448lb # [ 201.944447] envoy[934]: conn peer=[2001:db8:1::1]:58212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300656449client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6450client # farm-b> LOG-farm-b-drain6451worker2 # [ 202.617844] nix-daemon[863]: reaped child process 2002, status = succeeded6452worker2 # [ 202.780330] nix-daemon[863]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)6453worker2 # [ 202.798731] nix-daemon[863]: reaped child process 2021, status = succeeded6454lb # [ 202.802909] niks3-server[1025]: 2026/09/21 10:45:31 INFO Received uploads request method=POST path=/api/pending_closures6455worker2 # [ 202.824408] nix-grpc-daemon[1734]: time=2026-09-21T10:45:31.416Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6456worker2 # [ 202.825873] nix-grpc-daemon[1734]: time=2026-09-21T10:45:31.416Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6457lb # [ 202.847280] niks3-server[1025]: 2026/09/21 10:45:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6458worker2 # [ 202.865545] nix-grpc-daemon[1734]: time=2026-09-21T10:45:31.460Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 202.862169] niks3-server[1025]: 2026/09/21 10:45:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6460lb # [ 202.874199] niks3-server[1025]: 2026/09/21 10:45:31 INFO Signed narinfos id=67 count=16461lb # [ 202.878995] niks3-server[1025]: 2026/09/21 10:45:31 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6462worker2 # [ 202.897995] nix-grpc-daemon[1734]: time=2026-09-21T10:45:31.493Z level=INFO msg="Upload complete. (300ms)"6463lb # [ 202.889217] niks3-server[1025]: 2026/09/21 10:45:31 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6464worker2 # [ 202.904704] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=16465lb # [ 202.894925] niks3-server[1025]: 2026/09/21 10:45:31 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6466lb # [ 202.899521] niks3-server[1025]: 2026/09/21 10:45:31 INFO Completed upload id=676467lb # [ 202.901953] niks3-server[1025]: 2026/09/21 10:45:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6468worker2 # [ 202.916371] nix-daemon[863]: reaped child process 2000, status = succeeded6469worker2 # [ 202.920598] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06470worker2 # [ 202.923727] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16471worker2 # [ 202.969054] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266472worker2 # [ 202.983767] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594266473worker2 # [ 202.997591] nix-grpc-daemon[1734]: time=2026-09-21T10:45:31.592Z level=INFO msg="All 2 paths already cached"6474client # copying 1 paths to 'lb:50051'...6475client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6476worker2 # [ 203.007655] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594266477worker2 # [ 203.022480] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6478worker2 # [ 203.218505] nix-daemon[863]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6479worker2 # [ 203.235483] nix-daemon[863]: reaped child process 2030, status = succeeded6480worker2 # [ 203.410791] nix-daemon[863]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6481worker2 # [ 203.426540] nix-daemon[863]: reaped child process 2035, status = succeeded6482lb # [ 203.430320] niks3-server[1025]: 2026/09/21 10:45:32 INFO Received uploads request method=POST path=/api/pending_closures6483worker2 # [ 203.453411] nix-grpc-daemon[1734]: time=2026-09-21T10:45:32.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6484worker2 # [ 203.454920] nix-grpc-daemon[1734]: time=2026-09-21T10:45:32.048Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6485lb # [ 203.464091] niks3-server[1025]: 2026/09/21 10:45:32 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6486worker2 # [ 203.486364] nix-grpc-daemon[1734]: time=2026-09-21T10:45:32.081Z level=INFO msg="Uploading 1 narinfos"6487lb # [ 203.479715] niks3-server[1025]: 2026/09/21 10:45:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6488lb # [ 203.489681] niks3-server[1025]: 2026/09/21 10:45:32 INFO Signed narinfos id=68 count=16489lb # [ 203.493468] niks3-server[1025]: 2026/09/21 10:45:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6490lb # [ 203.501595] niks3-server[1025]: 2026/09/21 10:45:32 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6491lb # [ 203.507201] niks3-server[1025]: 2026/09/21 10:45:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6492lb # [ 203.511784] niks3-server[1025]: 2026/09/21 10:45:32 INFO Completed upload id=686493worker2 # [ 203.521401] nix-grpc-daemon[1734]: time=2026-09-21T10:45:32.116Z level=INFO msg="Upload complete. (484ms)"6494worker2 # [ 203.528332] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=0 paths=1 nar_bytes_in=7606495worker2 # [ 203.543039] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594266496worker2 # [ 203.550152] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6497worker2 # [ 203.555085] nix-daemon[863]: reaped child process 2025, status = succeeded6498client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6499client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6500worker2 # [ 203.566930] nix-daemon[863]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6501client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6502client # farm-top> LOG-farm-top-drain6503worker2 # [ 205.014644] nix-daemon[863]: reaped child process 2039, status = succeeded6504worker2 # [ 205.196889] nix-daemon[863]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6505worker2 # [ 205.213027] nix-daemon[863]: reaped child process 2058, status = succeeded6506lb # [ 205.218039] niks3-server[1025]: 2026/09/21 10:45:33 INFO Received uploads request method=POST path=/api/pending_closures6507worker2 # [ 205.237873] nix-grpc-daemon[1734]: time=2026-09-21T10:45:33.831Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6508worker2 # [ 205.239357] nix-grpc-daemon[1734]: time=2026-09-21T10:45:33.831Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6509lb # [ 205.250070] niks3-server[1025]: 2026/09/21 10:45:33 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6510lb # [ 205.262414] niks3-server[1025]: 2026/09/21 10:45:33 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6511worker2 # [ 205.288202] nix-grpc-daemon[1734]: time=2026-09-21T10:45:33.882Z level=INFO msg="Uploading 1 narinfos"6512lb # [ 205.272876] niks3-server[1025]: 2026/09/21 10:45:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6513lb # [ 205.282898] niks3-server[1025]: 2026/09/21 10:45:33 INFO Signed narinfos id=69 count=16514lb # [ 205.288702] niks3-server[1025]: 2026/09/21 10:45:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6515lb # [ 205.297521] niks3-server[1025]: 2026/09/21 10:45:33 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6516lb # [ 205.302698] niks3-server[1025]: 2026/09/21 10:45:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6517lb # [ 205.307328] niks3-server[1025]: 2026/09/21 10:45:33 INFO Completed upload id=696518worker2 # [ 205.317073] nix-grpc-daemon[1734]: time=2026-09-21T10:45:33.912Z level=INFO msg="Upload complete. (337ms)"6519worker2 # [ 205.329171] nix-grpc-daemon[1656]: ts=2026-09-21T10:45:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59426 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=16520worker2 # [ 205.348666] nix-daemon[863]: reaped child process 2037, status = succeeded6521client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.55 seconds)6522client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6523client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6524worker1: must succeed: rm /nix/.rw-store/fill6525worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6526worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6527worker1 # [ 206.806659] nix-grpc-daemon[1399]: ts=2026-09-21T10:45:35Z level=info event=healthy reason=min_free6528worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6529(finished: subtest: low disk drains a worker and builds go to the other, in 10.40 seconds)6530(finished: run the VM test script, in 207.90 seconds)6531test script finished in 208.19s6532cleanup6533kill QemuMachine (pid 50)6534client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6535kill QemuMachine (pid 49)6536lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6537kill QemuMachine (pid 51)6538worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6539kill QemuMachine (pid 52)6540worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6541(finished: cleanup, in 1.36 seconds)6542additionally exposed symbols:6543 client, lb, worker1, worker2,6544 vlan1,6545 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh