vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #482
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.OaXB5tiLxN', fmt=raw size=107374182414client # mke2fs 1.47.4 (6-Mar-2025)15client: QEMU running (pid 49)16client # Discarding device blocks: 0/262144 done17worker2: QEMU running (pid 52)18worker1 # Disk image does not exist, creating the virtualisation disk image...19lb: QEMU running (pid 50)20worker1: QEMU running (pid 51)21worker2 # Disk image does not exist, creating the virtualisation disk image...22client # Creating filesystem with 262144 4k blocks and 65536 inodes23worker1 # Formatting '/build/vm-state-worker1/tmp.5Gf2dfPdZX', fmt=raw size=107374182424lb # Disk image does not exist, creating the virtualisation disk image...25worker2 # Formatting '/build/vm-state-worker2/tmp.yu1NS20XvZ', fmt=raw size=107374182426worker1 # mke2fs 1.47.4 (6-Mar-2025)27lb # Formatting '/build/vm-state-lb/tmp.R6EgkM3Dv6', fmt=raw size=107374182428worker2 # mke2fs 1.47.4 (6-Mar-2025)29worker1 # Discarding device blocks: 0/262144 done30lb # mke2fs 1.47.4 (6-Mar-2025)31worker2 # Discarding device blocks: 0/262144 done32worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Discarding device blocks: 0/262144 done34worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # Filesystem UUID: 3cfca95e-4bb4-4ee8-be7e-aa06a0edcc5136lb # Creating filesystem with 262144 4k blocks and 65536 inodes37worker2 # Filesystem UUID: d54883b0-8719-4c61-8db3-af0db3350b2338worker1 # Superblock backups stored on blocks:39lb # Filesystem UUID: 40dcd69c-941f-4bcf-a1d9-bb695d63980640worker2 # Superblock backups stored on blocks:41worker1 # 32768, 98304, 163840, 22937642lb # Superblock backups stored on blocks:43worker2 # 32768, 98304, 163840, 22937644worker1 # 45lb # 32768, 98304, 163840, 22937646worker2 # 47worker1 # Allocating group tables: 0/8 done48lb # 49worker2 # Allocating group tables: 0/8 done50worker1 # Writing inode tables: 0/8 done51lb # Allocating group tables: 0/8 done52worker2 # Writing inode tables: 0/8 done53worker1 # Creating journal (8192 blocks): done54lb # Writing inode tables: 0/8 done55worker2 # Creating journal (8192 blocks): done56worker1 # Writing superblocks and filesystem accounting information: 0/8 done57lb # Creating journal (8192 blocks): done58worker2 # Writing superblocks and filesystem accounting information: 0/8 done59worker1 # 60lb # Writing superblocks and filesystem accounting information: 0/8 done61worker2 # 62worker1 # Virtualisation disk image created.63lb # 64worker2 # Virtualisation disk image created.65(finished: start all VMs, in 0.51 seconds)66client # Filesystem UUID: 74a6c8b5-3787-4fd7-a092-20ab8621e6ef67client: waiting for unit network-addresses-eth1.service68client # Superblock backups stored on blocks:69client: waiting for the VM to finish booting70client # 32768, 98304, 163840, 22937671lb # Virtualisation disk image created.72client # 73client # Allocating group tables: 0/8 done74client # Writing inode tables: 0/8 done75client # Creating journal (8192 blocks): done76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84worker2 # [ 0.000000] efi: UEFI not found.85worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker2 # [ 0.000000] Zone ranges:89worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker2 # [ 0.000000] DMA32 empty91worker2 # [ 0.000000] Normal empty92worker2 # [ 0.000000] Device empty93worker2 # [ 0.000000] Movable zone start for each node94worker2 # [ 0.000000] Early memory node ranges95worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker2 # [ 0.000000] psci: probing for conduit method from DT.99worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker2 # [ 0.000000] psci: Trusted OS migration not required102worker2 # [ 0.000000] psci: SMC Calling Convention v1.1103worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker2 # [ 0.000000] Detected PIPT I-cache on CPU0106worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker2 # [ 0.000000] CPU features: detected: Spectre-v4109worker2 # [ 0.000000] CPU features: detected: Spectre-BHB110worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker2 # [ 0.000000] alternatives: applying boot alternatives113worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/14hr8lvd5yx0xy2any9p8ljdzcwwnbz0-nixos-system-worker2-test/init regInfo=/nix/store/xhr10qi0r1akkix7571ql8y9sic62ycg-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/xhr10qi0r1akkix7571ql8y9sic62ycg-closure-info/registration", will be passed to user space.115worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker2 # [ 0.000000] software IO TLB: area num 1.120worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker2 # [ 0.000000] Fallback order for Node 0: 0123worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124worker2 # [ 0.000000] Policy zone: DMA125worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127worker2 # [ 0.000000] allocated 3145728 bytes of page_ext128worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups130worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.131worker2 # [ 0.000000] rcu: RCU event tracing is enabled.132client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]133worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]134worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.135worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.136client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026137worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026138worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.139client # [ 0.000000] KASLR enabled140worker1 # [ 0.000000] KASLR enabled141worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.142client # [ 0.000000] random: crng init done143worker1 # [ 0.000000] random: crng init done144client # [ 0.000000] Machine model: linux,dummy-virt145lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]146worker1 # [ 0.000000] Machine model: linux,dummy-virt147client # [ 0.000000] efi: UEFI not found.148worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.149worker1 # [ 0.000000] efi: UEFI not found.150worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1151client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT152worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT153lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026154lb # [ 0.000000] KASLR enabled155client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]156worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]157lb # [ 0.000000] random: crng init done158worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.159lb # [ 0.000000] Machine model: linux,dummy-virt160worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]161client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]162client # [ 0.000000] Zone ranges:163worker1 # [ 0.000000] Zone ranges:164lb # [ 0.000000] efi: UEFI not found.165worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.166worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]167client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]168lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT169worker1 # [ 0.000000] DMA32 empty170client # [ 0.000000] DMA32 empty171worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.172worker1 # [ 0.000000] Normal empty173client # [ 0.000000] Normal empty174worker1 # [ 0.000000] Device empty175lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]176client # [ 0.000000] Device empty177worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0178worker1 # [ 0.000000] Movable zone start for each node179client # [ 0.000000] Movable zone start for each node180lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]181worker2 # [ 0.000000] GICv3: 256 SPIs implemented182lb # [ 0.000000] Zone ranges:183worker1 # [ 0.000000] Early memory node ranges184client # [ 0.000000] Early memory node ranges185worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented186lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]187worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]188client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]189worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq190lb # [ 0.000000] DMA32 empty191worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI192lb # [ 0.000000] Normal empty193worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]194client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]195lb # [ 0.000000] Device empty196worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0197worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000198client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000199lb # [ 0.000000] Movable zone start for each node200worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000201lb # [ 0.000000] Early memory node ranges202worker1 # [ 0.000000] psci: probing for conduit method from DT.203client # [ 0.000000] psci: probing for conduit method from DT.204worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]205worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.206client # [ 0.000000] psci: PSCIv1.3 detected in firmware.207lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]208worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs209client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs210lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]211worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)212worker1 # [ 0.000000] psci: Trusted OS migration not required213client # [ 0.000000] psci: Trusted OS migration not required214lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000215worker1 # [ 0.000000] psci: SMC Calling Convention v1.1216client # [ 0.000000] psci: SMC Calling Convention v1.1217worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)218lb # [ 0.000000] psci: probing for conduit method from DT.219lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.220worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)221worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000222client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)223lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs224worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296225client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296226worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000227lb # [ 0.000000] psci: Trusted OS migration not required228worker1 # [ 0.000000] Detected PIPT I-cache on CPU0229client # [ 0.000000] Detected PIPT I-cache on CPU0230lb # [ 0.000000] psci: SMC Calling Convention v1.1231worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.232worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)233client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)234lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)235worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface236worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns237client # [ 0.000000] CPU features: detected: GICv3 CPU interface238worker1 # [ 0.000000] CPU features: detected: Spectre-v4239lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296240worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).241client # [ 0.000000] CPU features: detected: Spectre-v4242lb # [ 0.000000] Detected PIPT I-cache on CPU0243worker1 # [ 0.000000] CPU features: detected: Spectre-BHB244client # [ 0.000000] CPU features: detected: Spectre-BHB245worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38246client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38247worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns248lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)249worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23250client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23251lb # [ 0.000000] CPU features: detected: GICv3 CPU interface252worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns253worker1 # [ 0.000000] alternatives: applying boot alternatives254lb # [ 0.000000] CPU features: detected: Spectre-v4255client # [ 0.000000] alternatives: applying boot alternatives256worker2 # [ 0.000034] arm-pv: using stolen time PV257lb # [ 0.000000] CPU features: detected: Spectre-BHB258lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38259worker2 # [ 0.000655] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)260worker2 # [ 0.000821] Console: colour dummy device 80x25261lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23262worker2 # [ 0.000828] printk: legacy console [tty0] enabled263lb # [ 0.000000] alternatives: applying boot alternatives264worker2 # [ 0.001016] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)265worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/378wk3kp5ahmavfrq31vxapf3z8y7hlm-nixos-system-worker1-test/init regInfo=/nix/store/ncd58zbngf582vlqw90ss78w54maxk0z-closure-info/registration console=ttyAMA0,115200n8 console=tty0266client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test/init regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration console=ttyAMA0,115200n8 console=tty0267worker2 # [ 0.001023] pid_max: default: 32768 minimum: 301268worker2 # [ 0.001118] LSM: initializing lsm=capability,landlock,yama,bpf,ima269worker2 # [ 0.001289] landlock: Up and running.270worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/ncd58zbngf582vlqw90ss78w54maxk0z-closure-info/registration", will be passed to user space.271client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration", will be passed to user space.272worker2 # [ 0.001291] Yama: becoming mindful.273worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes274worker2 # [ 0.001926] LSM support for eBPF active275lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test/init regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration console=ttyAMA0,115200n8 console=tty0276client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes277worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)278worker2 # [ 0.002070] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)279client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)280lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration", will be passed to user space.281worker2 # [ 0.002102] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)282worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)283client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)284worker2 # [ 0.003296] cacheinfo: Unable to detect cache hierarchy for CPU 0285lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes286worker2 # [ 0.004065] rcu: Hierarchical SRCU implementation.287worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB288lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)289client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB290worker1 # [ 0.000000] software IO TLB: area num 1.291worker2 # [ 0.004070] rcu: Max phase no-delay instances is 1000.292client # [ 0.000000] software IO TLB: area num 1.293worker2 # [ 0.005236] fsl-mc MSI: its@8080000 domain created294worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB295lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)296client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB297worker2 # [ 0.005323] EFI services will not be available.298worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)299worker2 # [ 0.005402] smp: Bringing up secondary CPUs ...300client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)301worker1 # [ 0.000000] Fallback order for Node 0: 0302lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB303worker2 # [ 0.005410] smp: Brought up 1 node, 1 CPU304client # [ 0.000000] Fallback order for Node 0: 0305lb # [ 0.000000] software IO TLB: area num 1.306worker2 # [ 0.005412] SMP: Total of 1 processors activated.307worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216308worker2 # [ 0.005418] CPU: All CPU(s) started at EL1309worker1 # [ 0.000000] Policy zone: DMA310client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216311lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB312client # [ 0.000000] Policy zone: DMA313worker2 # [ 0.005429] CPU features: detected: Branch Target Identification314worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off315lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)316client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off317worker2 # [ 0.005433] CPU features: detected: ARMv8.4 Translation Table Level318lb # [ 0.000000] Fallback order for Node 0: 0319worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1320client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1321worker1 # [ 0.000000] allocated 3145728 bytes of page_ext322lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216323worker2 # [ 0.005437] CPU features: detected: Instruction cache invalidation not required for I/D coherence324client # [ 0.000000] allocated 3145728 bytes of page_ext325lb # [ 0.000000] Policy zone: DMA326worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages327client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages328worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups329worker2 # [ 0.005440] CPU features: detected: Data cache clean to the PoU not required for I/D coherence330lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off331client # [ 0.000000] ftrace: allocated 294 pages with 4 groups332worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.333lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1334worker2 # [ 0.005444] CPU features: detected: Common not Private translations335client # [ 0.000000] rcu: Hierarchical RCU implementation.336worker1 # [ 0.000000] rcu: RCU event tracing is enabled.337lb # [ 0.000000] allocated 3145728 bytes of page_ext338worker2 # [ 0.005447] CPU features: detected: CRC32 instructions339client # [ 0.000000] rcu: RCU event tracing is enabled.340worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.341lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages342client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.343worker2 # [ 0.005449] CPU features: detected: Data cache clean to Point of Deep Persistence344worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.345lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups346client # [ 0.000000] Trampoline variant of Tasks RCU enabled.347worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.348worker2 # [ 0.005453] CPU features: detected: Data cache clean to Point of Persistence349lb # [ 0.000000] rcu: Hierarchical RCU implementation.350client # [ 0.000000] Rude variant of Tasks RCU enabled.351worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.352lb # [ 0.000000] rcu: RCU event tracing is enabled.353client # [ 0.000000] Tracing variant of Tasks RCU enabled.354worker2 # [ 0.005456] CPU features: detected: Data independent timing control (DIT)355worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.356worker2 # [ 0.005459] CPU features: detected: E0PD357lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.358client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.359lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.360worker2 # [ 0.005462] CPU features: detected: Enhanced Counter Virtualization361worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1362client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1363lb # [ 0.000000] Rude variant of Tasks RCU enabled.364worker2 # [ 0.005465] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)365lb # [ 0.000000] Tracing variant of Tasks RCU enabled.366worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.367client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.368worker2 # [ 0.005468] CPU features: detected: Enhanced Virtualization Traps369lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.370worker2 # [ 0.005471] CPU features: detected: Fine Grained Traps371worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.373lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1374worker2 # [ 0.005474] CPU features: detected: Generic authentication (architected QARMA5 algorithm)375worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.376client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.377lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0379worker2 # [ 0.005480] CPU features: detected: RCpc load-acquire (LDAPR)380worker1 # [ 0.000000] GICv3: 256 SPIs implemented381client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0382worker2 # [ 0.005483] CPU features: detected: LSE atomic instructions383client # [ 0.000000] GICv3: 256 SPIs implemented384lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.385worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented386worker2 # [ 0.005486] CPU features: detected: Privileged Access Never387worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq388client # [ 0.000000] GICv3: 0 Extended SPIs implemented389worker2 # [ 0.005488] CPU features: detected: PMUv3390client # [ 0.000000] Root IRQ handler: gic_handle_irq391lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.392worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI393worker2 # [ 0.005491] CPU features: detected: RAS Extension Support394client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI395worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0396lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0397lb # [ 0.000000] GICv3: 256 SPIs implemented398client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0399worker2 # [ 0.005494] CPU features: detected: RASv1p1 Extension Support400worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000401lb # [ 0.000000] GICv3: 0 Extended SPIs implemented402worker2 # [ 0.005496] CPU features: detected: Random Number Generator403worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]404client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000405lb # [ 0.000000] Root IRQ handler: gic_handle_irq406client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]407lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI408worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)409worker2 # [ 0.005499] CPU features: detected: Speculation barrier (SB)410lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0411client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)412worker2 # [ 0.005502] CPU features: detected: Stage-2 Force Write-Back413lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000414worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)415lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]416worker2 # [ 0.005505] CPU features: detected: TLB range maintenance instructions417client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)418worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000419worker2 # [ 0.005509] CPU features: detected: Speculative Store Bypassing Safe (SSBS)420client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000421lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)422worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000423worker2 # [ 0.005549] alternatives: applying system-wide alternatives424client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000425worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.426lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)427worker2 # [ 0.008472] CPU features: detected: BBM Level 2 without TLB conflict abort428client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.429lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000430worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns431client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns432lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000433worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).434worker2 # [ 0.008612] Memory: 1406536K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)435worker2 # [ 0.009025] devtmpfs: initialized436client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).437lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.438worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns439worker2 # [ 0.010813] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)440lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns441client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns442worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns443worker2 # [ 0.010835] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).444lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).445worker1 # [ 0.000028] arm-pv: using stolen time PV446client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns447worker2 # [ 0.011021] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL448client # [ 0.000030] arm-pv: using stolen time PV449worker2 # [ 0.011026] 0 pages in range for non-PLT usage450lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns451worker1 # [ 0.000399] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)452worker2 # [ 0.011027] 508288 pages in range for PLT usage453worker1 # [ 0.000560] Console: colour dummy device 80x25454client # [ 0.000398] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)455lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns456worker2 # [ 0.011155] pinctrl core: initialized pinctrl subsystem457worker1 # [ 0.000568] printk: legacy console [tty0] enabled458client # [ 0.000544] Console: colour dummy device 80x25459lb # [ 0.000030] arm-pv: using stolen time PV460worker2 # [ 0.011906] DMI not present or invalid.461client # [ 0.000552] printk: legacy console [tty0] enabled462worker2 # [ 0.015671] NET: Registered PF_NETLINK/PF_ROUTE protocol family463worker1 # [ 0.000759] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)464lb # [ 0.000406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)465worker1 # [ 0.000765] pid_max: default: 32768 minimum: 301466client # [ 0.000766] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)467worker2 # [ 0.017869] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations468lb # [ 0.000578] Console: colour dummy device 80x25469client # [ 0.000773] pid_max: default: 32768 minimum: 301470worker1 # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima471lb # [ 0.000585] printk: legacy console [tty0] enabled472worker2 # [ 0.018082] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations473worker1 # [ 0.000970] landlock: Up and running.474client # [ 0.000845] LSM: initializing lsm=capability,landlock,yama,bpf,ima475worker1 # [ 0.000973] Yama: becoming mindful.476client # [ 0.000985] landlock: Up and running.477lb # [ 0.000787] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)478worker2 # [ 0.018293] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations479worker1 # [ 0.001431] LSM support for eBPF active480client # [ 0.000987] Yama: becoming mindful.481lb # [ 0.000794] pid_max: default: 32768 minimum: 301482worker2 # [ 0.018314] audit: initializing netlink subsys (disabled)483client # [ 0.001425] LSM support for eBPF active484worker1 # [ 0.001572] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)485lb # [ 0.000862] LSM: initializing lsm=capability,landlock,yama,bpf,ima486worker2 # [ 0.018924] thermal_sys: Registered thermal governor 'fair_share'487client # [ 0.001575] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)488lb # [ 0.001018] landlock: Up and running.489worker1 # [ 0.001610] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)490worker2 # [ 0.018926] thermal_sys: Registered thermal governor 'bang_bang'491lb # [ 0.001021] Yama: becoming mindful.492client # [ 0.001606] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)493lb # [ 0.001457] LSM support for eBPF active494worker1 # [ 0.002707] cacheinfo: Unable to detect cache hierarchy for CPU 0495worker2 # [ 0.018929] thermal_sys: Registered thermal governor 'step_wise'496worker1 # [ 0.003453] rcu: Hierarchical SRCU implementation.497client # [ 0.002662] cacheinfo: Unable to detect cache hierarchy for CPU 0498lb # [ 0.001598] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)499worker2 # [ 0.018934] thermal_sys: Registered thermal governor 'user_space'500worker1 # [ 0.003457] rcu: Max phase no-delay instances is 1000.501client # [ 0.003409] rcu: Hierarchical SRCU implementation.502worker2 # [ 0.018937] thermal_sys: Registered thermal governor 'power_allocator'503worker1 # [ 0.004642] fsl-mc MSI: its@8080000 domain created504lb # [ 0.001631] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)505client # [ 0.003413] rcu: Max phase no-delay instances is 1000.506worker1 # [ 0.004732] EFI services will not be available.507client # [ 0.004617] fsl-mc MSI: its@8080000 domain created508lb # [ 0.002718] cacheinfo: Unable to detect cache hierarchy for CPU 0509worker2 # [ 0.018962] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1510worker1 # [ 0.004816] smp: Bringing up secondary CPUs ...511client # [ 0.004707] EFI services will not be available.512worker2 # [ 0.018971] cpuidle: using governor ladder513lb # [ 0.003419] rcu: Hierarchical SRCU implementation.514worker1 # [ 0.004823] smp: Brought up 1 node, 1 CPU515worker2 # [ 0.018976] cpuidle: using governor menu516client # [ 0.004786] smp: Bringing up secondary CPUs ...517lb # [ 0.003423] rcu: Max phase no-delay instances is 1000.518worker1 # [ 0.004826] SMP: Total of 1 processors activated.519client # [ 0.004793] smp: Brought up 1 node, 1 CPU520lb # [ 0.004647] fsl-mc MSI: its@8080000 domain created521worker1 # [ 0.004832] CPU: All CPU(s) started at EL1522worker2 # [ 0.019182] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.523client # [ 0.004797] SMP: Total of 1 processors activated.524lb # [ 0.004747] EFI services will not be available.525worker2 # [ 0.019198] ASID allocator initialised with 65536 entries526client # [ 0.004802] CPU: All CPU(s) started at EL1527worker1 # [ 0.004843] CPU features: detected: Branch Target Identification528lb # [ 0.004813] smp: Bringing up secondary CPUs ...529worker2 # [ 0.020493] Serial: AMBA PL011 UART driver530worker1 # [ 0.004848] CPU features: detected: ARMv8.4 Translation Table Level531lb # [ 0.004821] smp: Brought up 1 node, 1 CPU532client # [ 0.004812] CPU features: detected: Branch Target Identification533lb # [ 0.004825] SMP: Total of 1 processors activated.534client # [ 0.004817] CPU features: detected: ARMv8.4 Translation Table Level535worker2 # [ 0.025850] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1536worker1 # [ 0.004851] CPU features: detected: Instruction cache invalidation not required for I/D coherence537lb # [ 0.004830] CPU: All CPU(s) started at EL1538worker2 # [ 0.026006] printk: console [ttyAMA0] enabled539lb # [ 0.004840] CPU features: detected: Branch Target Identification540client # [ 0.004821] CPU features: detected: Instruction cache invalidation not required for I/D coherence541worker1 # [ 0.004854] CPU features: detected: Data cache clean to the PoU not required for I/D coherence542lb # [ 0.004844] CPU features: detected: ARMv8.4 Translation Table Level543worker1 # [ 0.004858] CPU features: detected: Common not Private translations544client # [ 0.004824] CPU features: detected: Data cache clean to the PoU not required for I/D coherence545worker1 # [ 0.004861] CPU features: detected: CRC32 instructions546lb # [ 0.004847] CPU features: detected: Instruction cache invalidation not required for I/D coherence547client # [ 0.004828] CPU features: detected: Common not Private translations548client # [ 0.004831] CPU features: detected: CRC32 instructions549worker1 # [ 0.004864] CPU features: detected: Data cache clean to Point of Deep Persistence550lb # [ 0.004850] CPU features: detected: Data cache clean to the PoU not required for I/D coherence551worker1 # [ 0.004868] CPU features: detected: Data cache clean to Point of Persistence552client # [ 0.004834] CPU features: detected: Data cache clean to Point of Deep Persistence553lb # [ 0.004854] CPU features: detected: Common not Private translations554lb # [ 0.004857] CPU features: detected: CRC32 instructions555worker1 # [ 0.004871] CPU features: detected: Data independent timing control (DIT)556client # [ 0.004838] CPU features: detected: Data cache clean to Point of Persistence557worker1 # [ 0.004874] CPU features: detected: E0PD558lb # [ 0.004860] CPU features: detected: Data cache clean to Point of Deep Persistence559client # [ 0.004841] CPU features: detected: Data independent timing control (DIT)560worker1 # [ 0.004876] CPU features: detected: Enhanced Counter Virtualization561client # [ 0.004844] CPU features: detected: E0PD562lb # [ 0.004863] CPU features: detected: Data cache clean to Point of Persistence563client # [ 0.004846] CPU features: detected: Enhanced Counter Virtualization564worker1 # [ 0.004880] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)565lb # [ 0.004867] CPU features: detected: Data independent timing control (DIT)566worker1 # [ 0.004883] CPU features: detected: Enhanced Virtualization Traps567client # [ 0.004849] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)568worker2 # [ 0.144043] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages569lb # [ 0.004870] CPU features: detected: E0PD570worker1 # [ 0.004886] CPU features: detected: Fine Grained Traps571lb # [ 0.004872] CPU features: detected: Enhanced Counter Virtualization572worker2 # [ 0.144060] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page573client # [ 0.004852] CPU features: detected: Enhanced Virtualization Traps574client # [ 0.004855] CPU features: detected: Fine Grained Traps575worker1 # [ 0.004889] CPU features: detected: Generic authentication (architected QARMA5 algorithm)576worker2 # [ 0.144065] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages577lb # [ 0.004875] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)578worker1 # [ 0.004894] CPU features: detected: RCpc load-acquire (LDAPR)579client # [ 0.004859] CPU features: detected: Generic authentication (architected QARMA5 algorithm)580worker2 # [ 0.144069] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page581lb # [ 0.004879] CPU features: detected: Enhanced Virtualization Traps582worker1 # [ 0.004897] CPU features: detected: LSE atomic instructions583lb # [ 0.004882] CPU features: detected: Fine Grained Traps584worker2 # [ 0.144074] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages585client # [ 0.004864] CPU features: detected: RCpc load-acquire (LDAPR)586worker1 # [ 0.004900] CPU features: detected: Privileged Access Never587client # [ 0.004867] CPU features: detected: LSE atomic instructions588lb # [ 0.004886] CPU features: detected: Generic authentication (architected QARMA5 algorithm)589worker1 # [ 0.004903] CPU features: detected: PMUv3590worker2 # [ 0.144079] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page591client # [ 0.004870] CPU features: detected: Privileged Access Never592worker1 # [ 0.004905] CPU features: detected: RAS Extension Support593worker2 # [ 0.144083] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages594lb # [ 0.004891] CPU features: detected: RCpc load-acquire (LDAPR)595client # [ 0.004872] CPU features: detected: PMUv3596worker1 # [ 0.004908] CPU features: detected: RASv1p1 Extension Support597worker2 # [ 0.144087] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page598lb # [ 0.004894] CPU features: detected: LSE atomic instructions599client # [ 0.004875] CPU features: detected: RAS Extension Support600worker1 # [ 0.004911] CPU features: detected: Random Number Generator601lb # [ 0.004897] CPU features: detected: Privileged Access Never602client # [ 0.004878] CPU features: detected: RASv1p1 Extension Support603lb # [ 0.004899] CPU features: detected: PMUv3604worker1 # [ 0.004913] CPU features: detected: Speculation barrier (SB)605client # [ 0.004881] CPU features: detected: Random Number Generator606lb # [ 0.004902] CPU features: detected: RAS Extension Support607worker1 # [ 0.004916] CPU features: detected: Stage-2 Force Write-Back608client # [ 0.004883] CPU features: detected: Speculation barrier (SB)609lb # [ 0.004905] CPU features: detected: RASv1p1 Extension Support610worker2 # [ 0.151174] fbcon: Taking over console611worker1 # [ 0.004919] CPU features: detected: TLB range maintenance instructions612worker2 # [ 0.151189] ACPI: Interpreter disabled.613client # [ 0.004886] CPU features: detected: Stage-2 Force Write-Back614lb # [ 0.004907] CPU features: detected: Random Number Generator615worker1 # [ 0.004923] CPU features: detected: Speculative Store Bypassing Safe (SSBS)616client # [ 0.004889] CPU features: detected: TLB range maintenance instructions617lb # [ 0.004910] CPU features: detected: Speculation barrier (SB)618worker1 # [ 0.004959] alternatives: applying system-wide alternatives619lb # [ 0.004913] CPU features: detected: Stage-2 Force Write-Back620client # [ 0.004894] CPU features: detected: Speculative Store Bypassing Safe (SSBS)621worker1 # [ 0.007856] CPU features: detected: BBM Level 2 without TLB conflict abort622client # [ 0.004930] alternatives: applying system-wide alternatives623lb # [ 0.004916] CPU features: detected: TLB range maintenance instructions624client # [ 0.007836] CPU features: detected: BBM Level 2 without TLB conflict abort625lb # [ 0.004920] CPU features: detected: Speculative Store Bypassing Safe (SSBS)626lb # [ 0.004957] alternatives: applying system-wide alternatives627worker1 # [ 0.008052] Memory: 1406608K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)628worker1 # [ 0.008439] devtmpfs: initialized629lb # [ 0.007904] CPU features: detected: BBM Level 2 without TLB conflict abort630worker1 # [ 0.010226] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)631worker1 # [ 0.010248] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).632worker1 # [ 0.010416] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL633worker1 # [ 0.010421] 0 pages in range for non-PLT usage634worker1 # [ 0.010422] 508288 pages in range for PLT usage635worker2 # [ 0.160071] iommu: Default domain type: Translated636worker1 # [ 0.010537] pinctrl core: initialized pinctrl subsystem637worker2 # [ 0.160082] iommu: DMA domain TLB invalidation policy: strict mode638worker1 # [ 0.011339] DMI not present or invalid.639worker2 # [ 0.160450] SCSI subsystem initialized640worker1 # [ 0.014580] NET: Registered PF_NETLINK/PF_ROUTE protocol family641worker1 # [ 0.016799] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations642worker2 # [ 0.162421] usbcore: registered new interface driver usbfs643worker2 # [ 0.162457] usbcore: registered new interface driver hub644worker1 # [ 0.016975] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations645worker2 # [ 0.162481] usbcore: registered new device driver usb646worker2 # [ 0.162746] pps_core: LinuxPPS API ver. 1 registered647client # [ 0.007989] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)648client # [ 0.008411] devtmpfs: initialized649worker2 # [ 0.162751] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>650worker2 # [ 0.162761] PTP clock support registered651client # [ 0.010196] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)652lb # [ 0.008125] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)653worker2 # [ 0.162811] EDAC MC: Ver: 3.0.0654lb # [ 0.008506] devtmpfs: initialized655client # [ 0.010218] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).656worker2 # [ 0.167381] scmi_core: SCMI protocol bus registered657lb # [ 0.010259] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)658client # [ 0.010406] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL659worker2 # [ 0.168304] FPGA manager framework660client # [ 0.010410] 0 pages in range for non-PLT usage661lb # [ 0.010281] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).662worker2 # [ 0.169193] vgaarb: loaded663client # [ 0.010411] 508288 pages in range for PLT usage664lb # [ 0.010455] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL665client # [ 0.010516] pinctrl core: initialized pinctrl subsystem666worker2 # [ 0.169796] clocksource: Switched to clocksource arch_sys_counter667lb # [ 0.010459] 0 pages in range for non-PLT usage668client # [ 0.011273] DMI not present or invalid.669lb # [ 0.010460] 508288 pages in range for PLT usage670client # [ 0.014457] NET: Registered PF_NETLINK/PF_ROUTE protocol family671lb # [ 0.010575] pinctrl core: initialized pinctrl subsystem672lb # [ 0.011348] DMI not present or invalid.673client # [ 0.016641] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations674lb # [ 0.014459] NET: Registered PF_NETLINK/PF_ROUTE protocol family675client # [ 0.016825] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations676worker1 # [ 0.017178] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations677worker2 # [ 0.173023] VFS: Disk quotas dquot_6.6.0678lb # [ 0.016646] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations679worker1 # [ 0.017201] audit: initializing netlink subsys (disabled)680client # [ 0.017022] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations681worker2 # [ 0.173058] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)682worker1 # [ 0.017774] thermal_sys: Registered thermal governor 'fair_share'683lb # [ 0.016828] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations684client # [ 0.017043] audit: initializing netlink subsys (disabled)685worker1 # [ 0.017776] thermal_sys: Registered thermal governor 'bang_bang'686lb # [ 0.017023] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations687client # [ 0.017619] thermal_sys: Registered thermal governor 'fair_share'688worker1 # [ 0.017780] thermal_sys: Registered thermal governor 'step_wise'689lb # [ 0.017045] audit: initializing netlink subsys (disabled)690client # [ 0.017621] thermal_sys: Registered thermal governor 'bang_bang'691worker1 # [ 0.017785] thermal_sys: Registered thermal governor 'user_space'692lb # [ 0.017572] thermal_sys: Registered thermal governor 'fair_share'693client # [ 0.017625] thermal_sys: Registered thermal governor 'step_wise'694worker2 # [ 0.176606] netfs: FS-Cache loaded695worker1 # [ 0.017788] thermal_sys: Registered thermal governor 'power_allocator'696lb # [ 0.017574] thermal_sys: Registered thermal governor 'bang_bang'697worker2 # [ 0.176722] pnp: PnP ACPI: disabled698client # [ 0.017630] thermal_sys: Registered thermal governor 'user_space'699lb # [ 0.017578] thermal_sys: Registered thermal governor 'step_wise'700worker1 # [ 0.017811] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1701client # [ 0.017633] thermal_sys: Registered thermal governor 'power_allocator'702worker1 # [ 0.017821] cpuidle: using governor ladder703lb # [ 0.017583] thermal_sys: Registered thermal governor 'user_space'704worker1 # [ 0.017827] cpuidle: using governor menu705client # [ 0.017655] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1706lb # [ 0.017586] thermal_sys: Registered thermal governor 'power_allocator'707client # [ 0.017665] cpuidle: using governor ladder708worker1 # [ 0.018008] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.709client # [ 0.017670] cpuidle: using governor menu710lb # [ 0.017608] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1711worker1 # [ 0.018024] ASID allocator initialised with 65536 entries712lb # [ 0.017618] cpuidle: using governor ladder713client # [ 0.017850] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.714worker1 # [ 0.019222] Serial: AMBA PL011 UART driver715lb # [ 0.017623] cpuidle: using governor menu716client # [ 0.017865] ASID allocator initialised with 65536 entries717worker1 # [ 0.024370] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1718client # [ 0.019049] Serial: AMBA PL011 UART driver719lb # [ 0.017817] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.720worker2 # [ 0.180525] NET: Registered PF_INET protocol family721worker1 # [ 0.024487] printk: console [ttyAMA0] enabled722lb # [ 0.017832] ASID allocator initialised with 65536 entries723client # [ 0.024074] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1724worker2 # [ 0.180787] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)725lb # [ 0.018988] Serial: AMBA PL011 UART driver726client # [ 0.024233] printk: console [ttyAMA0] enabled727worker1 # [ 0.143056] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages728lb # [ 0.024173] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1729worker1 # [ 0.143074] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page730lb # [ 0.024286] printk: console [ttyAMA0] enabled731client # [ 0.143914] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages732worker1 # [ 0.143079] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages733client # [ 0.143931] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page734worker1 # [ 0.143084] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page735lb # [ 0.142459] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages736client # [ 0.143936] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages737worker1 # [ 0.143088] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages738lb # [ 0.142477] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page739client # [ 0.143940] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page740worker1 # [ 0.143092] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page741lb # [ 0.142482] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages742client # [ 0.143945] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages743worker1 # [ 0.143097] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages744lb # [ 0.142486] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page745client # [ 0.143949] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page746worker1 # [ 0.143101] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page747lb # [ 0.142491] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages748client # [ 0.143954] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages749lb # [ 0.142495] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page750client # [ 0.143958] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page751lb # [ 0.142499] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages752lb # [ 0.142503] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page753worker1 # [ 0.150218] fbcon: Taking over console754worker1 # [ 0.150234] ACPI: Interpreter disabled.755worker1 # [ 0.152072] iommu: Default domain type: Translated756client # [ 0.151179] fbcon: Taking over console757client # [ 0.151192] ACPI: Interpreter disabled.758worker1 # [ 0.152082] iommu: DMA domain TLB invalidation policy: strict mode759client # [ 0.153001] iommu: Default domain type: Translated760lb # [ 0.149672] fbcon: Taking over console761worker1 # [ 0.153747] SCSI subsystem initialized762lb # [ 0.149685] ACPI: Interpreter disabled.763client # [ 0.153010] iommu: DMA domain TLB invalidation policy: strict mode764lb # [ 0.151475] iommu: Default domain type: Translated765client # [ 0.154698] SCSI subsystem initialized766lb # [ 0.151485] iommu: DMA domain TLB invalidation policy: strict mode767lb # [ 0.153170] SCSI subsystem initialized768client # [ 0.155372] usbcore: registered new interface driver usbfs769client # [ 0.155403] usbcore: registered new interface driver hub770lb # [ 0.153845] usbcore: registered new interface driver usbfs771client # [ 0.155420] usbcore: registered new device driver usb772lb # [ 0.153882] usbcore: registered new interface driver hub773client # [ 0.155673] pps_core: LinuxPPS API ver. 1 registered774lb # [ 0.153901] usbcore: registered new device driver usb775client # [ 0.155679] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>776lb # [ 0.154177] pps_core: LinuxPPS API ver. 1 registered777client # [ 0.155696] PTP clock support registered778client # [ 0.155742] EDAC MC: Ver: 3.0.0779lb # [ 0.154183] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>780lb # [ 0.154193] PTP clock support registered781lb # [ 0.154249] EDAC MC: Ver: 3.0.0782worker1 # [ 0.161129] usbcore: registered new interface driver usbfs783worker1 # [ 0.161162] usbcore: registered new interface driver hub784worker1 # [ 0.161180] usbcore: registered new device driver usb785worker1 # [ 0.161436] pps_core: LinuxPPS API ver. 1 registered786worker1 # [ 0.161442] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>787worker1 # [ 0.161452] PTP clock support registered788worker1 # [ 0.161505] EDAC MC: Ver: 3.0.0789worker1 # [ 0.166110] scmi_core: SCMI protocol bus registered790worker1 # [ 0.167079] FPGA manager framework791client # [ 0.167206] scmi_core: SCMI protocol bus registered792worker1 # [ 0.168089] vgaarb: loaded793client # [ 0.168212] FPGA manager framework794lb # [ 0.165754] scmi_core: SCMI protocol bus registered795client # [ 0.169128] vgaarb: loaded796worker1 # [ 0.168720] clocksource: Switched to clocksource arch_sys_counter797lb # [ 0.166764] FPGA manager framework798worker1 # [ 0.169298] VFS: Disk quotas dquot_6.6.0799lb # [ 0.167682] vgaarb: loaded800worker1 # [ 0.169331] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)801client # [ 0.169744] clocksource: Switched to clocksource arch_sys_counter802client # [ 0.170315] VFS: Disk quotas dquot_6.6.0803worker1 # [ 0.171638] netfs: FS-Cache loaded804worker1 # [ 0.171753] pnp: PnP ACPI: disabled805lb # [ 0.168307] clocksource: Switched to clocksource arch_sys_counter806client # [ 0.170347] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807lb # [ 0.168899] VFS: Disk quotas dquot_6.6.0808client # [ 0.172715] netfs: FS-Cache loaded809client # [ 0.172827] pnp: PnP ACPI: disabled810lb # [ 0.168932] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)811lb # [ 0.171270] netfs: FS-Cache loaded812lb # [ 0.171381] pnp: PnP ACPI: disabled813worker2 # [ 0.209036] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)814worker2 # [ 0.209074] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)815worker2 # [ 0.209100] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)816worker1 # [ 0.177499] NET: Registered PF_INET protocol family817worker2 # [ 0.209174] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)818worker1 # [ 0.177758] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819worker2 # [ 0.209318] TCP: Hash tables configured (established 16384 bind 16384)820client # [ 0.178655] NET: Registered PF_INET protocol family821worker2 # [ 0.209391] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)822client # [ 0.178916] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)823lb # [ 0.177221] NET: Registered PF_INET protocol family824worker2 # [ 0.209424] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)825lb # [ 0.177490] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)826worker2 # [ 0.209460] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)827worker2 # [ 0.209544] NET: Registered PF_UNIX/PF_LOCAL protocol family828worker2 # [ 0.209585] NET: Registered PF_XDP protocol family829worker2 # [ 0.209602] PCI: CLS 0 bytes, default 64830worker2 # [ 0.209849] Trying to unpack rootfs image as initramfs...831worker2 # [ 0.227396] kvm [1]: HYP mode not available832client # [ 0.207890] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833client # [ 0.207933] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834lb # [ 0.206687] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)835client # [ 0.207961] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)836lb # [ 0.206730] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)837client # [ 0.208034] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)838worker1 # [ 0.207980] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)839lb # [ 0.206758] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)840client # [ 0.208176] TCP: Hash tables configured (established 16384 bind 16384)841worker1 # [ 0.208031] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)842lb # [ 0.206829] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)843client # [ 0.208253] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)844worker1 # [ 0.208061] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)845lb # [ 0.206973] TCP: Hash tables configured (established 16384 bind 16384)846client # [ 0.208289] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)847worker1 # [ 0.208143] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)848lb # [ 0.207061] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)849client # [ 0.208325] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)850worker1 # [ 0.208286] TCP: Hash tables configured (established 16384 bind 16384)851lb # [ 0.207095] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852client # [ 0.208425] NET: Registered PF_UNIX/PF_LOCAL protocol family853worker1 # [ 0.208366] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)854lb # [ 0.207131] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)855client # [ 0.208447] NET: Registered PF_XDP protocol family856client # [ 0.208464] PCI: CLS 0 bytes, default 64857lb # [ 0.207233] NET: Registered PF_UNIX/PF_LOCAL protocol family858worker1 # [ 0.208408] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)859client # [ 0.208699] Trying to unpack rootfs image as initramfs...860lb # [ 0.207271] NET: Registered PF_XDP protocol family861worker1 # [ 0.208443] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)862lb # [ 0.207289] PCI: CLS 0 bytes, default 64863worker1 # [ 0.208540] NET: Registered PF_UNIX/PF_LOCAL protocol family864lb # [ 0.207530] Trying to unpack rootfs image as initramfs...865worker1 # [ 0.208562] NET: Registered PF_XDP protocol family866worker1 # [ 0.208579] PCI: CLS 0 bytes, default 64867worker1 # [ 0.208807] Trying to unpack rootfs image as initramfs...868client # [ 0.223275] kvm [1]: HYP mode not available869lb # [ 0.221854] kvm [1]: HYP mode not available870worker1 # [ 0.226189] kvm [1]: HYP mode not available871worker2 # [ 0.311061] Initialise system trusted keyrings872worker2 # [ 0.317850] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.319224] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.319976] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.340567] Key type asymmetric registered876worker2 # [ 0.340586] Asymmetric key parser 'x509' registered877worker2 # [ 0.340666] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker2 # [ 0.345858] io scheduler mq-deadline registered879worker2 # [ 0.345873] io scheduler kyber registered880worker1 # [ 0.311437] Initialise system trusted keyrings881client # [ 0.314284] Initialise system trusted keyrings882worker2 # [ 0.353927] pl061_gpio 9030000.pl061: PL061 GPIO chip registered883client # [ 0.315027] workingset: timestamp_bits=42 max_order=19 bucket_order=0884lb # [ 0.312880] Initialise system trusted keyrings885client # [ 0.316283] squashfs: version 4.0 (2009/01/31) Phillip Lougher886lb # [ 0.313620] workingset: timestamp_bits=42 max_order=19 bucket_order=0887lb # [ 0.314872] squashfs: version 4.0 (2009/01/31) Phillip Lougher888client # [ 0.317054] 9p: Installing v9fs 9p2000 file system support889worker2 # [ 0.355292] ledtrig-cpu: registered to indicate activity on CPUs890worker2 # [ 0.355672] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:891lb # [ 0.315609] 9p: Installing v9fs 9p2000 file system support892worker2 # [ 0.355691] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000893worker2 # [ 0.355703] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000894worker1 # [ 0.320788] workingset: timestamp_bits=42 max_order=19 bucket_order=0895worker2 # [ 0.355712] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000896worker1 # [ 0.322214] squashfs: version 4.0 (2009/01/31) Phillip Lougher897worker2 # [ 0.355737] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits898worker2 # [ 0.355760] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]899worker2 # [ 0.355831] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00900worker2 # [ 0.355840] pci_bus 0000:00: root bus resource [bus 00-ff]901worker2 # [ 0.355846] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]902worker2 # [ 0.355851] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]903worker2 # [ 0.355856] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]904worker1 # [ 0.322979] 9p: Installing v9fs 9p2000 file system support905worker2 # [ 0.355916] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint906worker2 # [ 0.356364] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint907worker2 # [ 0.356606] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]908worker2 # [ 0.356624] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]909worker2 # [ 0.356654] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]910worker2 # [ 0.356669] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]911worker2 # [ 0.357133] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint912worker2 # [ 0.357324] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]913worker2 # [ 0.357342] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]914worker1 # [ 0.335640] Key type asymmetric registered915worker2 # [ 0.357372] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]916worker1 # [ 0.335649] Asymmetric key parser 'x509' registered917worker1 # [ 0.335696] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)918client # [ 0.337608] Key type asymmetric registered919client # [ 0.337628] Asymmetric key parser 'x509' registered920client # [ 0.337688] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)921lb # [ 0.339510] Key type asymmetric registered922lb # [ 0.339533] Asymmetric key parser 'x509' registered923lb # [ 0.339596] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)924worker1 # [ 0.344772] io scheduler mq-deadline registered925worker1 # [ 0.344789] io scheduler kyber registered926client # [ 0.345800] io scheduler mq-deadline registered927client # [ 0.345817] io scheduler kyber registered928worker2 # [ 0.381853] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint929worker2 # [ 0.382052] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]930worker2 # [ 0.382068] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]931worker2 # [ 0.382097] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]932worker2 # [ 0.382574] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint933worker2 # [ 0.382762] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]934worker2 # [ 0.382777] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]935client # [ 0.353884] pl061_gpio 9030000.pl061: PL061 GPIO chip registered936lb # [ 0.348371] io scheduler mq-deadline registered937worker2 # [ 0.382809] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]938lb # [ 0.348389] io scheduler kyber registered939worker2 # [ 0.383295] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint940worker2 # [ 0.383481] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]941lb # [ 0.353720] pl061_gpio 9030000.pl061: PL061 GPIO chip registered942worker1 # [ 0.356852] pl061_gpio 9030000.pl061: PL061 GPIO chip registered943worker2 # [ 0.383497] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]944client # [ 0.355221] ledtrig-cpu: registered to indicate activity on CPUs945worker2 # [ 0.383526] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]946client # [ 0.355579] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:947lb # [ 0.354984] ledtrig-cpu: registered to indicate activity on CPUs948worker2 # [ 0.383972] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint949client # [ 0.355596] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000950lb # [ 0.355344] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:951worker2 # [ 0.384157] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]952worker1 # [ 0.358197] ledtrig-cpu: registered to indicate activity on CPUs953worker2 # [ 0.384173] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]954lb # [ 0.355361] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000955client # [ 0.355634] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000956worker1 # [ 0.358602] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:957worker2 # [ 0.384202] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]958client # [ 0.355643] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000959lb # [ 0.355382] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000960worker1 # [ 0.358619] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961worker2 # [ 0.384647] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint962client # [ 0.355669] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits963lb # [ 0.355390] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964worker2 # [ 0.384831] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]965worker1 # [ 0.358632] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000966worker2 # [ 0.384847] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]967lb # [ 0.355410] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits968client # [ 0.355692] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]969worker1 # [ 0.358640] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000970worker2 # [ 0.384876] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]971client # [ 0.355761] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972lb # [ 0.355431] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]973client # [ 0.355770] pci_bus 0000:00: root bus resource [bus 00-ff]974worker2 # [ 0.384892] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]975worker1 # [ 0.358666] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits976lb # [ 0.355511] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00977client # [ 0.355776] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]978worker2 # [ 0.385383] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint979lb # [ 0.355521] pci_bus 0000:00: root bus resource [bus 00-ff]980worker1 # [ 0.358688] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]981client # [ 0.355781] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]982worker2 # [ 0.385570] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]983lb # [ 0.355527] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]984worker1 # [ 0.358776] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00985worker2 # [ 0.385600] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986client # [ 0.355786] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]987lb # [ 0.355532] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]988worker1 # [ 0.358785] pci_bus 0000:00: root bus resource [bus 00-ff]989worker1 # [ 0.358791] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]990worker2 # [ 0.386060] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint991lb # [ 0.355537] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]992client # [ 0.355842] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint993worker2 # [ 0.386249] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]994worker1 # [ 0.358797] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]995client # [ 0.356283] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint996lb # [ 0.355635] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint997worker2 # [ 0.386279] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998client # [ 0.356476] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]999worker1 # [ 0.358802] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1000lb # [ 0.356062] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1001client # [ 0.356493] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1002worker2 # [ 0.386675] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1003lb # [ 0.356258] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1004worker1 # [ 0.358860] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1005client # [ 0.356522] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker2 # [ 0.386865] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1007lb # [ 0.356274] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1008worker1 # [ 0.359305] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1009client # [ 0.356538] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1010lb # [ 0.356303] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker1 # [ 0.359499] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1012worker2 # [ 0.387112] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1013worker2 # [ 0.387404] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1014worker1 # [ 0.359515] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1015client # [ 0.357000] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1016client # [ 0.357182] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1017worker2 # [ 0.387421] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1018worker1 # [ 0.359543] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019client # [ 0.357198] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1020worker2 # [ 0.387450] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021worker1 # [ 0.359559] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1022client # [ 0.357228] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1023worker2 # [ 0.387891] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1024worker1 # [ 0.360016] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1025worker2 # [ 0.388072] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1026worker1 # [ 0.360197] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1027client # [ 0.357684] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1028worker2 # [ 0.388087] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1029worker1 # [ 0.360211] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker1 # [ 0.360240] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031worker2 # [ 0.388116] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032worker2 # [ 0.388682] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1033worker1 # [ 0.360690] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1034worker2 # [ 0.388693] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1035worker1 # [ 0.379906] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1036worker2 # [ 0.388699] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1037worker1 # [ 0.379925] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1038worker2 # [ 0.388743] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1039worker1 # [ 0.379954] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040worker2 # [ 0.388789] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1041worker1 # [ 0.380410] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1042worker2 # [ 0.388835] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1043worker1 # [ 0.380590] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1044worker1 # [ 0.380605] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker2 # [ 0.388882] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1046worker1 # [ 0.380637] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047worker2 # [ 0.388927] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1048worker2 # [ 0.388973] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1049lb # [ 0.376377] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1050worker2 # [ 0.389020] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1051lb # [ 0.376861] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1052worker2 # [ 0.389066] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1053lb # [ 0.377047] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1054worker1 # [ 0.386370] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1055lb # [ 0.377063] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1056worker2 # [ 0.389113] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1057worker1 # [ 0.386554] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1058client # [ 0.381912] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1059lb # [ 0.377093] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060worker1 # [ 0.386569] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1061worker2 # [ 0.389177] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1062client # [ 0.381938] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1063worker1 # [ 0.386598] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064lb # [ 0.377543] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1065worker2 # [ 0.389223] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1066client # [ 0.381968] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067lb # [ 0.377723] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1068worker1 # [ 0.387045] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1069worker2 # [ 0.389245] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1070client # [ 0.382449] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1071worker1 # [ 0.387226] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1072lb # [ 0.377739] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1073worker2 # [ 0.389266] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1074client # [ 0.382633] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1075worker1 # [ 0.387241] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1076lb # [ 0.377768] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077worker2 # [ 0.389287] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1078client # [ 0.382649] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1079worker1 # [ 0.387276] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1080lb # [ 0.378216] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1081worker2 # [ 0.389311] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1082client # [ 0.382681] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083lb # [ 0.378395] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1084worker1 # [ 0.387764] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1085worker2 # [ 0.389333] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1086client # [ 0.383145] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087worker1 # [ 0.387946] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1088lb # [ 0.378411] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1089client # [ 0.383325] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1090worker2 # [ 0.389354] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1091worker1 # [ 0.387962] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1092lb # [ 0.378442] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093client # [ 0.383341] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1094worker2 # [ 0.389376] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1095worker1 # [ 0.387990] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096lb # [ 0.378889] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1097client # [ 0.383371] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker2 # [ 0.389398] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1099worker1 # [ 0.388006] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1100lb # [ 0.379073] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1101worker2 # [ 0.389420] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1102client # [ 0.383817] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1103lb # [ 0.379089] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1104worker1 # [ 0.388458] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1105client # [ 0.383997] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1106worker2 # [ 0.389442] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1107worker1 # [ 0.388642] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1108lb # [ 0.379118] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109client # [ 0.384013] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker2 # [ 0.389464] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1111worker1 # [ 0.388671] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112lb # [ 0.379562] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1113client # [ 0.384042] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114worker2 # [ 0.389485] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1115lb # [ 0.379742] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1116worker2 # [ 0.389506] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1117client # [ 0.384501] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1118lb # [ 0.379757] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1119client # [ 0.384684] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1120worker2 # [ 0.389527] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1121lb # [ 0.379786] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker2 # [ 0.389548] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1123client # [ 0.384700] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1124lb # [ 0.380227] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1125worker2 # [ 0.389569] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1126client # [ 0.384729] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127lb # [ 0.380431] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1128worker2 # [ 0.389590] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1129client # [ 0.384746] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1130lb # [ 0.380447] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1131worker2 # [ 0.389611] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1132client # [ 0.385198] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1133lb # [ 0.380476] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134worker2 # [ 0.389633] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1135client # [ 0.385383] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1136lb # [ 0.380491] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1137worker2 # [ 0.389654] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1138client # [ 0.385413] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139worker2 # [ 0.389680] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1140lb # [ 0.380973] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1141worker2 # [ 0.389689] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1142client # [ 0.385876] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1143lb # [ 0.381156] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.386064] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker2 # [ 0.389694] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1146lb # [ 0.381186] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147client # [ 0.386094] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148lb # [ 0.381637] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1149client # [ 0.386489] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1150lb # [ 0.381822] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1151client # [ 0.386669] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1152lb # [ 0.381851] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.386911] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1154lb # [ 0.382242] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1155client # [ 0.387150] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1156lb # [ 0.382420] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1157client # [ 0.387167] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1158lb # [ 0.382667] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1159client # [ 0.387197] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160lb # [ 0.382917] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1161client # [ 0.387651] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1162lb # [ 0.382934] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1163client # [ 0.387834] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1164worker1 # [ 0.409177] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1165lb # [ 0.382964] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166client # [ 0.387851] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1167worker2 # [ 0.454915] pci 0000:00:0a.0: enabling device (0000 -> 0002)1168worker1 # [ 0.409375] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1169lb # [ 0.383426] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1170client # [ 0.387880] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171worker1 # [ 0.409405] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172lb # [ 0.383606] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1173client # [ 0.388448] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1174worker1 # [ 0.409799] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1175lb # [ 0.383621] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1176client # [ 0.388459] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1177worker1 # [ 0.409978] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1178lb # [ 0.383650] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179client # [ 0.388465] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1180worker1 # [ 0.410220] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1181lb # [ 0.384209] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1182worker1 # [ 0.410465] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1183client # [ 0.388508] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1184lb # [ 0.384220] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1185worker1 # [ 0.410482] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1186client # [ 0.388554] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1187lb # [ 0.384226] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1188worker1 # [ 0.410511] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189client # [ 0.388600] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1190lb # [ 0.384268] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1191worker1 # [ 0.410975] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1192client # [ 0.388644] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1193worker1 # [ 0.411156] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1194worker1 # [ 0.411171] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1195client # [ 0.388690] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1196worker1 # [ 0.411200] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197client # [ 0.388735] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1198worker1 # [ 0.411772] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1199client # [ 0.388781] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1200worker1 # [ 0.411783] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1201client # [ 0.388827] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1202worker1 # [ 0.411789] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1203client # [ 0.388873] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1204worker1 # [ 0.411832] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1205client # [ 0.388938] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1206worker1 # [ 0.411877] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1207client # [ 0.388983] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1208worker1 # [ 0.411923] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1209client # [ 0.389005] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1210lb # [ 0.424357] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1211worker1 # [ 0.411969] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1212client # [ 0.389026] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1213lb # [ 0.424432] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214client # [ 0.389047] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1215worker1 # [ 0.412014] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1216lb # [ 0.424479] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217client # [ 0.389071] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1218worker1 # [ 0.412060] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1219lb # [ 0.424526] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1220client # [ 0.389092] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1221worker1 # [ 0.412107] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1222lb # [ 0.424572] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1223client # [ 0.389114] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1224worker1 # [ 0.412153] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1225client # [ 0.389136] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1226lb # [ 0.424620] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1227worker1 # [ 0.412198] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1228client # [ 0.389158] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1229lb # [ 0.424667] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1230client # [ 0.389180] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1231worker1 # [ 0.412263] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1232lb # [ 0.424715] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1233client # [ 0.389202] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1234worker1 # [ 0.412309] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1235lb # [ 0.424814] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1236client # [ 0.389224] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1237worker1 # [ 0.412329] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1238lb # [ 0.424862] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1239client # [ 0.389246] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1240worker1 # [ 0.412350] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1241lb # [ 0.424884] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1242client # [ 0.389267] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1243worker1 # [ 0.412371] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1244client # [ 0.389288] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1245lb # [ 0.424906] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1246worker1 # [ 0.412395] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1247client # [ 0.389309] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1248lb # [ 0.424927] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1249worker2 # [ 0.480009] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1250worker1 # [ 0.412416] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1251client # [ 0.389331] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1252lb # [ 0.424951] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1253worker1 # [ 0.412437] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1254client # [ 0.389352] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1255lb # [ 0.424973] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1256worker1 # [ 0.412459] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1257client # [ 0.389373] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1258lb # [ 0.424995] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1259client # [ 0.389395] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1260worker1 # [ 0.412480] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1261lb # [ 0.425017] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1262client # [ 0.389416] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1263worker1 # [ 0.412501] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1264lb # [ 0.425039] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1265client # [ 0.389443] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1266worker1 # [ 0.412523] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1267lb # [ 0.425061] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1268client # [ 0.389452] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1269worker1 # [ 0.412546] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1270lb # [ 0.425084] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1271client # [ 0.389457] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1272worker1 # [ 0.412566] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1273lb # [ 0.425107] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1274worker1 # [ 0.412587] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1275lb # [ 0.425128] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1276worker1 # [ 0.412608] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1277lb # [ 0.425149] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1278worker1 # [ 0.412628] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1279lb # [ 0.425170] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1280worker1 # [ 0.412649] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1281worker2 # [ 0.486010] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1282lb # [ 0.425191] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1283worker1 # [ 0.412670] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1284worker2 # [ 0.488107] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1285lb # [ 0.425212] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1286worker1 # [ 0.412690] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1287lb # [ 0.425234] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1288worker1 # [ 0.412711] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1289lb # [ 0.425255] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1290worker1 # [ 0.412750] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1291lb # [ 0.425277] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1292worker1 # [ 0.412775] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1293lb # [ 0.425299] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1294worker1 # [ 0.412785] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1295lb # [ 0.425326] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1296worker1 # [ 0.412789] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1297lb # [ 0.425336] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1298client # [ 0.454650] pci 0000:00:0a.0: enabling device (0000 -> 0002)1299worker1 # [ 0.413942] pci 0000:00:0a.0: enabling device (0000 -> 0002)1300lb # [ 0.425341] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1301lb # [ 0.426497] pci 0000:00:0a.0: enabling device (0000 -> 0002)1302worker2 # [ 0.498003] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1303worker2 # [ 0.499975] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1304worker2 # [ 0.509903] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1305worker2 # [ 0.512704] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1306worker2 # [ 0.514743] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1307worker2 # [ 0.516755] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1308worker1 # [ 0.479129] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1309client # [ 0.479766] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310lb # [ 0.478975] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1311client # [ 0.482658] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1312client # [ 0.484815] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1313worker1 # [ 0.485433] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1314worker1 # [ 0.487460] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker2 # [ 0.526867] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1316worker2 # [ 0.529317] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1317lb # [ 0.488941] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1318client # [ 0.495083] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1319client # [ 0.496981] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1320lb # [ 0.493303] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1321lb # [ 0.495176] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1322worker1 # [ 0.497731] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1323worker1 # [ 0.499738] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1324worker2 # [ 0.546803] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1325worker2 # [ 0.549346] msm_serial: driver initialized1326client # [ 0.508379] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1327worker2 # [ 0.549507] SuperH (H)SCI(F) driver initialized1328worker2 # [ 0.549561] STM32 USART driver initialized1329lb # [ 0.506190] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1330lb # [ 0.508247] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1331worker1 # [ 0.511023] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1332client # [ 0.511562] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1333client # [ 0.513529] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1334worker1 # [ 0.513953] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1335lb # [ 0.518138] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336worker1 # [ 0.515948] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337worker1 # [ 0.518000] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1338worker1 # [ 0.519905] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339client # [ 0.523523] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340lb # [ 0.521489] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1341client # [ 0.525397] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1342lb # [ 0.523601] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343client # [ 0.529419] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1344worker1 # [ 0.534250] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1345lb # [ 0.533795] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1346lb # [ 0.536124] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347worker2 # [ 0.583077] loop: module loaded1348worker2 # [ 0.583287] virtio_blk virtio5: 1/0/0 default/read/poll queues1349worker2 # [ 0.584070] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1350client # [ 0.546937] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1351worker1 # [ 0.551477] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352worker2 # [ 0.590366] megasas: 07.734.00.00-rc11353worker2 # [ 0.591057] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1354worker2 # [ 0.593219] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001355worker1 # [ 0.553992] msm_serial: driver initialized1356worker2 # [ 0.593243] Intel/Sharp Extended Query Table at 0x00311357worker1 # [ 0.554122] SuperH (H)SCI(F) driver initialized1358worker1 # [ 0.554173] STM32 USART driver initialized1359client # [ 0.555027] msm_serial: driver initialized1360client # [ 0.555199] SuperH (H)SCI(F) driver initialized1361client # [ 0.555252] STM32 USART driver initialized1362lb # [ 0.557033] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363lb # [ 0.559607] msm_serial: driver initialized1364lb # [ 0.559764] SuperH (H)SCI(F) driver initialized1365lb # [ 0.559819] STM32 USART driver initialized1366worker2 # [ 0.602771] Using buffer write method1367worker2 # [ 0.602865] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1368worker2 # [ 0.605619] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001369worker2 # [ 0.605658] Intel/Sharp Extended Query Table at 0x00311370worker2 # [ 0.618848] Using buffer write method1371worker2 # [ 0.618896] Concatenating MTD devices:1372worker2 # [ 0.618901] (0): "0.flash"1373worker2 # [ 0.618905] (1): "0.flash"1374worker2 # [ 0.618908] into device "0.flash"1375worker1 # [ 0.583498] loop: module loaded1376worker1 # [ 0.583670] virtio_blk virtio5: 1/0/0 default/read/poll queues1377client # [ 0.584465] loop: module loaded1378worker1 # [ 0.584366] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1379client # [ 0.584633] virtio_blk virtio5: 1/0/0 default/read/poll queues1380client # [ 0.585301] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1381client # [ 0.590307] megasas: 07.734.00.00-rc11382lb # [ 0.590021] loop: module loaded1383lb # [ 0.590192] virtio_blk virtio5: 1/0/0 default/read/poll queues1384client # [ 0.590949] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1385lb # [ 0.590901] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386client # [ 0.592763] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387client # [ 0.592786] Intel/Sharp Extended Query Table at 0x00311388lb # [ 0.596932] megasas: 07.734.00.00-rc11389lb # [ 0.597677] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1390worker1 # [ 0.597334] megasas: 07.734.00.00-rc11391lb # [ 0.599842] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001392worker1 # [ 0.598077] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1393client # [ 0.602440] Using buffer write method1394lb # [ 0.599865] Intel/Sharp Extended Query Table at 0x00311395worker1 # [ 0.600233] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001396client # [ 0.602506] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1397worker1 # [ 0.600259] Intel/Sharp Extended Query Table at 0x00311398worker1 # [ 0.601881] Using buffer write method1399client # [ 0.603995] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400client # [ 0.604026] Intel/Sharp Extended Query Table at 0x00311401worker1 # [ 0.601951] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1402client # [ 0.605690] Using buffer write method1403client # [ 0.605714] Concatenating MTD devices:1404client # [ 0.605718] (0): "0.flash"1405client # [ 0.605722] (1): "0.flash"1406client # [ 0.605725] into device "0.flash"1407worker1 # [ 0.603604] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408worker1 # [ 0.603634] Intel/Sharp Extended Query Table at 0x00311409worker1 # [ 0.612671] Using buffer write method1410worker1 # [ 0.612701] Concatenating MTD devices:1411worker1 # [ 0.612705] (0): "0.flash"1412worker1 # [ 0.612709] (1): "0.flash"1413worker1 # [ 0.612712] into device "0.flash"1414lb # [ 0.609315] Using buffer write method1415lb # [ 0.609389] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416lb # [ 0.612247] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417lb # [ 0.612276] Intel/Sharp Extended Query Table at 0x00311418lb # [ 0.625473] Using buffer write method1419lb # [ 0.625507] Concatenating MTD devices:1420lb # [ 0.625511] (0): "0.flash"1421lb # [ 0.625515] (1): "0.flash"1422lb # [ 0.625519] into device "0.flash"1423worker2 # [ 0.845453] Freeing initrd memory: 26148K1424worker2 # [ 0.851497] tun: Universal TUN/TAP device driver, 1.61425worker2 # [ 0.855093] thunder_xcv, ver 1.01426worker2 # [ 0.855152] thunder_bgx, ver 1.01427worker2 # [ 0.855173] nicpf, ver 1.01428worker2 # [ 0.855708] e1000: Intel(R) PRO/1000 Network Driver1429worker2 # [ 0.855715] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker2 # [ 0.855739] e1000e: Intel(R) PRO/1000 Network Driver1431worker2 # [ 0.855748] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker2 # [ 0.855776] igb: Intel(R) Gigabit Ethernet Network Driver1433worker2 # [ 0.855782] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker2 # [ 0.855804] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435worker2 # [ 0.855809] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436worker2 # [ 0.855940] sky2: driver version 1.301437worker2 # [ 0.857474] usbcore: registered new interface driver usb-storage1438worker2 # [ 0.857630] usbcore: registered new interface driver usbserial_generic1439worker2 # [ 0.857645] usbserial: USB Serial support registered for generic1440worker2 # [ 0.858498] ehci-pci 0000:00:0a.0: EHCI Host Controller1441worker2 # [ 0.858523] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11442worker2 # [ 0.858787] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001443worker2 # [ 0.870313] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001444worker2 # [ 0.870639] hub 1-0:1.0: USB hub found1445worker2 # [ 0.870665] hub 1-0:1.0: 6 ports detected1446worker2 # [ 0.873107] hv_vmbus: registering driver hyperv_keyboard1447worker2 # [ 0.874683] rtc-pl031 9010000.pl031: registered as rtc01448worker2 # [ 0.874708] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T15:07:20 UTC (1790003240)1449worker2 # [ 0.875007] i2c_dev: i2c /dev entries driver1450worker1 # [ 0.837169] Freeing initrd memory: 26148K1451client # [ 0.838756] Freeing initrd memory: 26160K1452worker2 # [ 0.879928] sdhci: Secure Digital Host Controller Interface driver1453worker2 # [ 0.879937] sdhci: Copyright(c) Pierre Ossman1454worker2 # [ 0.880199] Synopsys Designware Multimedia Card Interface Driver1455worker1 # [ 0.843231] tun: Universal TUN/TAP device driver, 1.61456worker2 # [ 0.880568] sdhci-pltfm: SDHCI platform and OF driver helper1457client # [ 0.844654] tun: Universal TUN/TAP device driver, 1.61458worker2 # [ 0.884912] hid: raw HID events driver (C) Jiri Kosina1459worker2 # [ 0.885162] usbcore: registered new interface driver usbhid1460worker2 # [ 0.885168] usbhid: USB HID core driver1461worker1 # [ 0.847063] thunder_xcv, ver 1.01462worker1 # [ 0.847109] thunder_bgx, ver 1.01463worker1 # [ 0.847133] nicpf, ver 1.01464client # [ 0.848207] thunder_xcv, ver 1.01465worker1 # [ 0.847692] e1000: Intel(R) PRO/1000 Network Driver1466client # [ 0.848248] thunder_bgx, ver 1.01467worker2 # [ 0.888034] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1468client # [ 0.848270] nicpf, ver 1.01469worker1 # [ 0.847700] e1000: Copyright (c) 1999-2006 Intel Corporation.1470lb # [ 0.847796] Freeing initrd memory: 26140K1471worker1 # [ 0.847729] e1000e: Intel(R) PRO/1000 Network Driver1472client # [ 0.848804] e1000: Intel(R) PRO/1000 Network Driver1473worker2 # [ 0.889540] drop_monitor: Initializing network drop monitor service1474worker1 # [ 0.847738] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1475worker2 # [ 0.889676] NET: Registered PF_INET6 protocol family1476client # [ 0.848811] e1000: Copyright (c) 1999-2006 Intel Corporation.1477client # [ 0.848834] e1000e: Intel(R) PRO/1000 Network Driver1478worker1 # [ 0.847768] igb: Intel(R) Gigabit Ethernet Network Driver1479worker2 # [ 0.892643] Segment Routing with IPv61480client # [ 0.848841] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1481worker1 # [ 0.847773] igb: Copyright (c) 2007-2014 Intel Corporation.1482worker2 # [ 0.892673] In-situ OAM (IOAM) with IPv61483client # [ 0.848871] igb: Intel(R) Gigabit Ethernet Network Driver1484worker2 # [ 0.892703] NET: Registered PF_PACKET protocol family1485worker1 # [ 0.847798] igbvf: Intel(R) Gigabit Virtual Function Network Driver1486client # [ 0.848877] igb: Copyright (c) 2007-2014 Intel Corporation.1487worker1 # [ 0.847804] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1488worker1 # [ 0.847937] sky2: driver version 1.301489client # [ 0.848899] igbvf: Intel(R) Gigabit Virtual Function Network Driver1490client # [ 0.848905] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1491client # [ 0.849034] sky2: driver version 1.301492lb # [ 0.853718] tun: Universal TUN/TAP device driver, 1.61493worker1 # [ 0.856418] usbcore: registered new interface driver usb-storage1494worker1 # [ 0.856501] usbcore: registered new interface driver usbserial_generic1495worker2 # [ 0.894480] 9pnet: Installing 9P2000 support1496client # [ 0.857540] usbcore: registered new interface driver usb-storage1497worker1 # [ 0.856517] usbserial: USB Serial support registered for generic1498worker2 # [ 0.896543] Key type dns_resolver registered1499client # [ 0.857591] usbcore: registered new interface driver usbserial_generic1500client # [ 0.857605] usbserial: USB Serial support registered for generic1501worker1 # [ 0.859523] ehci-pci 0000:00:0a.0: EHCI Host Controller1502worker1 # [ 0.859555] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11503client # [ 0.860595] ehci-pci 0000:00:0a.0: EHCI Host Controller1504worker1 # [ 0.859757] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001505client # [ 0.860621] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11506client # [ 0.860820] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001507worker1 # [ 0.862607] hv_vmbus: registering driver hyperv_keyboard1508lb # [ 0.857353] thunder_xcv, ver 1.01509client # [ 0.863630] hv_vmbus: registering driver hyperv_keyboard1510lb # [ 0.857396] thunder_bgx, ver 1.01511worker1 # [ 0.864122] rtc-pl031 9010000.pl031: registered as rtc01512lb # [ 0.857419] nicpf, ver 1.01513lb # [ 0.857968] e1000: Intel(R) PRO/1000 Network Driver1514worker1 # [ 0.864155] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T15:07:20 UTC (1790003240)1515worker2 # [ 0.903332] registered taskstats version 11516client # [ 0.865106] rtc-pl031 9010000.pl031: registered as rtc01517lb # [ 0.857975] e1000: Copyright (c) 1999-2006 Intel Corporation.1518worker1 # [ 0.864462] i2c_dev: i2c /dev entries driver1519worker2 # [ 0.903476] Loading compiled-in X.509 certificates1520lb # [ 0.857999] e1000e: Intel(R) PRO/1000 Network Driver1521client # [ 0.865134] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T15:07:20 UTC (1790003240)1522lb # [ 0.858007] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1523client # [ 0.865419] i2c_dev: i2c /dev entries driver1524lb # [ 0.858036] igb: Intel(R) Gigabit Ethernet Network Driver1525lb # [ 0.858041] igb: Copyright (c) 2007-2014 Intel Corporation.1526lb # [ 0.858065] igbvf: Intel(R) Gigabit Virtual Function Network Driver1527lb # [ 0.858070] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1528lb # [ 0.858205] sky2: driver version 1.301529lb # [ 0.859793] usbcore: registered new interface driver usb-storage1530lb # [ 0.859898] usbcore: registered new interface driver usbserial_generic1531worker2 # [ 0.911898] Demotion targets for Node 0: null1532lb # [ 0.859914] usbserial: USB Serial support registered for generic1533worker1 # [ 0.869274] sdhci: Secure Digital Host Controller Interface driver1534worker2 # [ 0.912002] Key type .fscrypt registered1535worker1 # [ 0.869285] sdhci: Copyright(c) Pierre Ossman1536lb # [ 0.861084] ehci-pci 0000:00:0a.0: EHCI Host Controller1537worker2 # [ 0.912008] Key type fscrypt-provisioning registered1538worker1 # [ 0.869544] Synopsys Designware Multimedia Card Interface Driver1539client # [ 0.870248] sdhci: Secure Digital Host Controller Interface driver1540worker2 # [ 0.912097] ima: No TPM chip found, activating TPM-bypass!1541lb # [ 0.861110] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11542client # [ 0.870257] sdhci: Copyright(c) Pierre Ossman1543worker2 # [ 0.912116] ima: Allocated hash algorithm: sha11544worker1 # [ 0.869910] sdhci-pltfm: SDHCI platform and OF driver helper1545lb # [ 0.861398] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001546worker2 # [ 0.912137] ima: No architecture policies found1547worker1 # [ 0.871581] hid: raw HID events driver (C) Jiri Kosina1548client # [ 0.870518] Synopsys Designware Multimedia Card Interface Driver1549worker1 # [ 0.871852] usbcore: registered new interface driver usbhid1550client # [ 0.870885] sdhci-pltfm: SDHCI platform and OF driver helper1551lb # [ 0.872651] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001552worker1 # [ 0.871859] usbhid: USB HID core driver1553worker2 # [ 0.916021] input: gpio-keys as /devices/platform/gpio-keys/input/input01554client # [ 0.872583] hid: raw HID events driver (C) Jiri Kosina1555lb # [ 0.872975] hub 1-0:1.0: USB hub found1556client # [ 0.872817] usbcore: registered new interface driver usbhid1557lb # [ 0.873022] hub 1-0:1.0: 6 ports detected1558client # [ 0.872823] usbhid: USB HID core driver1559worker1 # [ 0.877038] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001560worker1 # [ 0.877360] hub 1-0:1.0: USB hub found1561worker1 # [ 0.877381] hub 1-0:1.0: 6 ports detected1562lb # [ 0.875211] hv_vmbus: registering driver hyperv_keyboard1563client # [ 0.878094] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001564client # [ 0.878404] hub 1-0:1.0: USB hub found1565client # [ 0.878424] hub 1-0:1.0: 6 ports detected1566lb # [ 0.876771] rtc-pl031 9010000.pl031: registered as rtc01567worker1 # [ 0.879568] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1568lb # [ 0.876796] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T15:07:20 UTC (1790003240)1569lb # [ 0.877101] i2c_dev: i2c /dev entries driver1570client # [ 0.880522] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1571worker1 # [ 0.882191] drop_monitor: Initializing network drop monitor service1572worker1 # [ 0.882337] NET: Registered PF_INET6 protocol family1573client # [ 0.883140] drop_monitor: Initializing network drop monitor service1574worker1 # [ 0.884308] Segment Routing with IPv61575client # [ 0.883288] NET: Registered PF_INET6 protocol family1576worker1 # [ 0.884328] In-situ OAM (IOAM) with IPv61577worker1 # [ 0.884371] NET: Registered PF_PACKET protocol family1578client # [ 0.885216] Segment Routing with IPv61579client # [ 0.885238] In-situ OAM (IOAM) with IPv61580client # [ 0.885267] NET: Registered PF_PACKET protocol family1581lb # [ 0.882049] sdhci: Secure Digital Host Controller Interface driver1582lb # [ 0.882058] sdhci: Copyright(c) Pierre Ossman1583lb # [ 0.882331] Synopsys Designware Multimedia Card Interface Driver1584worker1 # [ 0.886044] 9pnet: Installing 9P2000 support1585worker1 # [ 0.888221] Key type dns_resolver registered1586lb # [ 0.882712] sdhci-pltfm: SDHCI platform and OF driver helper1587client # [ 0.886901] 9pnet: Installing 9P2000 support1588client # [ 0.889022] Key type dns_resolver registered1589lb # [ 0.887146] hid: raw HID events driver (C) Jiri Kosina1590lb # [ 0.887405] usbcore: registered new interface driver usbhid1591lb # [ 0.887412] usbhid: USB HID core driver1592worker2 # [ 0.933417] clk: Disabling unused clocks1593worker2 # [ 0.933446] PM: genpd: Disabling unused power domains1594lb # [ 0.890283] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1595lb # [ 0.891805] drop_monitor: Initializing network drop monitor service1596worker1 # [ 0.895006] registered taskstats version 11597lb # [ 0.891982] NET: Registered PF_INET6 protocol family1598worker1 # [ 0.895156] Loading compiled-in X.509 certificates1599lb # [ 0.895038] Segment Routing with IPv61600client # [ 0.895865] registered taskstats version 11601worker2 # [ 0.937618] Freeing unused kernel memory: 4736K1602lb # [ 0.895057] In-situ OAM (IOAM) with IPv61603client # [ 0.896027] Loading compiled-in X.509 certificates1604worker2 # [ 0.938390] Run /init as init process1605lb # [ 0.895087] NET: Registered PF_PACKET protocol family1606lb # [ 0.896897] 9pnet: Installing 9P2000 support1607lb # [ 0.898973] Key type dns_resolver registered1608worker1 # [ 0.903996] Demotion targets for Node 0: null1609worker1 # [ 0.904124] Key type .fscrypt registered1610client # [ 0.904799] Demotion targets for Node 0: null1611worker1 # [ 0.904131] Key type fscrypt-provisioning registered1612client # [ 0.904905] Key type .fscrypt registered1613worker1 # [ 0.904230] ima: No TPM chip found, activating TPM-bypass!1614client # [ 0.904912] Key type fscrypt-provisioning registered1615worker1 # [ 0.904249] ima: Allocated hash algorithm: sha11616client # [ 0.905000] ima: No TPM chip found, activating TPM-bypass!1617worker1 # [ 0.904272] ima: No architecture policies found1618client # [ 0.905019] ima: Allocated hash algorithm: sha11619client # [ 0.905040] ima: No architecture policies found1620worker1 # [ 0.908390] input: gpio-keys as /devices/platform/gpio-keys/input/input01621client # [ 0.908910] input: gpio-keys as /devices/platform/gpio-keys/input/input01622lb # [ 0.905767] registered taskstats version 11623lb # [ 0.905909] Loading compiled-in X.509 certificates1624worker2 # [ 0.952038] systemd[1]: Successfully made /usr/ read-only.1625lb # [ 0.914236] Demotion targets for Node 0: null1626lb # [ 0.914342] Key type .fscrypt registered1627lb # [ 0.914349] Key type fscrypt-provisioning registered1628lb # [ 0.914441] ima: No TPM chip found, activating TPM-bypass!1629lb # [ 0.914460] ima: Allocated hash algorithm: sha11630lb # [ 0.914481] ima: No architecture policies found1631lb # [ 0.918512] input: gpio-keys as /devices/platform/gpio-keys/input/input01632worker1 # [ 0.926322] clk: Disabling unused clocks1633client # [ 0.926314] clk: Disabling unused clocks1634worker1 # [ 0.926360] PM: genpd: Disabling unused power domains1635client # [ 0.926341] PM: genpd: Disabling unused power domains1636worker1 # [ 0.930614] Freeing unused kernel memory: 4736K1637client # [ 0.930550] Freeing unused kernel memory: 4736K1638worker1 # [ 0.930830] Run /init as init process1639client # [ 0.930767] Run /init as init process1640lb # [ 0.936302] clk: Disabling unused clocks1641lb # [ 0.936885] PM: genpd: Disabling unused power domains1642lb # [ 0.940545] Freeing unused kernel memory: 4736K1643lb # [ 0.940753] Run /init as init process1644worker1 # [ 0.946920] systemd[1]: Successfully made /usr/ read-only.1645client # [ 0.947958] systemd[1]: Successfully made /usr/ read-only.1646lb # [ 0.954546] systemd[1]: Successfully made /usr/ read-only.1647worker2 # [ 1.117869] usb 1-1: new high-speed USB device number 2 using ehci-pci1648lb # [ 1.120399] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.124796] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.125822] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker2 # [ 1.270170] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.286806] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1653worker2 # [ 1.298717] systemd[1]: Detected virtualization qemu.1654worker2 # [ 1.300728] systemd[1]: Detected architecture arm64.1655worker2 # [ 1.302639] systemd[1]: Running in initrd.1656worker2 # [ 1.305148] systemd[1]: Initializing machine ID from random generator.1657worker2 # [ 1.308037] systemd[1]: Hostname set to <worker2>.1658lb # [ 1.272559] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11659worker1 # [ 1.277036] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660client # [ 1.278083] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661worker1 # [ 1.282942] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662client # [ 1.283890] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1663worker1 # [ 1.283001] systemd[1]: Detected virtualization qemu.1664client # [ 1.283948] systemd[1]: Detected virtualization qemu.1665worker1 # [ 1.283074] systemd[1]: Detected architecture arm64.1666client # [ 1.284020] systemd[1]: Detected architecture arm64.1667worker1 # [ 1.283099] systemd[1]: Running in initrd.1668client # [ 1.284044] systemd[1]: Running in initrd.1669lb # [ 1.289296] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670worker1 # [ 1.284027] systemd[1]: Initializing machine ID from random generator.1671client # [ 1.302823] systemd[1]: Initializing machine ID from random generator.1672lb # [ 1.301313] systemd[1]: Detected virtualization qemu.1673worker1 # [ 1.284395] systemd[1]: Hostname set to <worker1>.1674client # [ 1.305589] systemd[1]: Hostname set to <client>.1675lb # [ 1.303295] systemd[1]: Detected architecture arm64.1676lb # [ 1.305245] systemd[1]: Running in initrd.1677lb # [ 1.307744] systemd[1]: Initializing machine ID from random generator.1678lb # [ 1.310577] systemd[1]: Hostname set to <lb>.1679worker2 # [ 1.358058] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680worker1 # [ 1.361001] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681client # [ 1.365989] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.364577] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker2 # [ 1.477893] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker1 # [ 1.480769] usb 1-2: new high-speed USB device number 3 using ehci-pci1685client # [ 1.485793] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.488361] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.604265] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker2 # [ 1.632343] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21689worker2 # [ 1.637395] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01690worker1 # [ 1.607324] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691client # [ 1.620778] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692lb # [ 1.619895] systemd[1]: bpf-restrict-fs: LSM BPF program attached1693worker1 # [ 1.637257] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21694worker1 # [ 1.642419] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01695client # [ 1.648471] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696lb # [ 1.647753] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21697client # [ 1.648796] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01698lb # [ 1.648124] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699worker2 # [ 1.720928] systemd[1]: Queued start job for default target Initrd Default Target.1700worker2 # [ 1.732287] systemd[1]: Created slice Slice /system/modprobe.1701worker2 # [ 1.733504] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1702worker2 # [ 1.734943] systemd[1]: Expecting device /dev/disk/by-label/nixos...1703worker2 # [ 1.736034] systemd[1]: Reached target Path Units.1704worker2 # [ 1.736869] systemd[1]: Reached target Slice Units.1705worker2 # [ 1.737725] systemd[1]: Reached target Swaps.1706worker2 # [ 1.738566] systemd[1]: Reached target Timer Units.1707worker2 # [ 1.739605] systemd[1]: Listening on D-Bus System Message Bus Socket.1708worker2 # [ 1.740854] systemd[1]: Listening on Journal Socket (/dev/log).1709worker2 # [ 1.742049] systemd[1]: Listening on Journal Sockets.1710worker2 # [ 1.742198] systemd[1]: Listening on udev Control Socket.1711worker2 # [ 1.742316] systemd[1]: Listening on udev Kernel Socket.1712worker2 # [ 1.742340] systemd[1]: Reached target Socket Units.1713worker2 # [ 1.747362] systemd[1]: Starting Create List of Static Device Nodes...1714worker2 # [ 1.757972] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1715worker2 # [ 1.759163] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1716worker2 # [ 1.762567] systemd[1]: Mounting Kernel Configuration File System...1717worker1 # [ 1.723511] systemd[1]: Queued start job for default target Initrd Default Target.1718worker2 # [ 1.774273] systemd[1]: Starting Journal Service...1719worker1 # [ 1.735186] systemd[1]: Created slice Slice /system/modprobe.1720worker1 # [ 1.736411] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1721client # [ 1.736958] systemd[1]: Queued start job for default target Initrd Default Target.1722worker1 # [ 1.737863] systemd[1]: Expecting device /dev/disk/by-label/nixos...1723worker1 # [ 1.738961] systemd[1]: Reached target Path Units.1724lb # [ 1.736197] systemd[1]: Queued start job for default target Initrd Default Target.1725worker1 # [ 1.739814] systemd[1]: Reached target Slice Units.1726worker1 # [ 1.740680] systemd[1]: Reached target Swaps.1727worker1 # [ 1.741549] systemd[1]: Reached target Timer Units.1728worker1 # [ 1.742663] systemd[1]: Listening on D-Bus System Message Bus Socket.1729worker1 # [ 1.743926] systemd[1]: Listening on Journal Socket (/dev/log).1730worker1 # [ 1.745122] systemd[1]: Listening on Journal Sockets.1731worker1 # [ 1.745270] systemd[1]: Listening on udev Control Socket.1732worker2 # [ 1.786608] systemd[1]: Starting Load Kernel Modules...1733worker1 # [ 1.745388] systemd[1]: Listening on udev Kernel Socket.1734client # [ 1.748153] systemd[1]: Created slice Slice /system/modprobe.1735worker2 # [ 1.786707] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1736worker1 # [ 1.745412] systemd[1]: Reached target Socket Units.1737lb # [ 1.746518] systemd[1]: Created slice Slice /system/modprobe.1738client # [ 1.749330] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739worker1 # [ 1.750467] systemd[1]: Starting Create List of Static Device Nodes...1740lb # [ 1.747603] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1741client # [ 1.750722] systemd[1]: Expecting device /dev/disk/by-label/nixos...1742lb # [ 1.748886] systemd[1]: Expecting device /dev/disk/by-label/nixos...1743client # [ 1.751793] systemd[1]: Reached target Path Units.1744lb # [ 1.749861] systemd[1]: Reached target Path Units.1745client # [ 1.752610] systemd[1]: Reached target Slice Units.1746lb # [ 1.750621] systemd[1]: Reached target Slice Units.1747client # [ 1.753442] systemd[1]: Reached target Swaps.1748lb # [ 1.751389] systemd[1]: Reached target Swaps.1749client # [ 1.754268] systemd[1]: Reached target Timer Units.1750lb # [ 1.752087] systemd[1]: Reached target Timer Units.1751client # [ 1.755356] systemd[1]: Listening on D-Bus System Message Bus Socket.1752lb # [ 1.753082] systemd[1]: Listening on D-Bus System Message Bus Socket.1753client # [ 1.756583] systemd[1]: Listening on Journal Socket (/dev/log).1754lb # [ 1.754321] systemd[1]: Listening on Journal Socket (/dev/log).1755client # [ 1.757704] systemd[1]: Listening on Journal Sockets.1756lb # [ 1.755374] systemd[1]: Listening on Journal Sockets.1757lb # [ 1.756296] systemd[1]: Listening on udev Control Socket.1758worker2 # [ 1.799337] systemd[1]: Starting Coldplug All udev Devices...1759lb # [ 1.757280] systemd[1]: Listening on udev Kernel Socket.1760client # [ 1.758753] systemd[1]: Listening on udev Control Socket.1761lb # [ 1.758118] systemd[1]: Reached target Socket Units.1762worker1 # [ 1.760902] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1763client # [ 1.758876] systemd[1]: Listening on udev Kernel Socket.1764client # [ 1.758900] systemd[1]: Reached target Socket Units.1765worker1 # [ 1.762076] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1766lb # [ 1.760384] systemd[1]: Starting Create List of Static Device Nodes...1767client # [ 1.763096] systemd[1]: Starting Create List of Static Device Nodes...1768worker1 # [ 1.765529] systemd[1]: Mounting Kernel Configuration File System...1769lb # [ 1.768587] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1770lb # [ 1.769652] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1771client # [ 1.773426] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1772client # [ 1.774752] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1773worker1 # [ 1.785162] systemd[1]: Starting Journal Service...1774lb # [ 1.781679] systemd[1]: Mounting Kernel Configuration File System...1775client # [ 1.787287] systemd[1]: Mounting Kernel Configuration File System...1776worker2 # [ 1.833979] systemd[1]: Finished Create List of Static Device Nodes.1777worker2 # [ 1.835199] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1778worker2 # [ 1.835503] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1779worker2 # [ 1.839343] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1780worker2 # [ 1.839579] systemd[1]: Mounted Kernel Configuration File System.1781lb # [ 1.800593] systemd[1]: Starting Journal Service...1782worker1 # [ 1.808951] systemd[1]: Starting Load Kernel Modules...1783lb # [ 1.806920] systemd[1]: Starting Load Kernel Modules...1784client # [ 1.809905] systemd[1]: Starting Journal Service...1785worker1 # [ 1.809842] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786lb # [ 1.807880] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1787worker2 # [ 1.849680] systemd-journald[73]: Collecting audit messages is disabled.1788client # [ 1.813960] systemd[1]: Starting Load Kernel Modules...1789client # [ 1.814876] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1790client # [ 1.818952] systemd[1]: Starting Coldplug All udev Devices...1791worker2 # [ 1.863280] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1792client # [ 1.830068] systemd[1]: Finished Create List of Static Device Nodes.1793lb # [ 1.824725] systemd[1]: Starting Coldplug All udev Devices...1794client # [ 1.831914] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1795worker1 # [ 1.831737] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1796worker1 # [ 1.836952] systemd[1]: Starting Coldplug All udev Devices...1797worker1 # [ 1.844888] systemd[1]: Finished Create List of Static Device Nodes.1798worker2 # [ 1.885962] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1799worker1 # [ 1.846567] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1800client # [ 1.846007] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1801worker2 # [ 1.890454] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1802worker2 # [ 1.894207] systemd[1]: Starting Create Static Device Nodes in /dev...1803worker1 # [ 1.848926] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1804worker1 # [ 1.853543] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01805lb # [ 1.850716] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1806worker1 # [ 1.853775] [drm] features: -virgl +edid -resource_blob -host_visible1807worker1 # [ 1.853783] [drm] features: -context_init1808worker1 # [ 1.854458] [drm] number of scanouts: 11809worker1 # [ 1.854474] [drm] number of cap sets: 01810lb # [ 1.853979] systemd[1]: Finished Create List of Static Device Nodes.1811lb # [ 1.855123] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1812worker1 # [ 1.859392] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1813lb # [ 1.855408] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1814lb # [ 1.861156] systemd-journald[73]: Collecting audit messages is disabled.1815client # [ 1.866728] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1816worker2 # [ 1.907733] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01817worker2 # [ 1.907973] [drm] features: -virgl +edid -resource_blob -host_visible1818worker2 # [ 1.907982] [drm] features: -context_init1819worker2 # [ 1.908671] [drm] number of scanouts: 11820worker2 # [ 1.908689] [drm] number of cap sets: 01821lb # [ 1.868771] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1822lb # [ 1.869039] systemd[1]: Mounted Kernel Configuration File System.1823worker2 # [ 1.912759] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1824worker2 # [ 1.912775] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01825client # [ 1.874540] systemd[1]: Mounted Kernel Configuration File System.1826client # [ 1.882475] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1827lb # [ 1.880453] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1828worker1 # [ 1.877149] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1829worker1 # [ 1.877168] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01830worker1 # [ 1.886183] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1831client # [ 1.887623] systemd-journald[73]: Collecting audit messages is disabled.1832worker1 # [ 1.886720] systemd[1]: Mounted Kernel Configuration File System.1833worker1 # [ 1.893193] systemd-journald[73]: Collecting audit messages is disabled.1834client # [ 1.898369] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1835client # [ 1.900586] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836lb # [ 1.898547] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01837lb # [ 1.898781] [drm] features: -virgl +edid -resource_blob -host_visible1838lb # [ 1.898790] [drm] features: -context_init1839lb # [ 1.899465] [drm] number of scanouts: 11840lb # [ 1.899482] [drm] number of cap sets: 01841client # [ 1.905957] systemd[1]: Starting Create Static Device Nodes in /dev...1842lb # [ 1.904595] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1843lb # [ 1.904615] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01844worker1 # [ 1.909053] Console: switching to colour frame buffer device 160x501845worker2 # [ 1.950068] Console: switching to colour frame buffer device 160x501846client # [ 1.913716] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01847worker2 # [ 1.956478] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1848worker2 # [ 1.959940] systemd[1]: Finished Create Static Device Nodes in /dev.1849worker2 # [ 1.960145] systemd[1]: Reached target Preparation for Local File Systems.1850client # [ 1.922387] [drm] features: -virgl +edid -resource_blob -host_visible1851worker2 # [ 1.960180] systemd[1]: Reached target Local File Systems.1852client # [ 1.922404] [drm] features: -context_init1853client # [ 1.923134] [drm] number of scanouts: 11854client # [ 1.923152] [drm] number of cap sets: 01855lb # [ 1.920977] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1856lb # [ 1.923375] systemd[1]: Starting Create Static Device Nodes in /dev...1857worker1 # [ 1.925331] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1858worker2 # [ 1.966244] systemd[1]: Starting Rule-based Manager for Device Events and Files...1859worker1 # [ 1.927781] systemd[1]: Starting Create Static Device Nodes in /dev...1860client # [ 1.930142] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1861client # [ 1.930158] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01862worker1 # [ 1.937423] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1863lb # [ 1.940612] Console: switching to colour frame buffer device 160x501864worker1 # [ 1.953956] systemd[1]: Finished Load Kernel Modules.1865worker1 # [ 1.960265] systemd[1]: Starting Apply Kernel Variables...1866worker2 # [ 1.998043] systemd[1]: Finished Load Kernel Modules.1867worker2 # [ 1.999584] systemd[1]: Starting Apply Kernel Variables...1868client # [ 1.957533] Console: switching to colour frame buffer device 160x501869lb # [ 1.968940] systemd[1]: Finished Create Static Device Nodes in /dev.1870lb # [ 1.969130] systemd[1]: Reached target Preparation for Local File Systems.1871client # [ 1.970404] systemd[1]: Finished Create Static Device Nodes in /dev.1872lb # [ 1.969155] systemd[1]: Reached target Local File Systems.1873client # [ 1.970575] systemd[1]: Reached target Preparation for Local File Systems.1874worker1 # [ 1.973876] systemd[1]: Finished Create Static Device Nodes in /dev.1875client # [ 1.970597] systemd[1]: Reached target Local File Systems.1876worker1 # [ 1.975970] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 1.976404] systemd[1]: Starting Rule-based Manager for Device Events and Files...1878worker1 # [ 1.978003] systemd[1]: Reached target Local File Systems.1879lb # [ 1.972970] systemd[1]: Starting Rule-based Manager for Device Events and Files...1880lb # [ 1.981072] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881client # [ 1.994395] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1882worker1 # [ 1.997142] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883worker2 # [ 2.031904] systemd-modules-load[74]: Inserted module 'dm_mod'1884worker2 # [ 2.046845] systemd[1]: Started Journal Service.1885lb # [ 2.004578] systemd[1]: Finished Load Kernel Modules.1886worker2 # [ 2.036298] systemd-modules-load[74]: Module 'virtio_balloon' is built in1887worker2 # [ 2.037447] systemd-modules-load[74]: Module 'virtio_console' is built in1888client # [ 2.010373] systemd[1]: Finished Load Kernel Modules.1889lb # [ 2.007717] systemd[1]: Starting Apply Kernel Variables...1890worker2 # [ 2.038509] systemd-modules-load[74]: Inserted module 'virtio_gpu'1891worker2 # [ 2.039496] systemd-modules-load[74]: Module 'virtio_rng' is built in1892worker1 # [ 2.021049] systemd[1]: Finished Apply Kernel Variables.1893client # [ 2.018163] systemd[1]: Starting Apply Kernel Variables...1894worker2 # [ 2.052387] systemd[1]: Starting Create System Files and Directories...1895worker2 # [ 2.055441] systemd[1]: Finished Apply Kernel Variables.1896worker2 # [ 2.065555] systemd-udevd[79]: Using default interface naming scheme 'v261'.1897lb # [ 2.028372] systemd-modules-load[74]: Inserted module 'dm_mod'1898lb # [ 2.045765] systemd[1]: Started Journal Service.1899lb # [ 2.032684] systemd-modules-load[74]: Module 'virtio_balloon' is built in1900lb # [ 2.033786] systemd-modules-load[74]: Module 'virtio_console' is built in1901lb # [ 2.034824] systemd-modules-load[74]: Inserted module 'virtio_gpu'1902lb # [ 2.035793] systemd-modules-load[74]: Module 'virtio_rng' is built in1903worker1 # [ 2.054085] systemd[1]: Started Journal Service.1904worker1 # [ 2.040909] systemd-modules-load[74]: Inserted module 'dm_mod'1905worker1 # [ 2.047840] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906worker2 # [ 2.093113] systemd[1]: Finished Create System Files and Directories.1907worker1 # [ 2.052368] systemd-modules-load[74]: Module 'virtio_console' is built in1908client # [ 2.067349] systemd[1]: Started Journal Service.1909worker1 # [ 2.053440] systemd-modules-load[74]: Inserted module 'virtio_gpu'1910worker1 # [ 2.054393] systemd-modules-load[74]: Module 'virtio_rng' is built in1911worker1 # [ 2.055504] systemd[1]: Starting Create System Files and Directories...1912lb # [ 2.055896] systemd[1]: Starting Create System Files and Directories...1913worker2 # [ 2.104303] systemd[1]: Started Rule-based Manager for Device Events and Files.1914client # [ 2.060323] systemd-modules-load[74]: Inserted module 'dm_mod'1915client # [ 2.061552] systemd-modules-load[74]: Module 'virtio_balloon' is built in1916client # [ 2.064795] systemd-modules-load[74]: Module 'virtio_console' is built in1917lb # [ 2.069917] systemd-udevd[80]: Using default interface naming scheme 'v261'.1918client # [ 2.073435] systemd-modules-load[74]: Inserted module 'virtio_gpu'1919client # [ 2.076003] systemd-modules-load[74]: Module 'virtio_rng' is built in1920lb # [ 2.077295] systemd[1]: Finished Apply Kernel Variables.1921worker1 # [ 2.080386] systemd-udevd[81]: Using default interface naming scheme 'v261'.1922client # [ 2.081572] systemd[1]: Starting Create System Files and Directories...1923lb # [ 2.082492] systemd[1]: Finished Create System Files and Directories.1924client # [ 2.085790] systemd-udevd[80]: Using default interface naming scheme 'v261'.1925worker1 # [ 2.089077] systemd[1]: Finished Create System Files and Directories.1926client # [ 2.093884] systemd[1]: Finished Apply Kernel Variables.1927client # [ 2.096772] systemd[1]: Finished Create System Files and Directories.1928lb # [ 2.106103] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker2 # [ 2.154813] systemd[1]: Starting Virtual Console Setup...1930worker1 # [ 2.113995] systemd[1]: Started Rule-based Manager for Device Events and Files.1931client # [ 2.117512] systemd[1]: Started Rule-based Manager for Device Events and Files.1932lb # [ 2.156930] systemd[1]: Starting Virtual Console Setup...1933worker1 # [ 2.164191] systemd[1]: Starting Virtual Console Setup...1934worker2 # [ 2.208461] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1935worker2 # [ 2.211801] systemd[1]: Finished Virtual Console Setup.1936client # [ 2.176135] systemd[1]: Starting Virtual Console Setup...1937lb # [ 2.208482] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker1 # [ 2.212467] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1939lb # [ 2.211744] systemd[1]: Finished Virtual Console Setup.1940worker1 # [ 2.215803] systemd[1]: Finished Virtual Console Setup.1941client # [ 2.224459] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942client # [ 2.227795] systemd[1]: Finished Virtual Console Setup.1943worker2 # [ 2.800073] systemd[1]: Finished Coldplug All udev Devices.1944worker2 # [ 2.800995] systemd[1]: Reached target System Initialization.1945worker2 # [ 2.802060] systemd[1]: Reached target Basic System.1946worker1 # [ 2.804079] systemd[1]: Finished Coldplug All udev Devices.1947worker1 # [ 2.805009] systemd[1]: Reached target System Initialization.1948worker1 # [ 2.806020] systemd[1]: Reached target Basic System.1949lb # [ 2.812602] systemd[1]: Finished Coldplug All udev Devices.1950lb # [ 2.813523] systemd[1]: Reached target System Initialization.1951lb # [ 2.816135] systemd[1]: Reached target Basic System.1952client # [ 2.829133] systemd[1]: Finished Coldplug All udev Devices.1953client # [ 2.830086] systemd[1]: Reached target System Initialization.1954client # [ 2.830884] systemd[1]: Reached target Basic System.1955worker2 # [ 2.920945] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1956worker2 # [ 2.944360] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957worker1 # [ 2.927997] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958worker2 # [ 2.985780] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 2.951439] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1960worker1 # [ 2.960612] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1961worker1 # [ 2.968181] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1962lb # [ 2.981425] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1963lb # [ 2.985920] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1964client # [ 2.987156] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1965worker2 # [ 3.034058] systemd[1]: Found device /dev/disk/by-label/nixos.1966worker2 # [ 3.035961] systemd[1]: Reached target Initrd Root Device.1967worker2 # [ 3.039523] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968client # [ 3.004381] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969client # [ 3.020993] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 3.032722] systemd[1]: Found device /dev/disk/by-label/nixos.1971worker1 # [ 3.036596] systemd[1]: Reached target Initrd Root Device.1972worker1 # [ 3.040190] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973worker2 # [ 3.091669] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1974worker2 # [ 3.097920] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1975lb # [ 3.060828] systemd[1]: Found device /dev/disk/by-label/nixos.1976worker2 # [ 3.108389] systemd[1]: Mounting /sysroot...1977lb # [ 3.064594] systemd[1]: Reached target Initrd Root Device.1978client # [ 3.065876] systemd[1]: Found device /dev/disk/by-label/nixos.1979client # [ 3.068386] systemd[1]: Reached target Initrd Root Device.1980lb # [ 3.068305] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981client # [ 3.071898] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker1 # [ 3.091323] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1983worker1 # [ 3.097270] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1984worker2 # [ 3.152224] EXT4-fs (vda): mounted filesystem d54883b0-8719-4c61-8db3-af0db3350b23 r/w with ordered data mode. Quota mode: none.1985worker2 # [ 3.144087] systemd[1]: Mounted /sysroot.1986worker2 # [ 3.146227] systemd[1]: Reached target Initrd Root File System.1987worker1 # [ 3.107915] systemd[1]: Mounting /sysroot...1988worker2 # [ 3.151372] systemd[1]: Starting Mountpoints Configured in the Real Root...1989lb # [ 3.113777] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1990client # [ 3.120843] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1991lb # [ 3.120131] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992lb # [ 3.122387] systemd[1]: Mounting /sysroot...1993client # [ 3.127115] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker1 # [ 3.149545] EXT4-fs (vda): mounted filesystem 3cfca95e-4bb4-4ee8-be7e-aa06a0edcc51 r/w with ordered data mode. Quota mode: none.1995worker2 # [ 3.179901] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1996client # [ 3.140092] systemd[1]: Mounting /sysroot...1997worker1 # [ 3.142104] systemd[1]: Mounted /sysroot.1998worker1 # [ 3.142818] systemd[1]: Reached target Initrd Root File System.1999worker2 # [ 3.186706] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...2000worker2 # [ 3.189522] systemd[1]: Reloading...
loading 1,846 hidden lines…
3847worker2 # [ 31.002093] nix-grpc-daemon[777]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500513848worker2 # [ 31.006306] nix-grpc-daemon[777]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3849worker1 # [ 31.054773] nix-grpc-daemon[776]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500513850worker1 # [ 31.059923] nix-grpc-daemon[776]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3851worker2 # [ 31.202815] nix-grpc-daemon[777]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500513852worker2 # [ 31.207922] nix-grpc-daemon[777]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3853worker1 # [ 31.260982] nix-grpc-daemon[776]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500513854worker1 # [ 31.266214] nix-grpc-daemon[776]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3855worker2 # [ 31.409116] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513856worker2 # [ 31.414139] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3857worker1 # [ 31.467041] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513858worker1 # [ 31.472325] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3859worker2 # [ 31.615204] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513860worker2 # [ 31.620540] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3861lb: must succeed: curl -sf localhost:9901/clusters3862worker1 # [ 31.674942] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513863worker1 # [ 31.679147] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3864lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)3865worker2 # [ 31.821913] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513866worker2 # [ 31.827258] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3867worker1 # [ 31.875276] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513868worker1 # [ 31.880554] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3869worker2 # [ 32.030301] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513870worker2 # [ 32.034571] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3871worker1 # [ 32.081544] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513872worker1 # [ 32.086721] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3873worker2 # [ 32.231225] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513874worker2 # [ 32.237119] nix-grpc-daemon[777]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3875worker1 # [ 32.287531] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500513876worker1 # [ 32.292767] nix-grpc-daemon[776]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3877worker2 # [ 32.438254] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513878worker2 # [ 32.444354] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3879worker1 # [ 32.493698] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513880worker1 # [ 32.499131] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3881worker2 # [ 32.645538] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513882worker2 # [ 32.650873] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3883lb: must succeed: curl -sf localhost:9901/clusters3884worker1 # [ 32.701695] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513885worker1 # [ 32.705785] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3887worker2 # [ 32.851828] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513888worker2 # [ 32.857241] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3889worker1 # [ 32.902285] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513890worker1 # [ 32.907543] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3891worker2 # [ 33.062081] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513892worker2 # [ 33.072179] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3893worker1 # [ 33.108106] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513894worker1 # [ 33.111123] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3895worker2 # [ 33.263487] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513896worker2 # [ 33.268852] nix-grpc-daemon[777]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3897worker1 # [ 33.312434] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500513898worker1 # [ 33.317521] nix-grpc-daemon[776]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3899worker2 # [ 33.470356] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513900worker2 # [ 33.476078] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3901worker1 # [ 33.518330] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513902worker1 # [ 33.524490] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3903worker2 # [ 33.677004] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513904worker2 # [ 33.682274] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3905worker1 # [ 33.726683] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513906worker1 # [ 33.731152] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3907lb: must succeed: curl -sf localhost:9901/clusters3908worker2 # [ 33.883435] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513909worker2 # [ 33.888685] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3910lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3911worker1 # [ 33.927025] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513912worker1 # [ 33.932217] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3913worker2 # [ 34.091666] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513914worker2 # [ 34.095873] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3915worker1 # [ 34.133164] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513916worker1 # [ 34.138367] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3917worker2 # [ 34.292589] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513918worker2 # [ 34.297697] nix-grpc-daemon[777]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3919worker1 # [ 34.339182] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500513920worker1 # [ 34.344725] nix-grpc-daemon[776]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3921worker2 # [ 34.499026] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513922worker2 # [ 34.504319] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3923worker1 # [ 34.545596] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513924worker1 # [ 34.551185] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3925worker2 # [ 34.705440] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513926worker2 # [ 34.710669] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3927worker1 # [ 34.753633] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513928worker1 # [ 34.758265] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3929lb: must succeed: curl -sf localhost:9901/clusters3930worker2 # [ 34.911724] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513931worker2 # [ 34.916882] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3932lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3933worker1 # [ 34.959119] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513934worker1 # [ 34.964364] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3935worker2 # [ 35.119922] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513936worker2 # [ 35.124209] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3937worker1 # [ 35.165393] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513938worker1 # [ 35.170869] nix-grpc-daemon[776]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3939worker2 # [ 35.320744] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500513940worker2 # [ 35.325966] nix-grpc-daemon[777]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3941worker1 # [ 35.371735] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513942worker1 # [ 35.377105] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3943worker2 # [ 35.527356] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513944worker2 # [ 35.532823] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3945worker1 # [ 35.577729] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513946worker1 # [ 35.582902] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3947worker2 # [ 35.733651] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513948worker2 # [ 35.738844] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3949worker1 # [ 35.785070] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513950worker1 # [ 35.789747] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3951worker2 # [ 35.939929] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513952worker2 # [ 35.945217] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3953lb: must succeed: curl -sf localhost:9901/clusters3954worker1 # [ 35.990559] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513955worker1 # [ 35.995665] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3956lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3957worker2 # [ 36.148076] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513958worker2 # [ 36.152151] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3959worker1 # [ 36.196606] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513960worker1 # [ 36.201589] nix-grpc-daemon[776]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3961worker2 # [ 36.351152] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_connected addr=lb:500513962worker2 # [ 36.360573] nix-grpc-daemon[777]: ts=2026-09-21T15:07:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3963worker1 # [ 36.402171] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513964worker1 # [ 36.410099] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3965worker2 # [ 36.557586] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513966worker2 # [ 36.562830] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3967worker1 # [ 36.607804] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513968worker1 # [ 36.613117] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3969worker2 # [ 36.763982] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513970worker2 # [ 36.769088] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3971worker1 # [ 36.815600] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513972worker1 # [ 36.819828] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3973worker2 # [ 36.969969] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513974worker2 # [ 36.975030] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3975worker1 # [ 37.016256] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513976worker1 # [ 37.021197] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3977lb: must succeed: curl -sf localhost:9901/clusters3978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3979worker2 # [ 37.177998] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513980worker2 # [ 37.182144] nix-grpc-daemon[777]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3981worker1 # [ 37.221982] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_connected addr=lb:500513982worker1 # [ 37.227301] nix-grpc-daemon[776]: ts=2026-09-21T15:07:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3983worker2 # [ 37.378791] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513984worker2 # [ 37.384002] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3985worker2 # [ 37.444666] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_take_over3986lb # [ 37.401722] niks3-server[997]: 2026/09/21 15:07:57 INFO lead: acquired remote=192.168.1.4:380363987worker1 # [ 37.428229] nix-grpc-daemon[776]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513988worker1 # [ 37.433271] nix-grpc-daemon[776]: ts=2026-09-21T15:07:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3989worker2 # [ 37.585138] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513990worker2 # [ 37.590301] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3991worker1 # [ 37.634006] nix-grpc-daemon[776]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513992worker1 # [ 37.639175] nix-grpc-daemon[776]: ts=2026-09-21T15:07:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3993worker2 # [ 37.791412] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513994worker2 # [ 37.813539] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=oidc_jwks provider=mock keys=13995worker2 # [ 37.818035] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:602883996worker2 # [ 37.823335] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03997worker1 # [ 37.841257] nix-grpc-daemon[776]: ts=2026-09-21T15:07:57Z level=info event=scheduler_connected addr=lb:500513998worker2 # [ 37.884726] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:602883999worker2 # [ 37.889911] nix-grpc-daemon[777]: ts=2026-09-21T15:07:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04000lb: must succeed: curl -sf localhost:9901/clusters4001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4002worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4003worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4004lb: must succeed: curl -sf localhost:9901/clusters4005lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4006lb: must succeed: curl -sf localhost:9901/clusters4007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4008lb: must succeed: curl -sf localhost:9901/clusters4009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4010lb: must succeed: curl -sf localhost:9901/clusters4011lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4012worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4013worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)4014worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4015worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4016lb: must succeed: curl -sf localhost:9901/clusters4017lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4018lb: must succeed: curl -sf localhost:9901/clusters4019lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4020worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4021worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4022worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4023worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4024worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4025worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4026worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4027worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4028(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.64 seconds)4029subtest: balancer auth: client cert, bearer token, nothing, unknown CN4030worker1 # [ 39.403116] nix-grpc-daemon[776]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324031worker1 # [ 39.419514] systemd[1]: Started Nix Daemon.4032worker1 # [ 39.779607] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4033worker1 # [ 39.809259] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)4034worker1 # [ 39.897235] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 2/5)4035worker1 # [ 39.926784] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 380 ms (attempt 3/5)4036worker1 # [ 40.314513] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 734 ms (attempt 4/5)4037worker1 # [ 41.053749] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4038worker1 # [ 41.054384] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4039worker1 # [ 41.085826] nix-grpc-daemon[776]: ts=2026-09-21T15:08:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324040worker1 # [ 41.101061] nix-grpc-daemon[776]: ts=2026-09-21T15:08:00Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4041worker2 # [ 41.166830] nix-grpc-daemon[777]: ts=2026-09-21T15:08:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384042worker2 # [ 41.185756] systemd[1]: Started Nix Daemon.4043worker2 # [ 41.505373] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4044worker2 # [ 41.531134] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)4045worker2 # [ 41.558007] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 2/5)4046worker2 # [ 41.708655] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 212 ms (attempt 3/5)4047worker2 # [ 41.921254] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 494 ms (attempt 4/5)4048worker2 # [ 42.418644] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4049worker2 # [ 42.424380] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4050worker2 # [ 42.437456] nix-grpc-daemon[777]: ts=2026-09-21T15:08:02Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4051client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4052client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)4053client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4054client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4055worker1 # [ 42.885326] nix-grpc-daemon[776]: ts=2026-09-21T15:08:02Z level=info event=oidc_jwks provider=mock keys=14056worker1 # [ 42.892453] nix-grpc-daemon[776]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:507324057worker2 # [ 42.947760] nix-grpc-daemon[777]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:475384058worker1 # [ 42.922567] nix-grpc-daemon[776]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:507324059worker2 # [ 42.961059] nix-grpc-daemon[777]: ts=2026-09-21T15:08:02Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4060worker1 # [ 42.934028] nix-grpc-daemon[776]: ts=2026-09-21T15:08:02Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4061client: must succeed: rm -r /var/lib/nix-grpc-store4062client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4063worker1 # [ 43.241375] nix-grpc-daemon[776]: ts=2026-09-21T15:08:02Z level=info event=denied method=QueryMissing cn=- role=none4064worker1 # [ 43.506620] nix-grpc-daemon[776]: ts=2026-09-21T15:08:03Z level=info event=denied method=QueryMissing cn=stranger role=none4065(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.51 seconds)4066subtest: DAG build is scheduled across workers and lands in the cache4067client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24068worker2 # [ 43.814583] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:475384069client # don't know how to build these paths:4070client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4071worker1 # [ 43.788367] nix-grpc-daemon[776]: ts=2026-09-21T15:08:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324072worker2 # [ 43.867620] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04073worker2 # [ 43.875596] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=14074worker2 # [ 43.915922] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384075worker2 # [ 43.945603] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384076client # copying 1 paths to 'lb:50051'...4077worker2 # [ 43.967054] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384078client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4079worker2 # [ 44.001342] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4080worker2 # [ 44.040390] nix-grpc-daemon[777]: ts=2026-09-21T15:08:03Z level=info event=niks3_push_started pid=8784081worker2 # [ 44.361300] nix-grpc-daemon[878]: time=2026-09-21T15:08:03.998Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4082worker2 # [ 44.579015] nix-daemon[863]: accepted connection from pid 891, user nix-grpc-daemon (trusted)4083worker2 # [ 44.599284] nix-daemon[863]: reaped child process 894, status = succeeded4084worker2 # [ 44.798592] nix-daemon[863]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4085worker2 # [ 44.818002] nix-daemon[863]: reaped child process 899, status = succeeded4086lb # [ 44.785959] niks3-server[997]: 2026/09/21 15:08:04 INFO Received uploads request method=POST path=/api/pending_closures4087worker2 # [ 44.857814] nix-grpc-daemon[878]: time=2026-09-21T15:08:04.496Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4088worker2 # [ 44.862209] nix-grpc-daemon[878]: time=2026-09-21T15:08:04.499Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4089lb # [ 44.879263] niks3-server[997]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4090lb # [ 44.888734] niks3-server[997]: 2026/09/21 15:08:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4091lb # [ 44.906312] niks3-server[997]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4092lb # [ 44.920420] niks3-server[997]: 2026/09/21 15:08:04 INFO Signed narinfos id=1 count=14093worker2 # [ 44.970258] nix-grpc-daemon[878]: time=2026-09-21T15:08:04.608Z level=INFO msg="Uploading 1 narinfos"4094lb # [ 44.934069] niks3-server[997]: 2026/09/21 15:08:04 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4095lb # [ 44.945104] niks3-server[997]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4096lb # [ 44.959412] niks3-server[997]: 2026/09/21 15:08:04 INFO Completed upload id=14097worker2 # [ 45.005279] nix-grpc-daemon[878]: time=2026-09-21T15:08:04.644Z level=INFO msg="Upload complete. (636ms)"4098worker2 # [ 45.021858] nix-grpc-daemon[777]: ts=2026-09-21T15:08:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 paths=1 nar_bytes_in=5044099worker2 # [ 45.036400] nix-grpc-daemon[777]: ts=2026-09-21T15:08:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384100worker2 # [ 45.053237] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4101worker2 # [ 45.060407] nix-daemon[863]: reaped child process 876, status = succeeded4102client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4103client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4104worker2 # [ 45.076236] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4105client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4106client # farm-a> LOG-farm-a-t14107worker2 # [ 46.506904] nix-daemon[863]: reaped child process 903, status = succeeded4108worker2 # [ 46.693237] nix-daemon[863]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4109worker2 # [ 46.716875] nix-daemon[863]: reaped child process 923, status = succeeded4110lb # [ 46.685508] niks3-server[997]: 2026/09/21 15:08:06 INFO Received uploads request method=POST path=/api/pending_closures4111worker2 # [ 46.749604] nix-grpc-daemon[878]: time=2026-09-21T15:08:06.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4112worker2 # [ 46.751391] nix-grpc-daemon[878]: time=2026-09-21T15:08:06.386Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4113lb # [ 46.756356] niks3-server[997]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4114lb # [ 46.764462] niks3-server[997]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4115lb # [ 46.770609] niks3-server[997]: 2026/09/21 15:08:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4116lb # [ 46.777231] niks3-server[997]: 2026/09/21 15:08:06 INFO Signed narinfos id=2 count=14117worker2 # [ 46.822039] nix-grpc-daemon[878]: time=2026-09-21T15:08:06.459Z level=INFO msg="Uploading 1 narinfos"4118lb # [ 46.779757] niks3-server[997]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4119lb # [ 46.788588] niks3-server[997]: 2026/09/21 15:08:06 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4120lb # [ 46.797584] niks3-server[997]: 2026/09/21 15:08:06 INFO Completed upload id=24121worker2 # [ 46.843412] nix-grpc-daemon[878]: time=2026-09-21T15:08:06.482Z level=INFO msg="Upload complete. (352ms)"4122lb # [ 46.800334] niks3-server[997]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4123worker2 # [ 46.876948] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14124worker2 # [ 46.888459] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04125worker2 # [ 46.891353] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=14126worker2 # [ 46.895737] nix-daemon[863]: reaped child process 901, status = succeeded4127worker2 # [ 46.929291] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384128worker2 # [ 46.953420] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384129worker2 # [ 46.972397] nix-grpc-daemon[878]: time=2026-09-21T15:08:06.611Z level=INFO msg="All 1 paths already cached"4130client # copying 1 paths to 'lb:50051'...4131worker2 # [ 46.984156] nix-grpc-daemon[777]: ts=2026-09-21T15:08:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384132client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4133worker2 # [ 47.014234] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4134worker2 # [ 47.242077] nix-daemon[863]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4135worker2 # [ 47.261707] nix-daemon[863]: reaped child process 932, status = succeeded4136worker2 # [ 47.432341] nix-daemon[863]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4137worker2 # [ 47.448257] nix-daemon[863]: reaped child process 937, status = succeeded4138lb # [ 47.416141] niks3-server[997]: 2026/09/21 15:08:07 INFO Received uploads request method=POST path=/api/pending_closures4139worker2 # [ 47.472094] nix-grpc-daemon[878]: time=2026-09-21T15:08:07.110Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4140worker2 # [ 47.473605] nix-grpc-daemon[878]: time=2026-09-21T15:08:07.110Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4141lb # [ 47.449537] niks3-server[997]: 2026/09/21 15:08:07 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4142lb # [ 47.463787] niks3-server[997]: 2026/09/21 15:08:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4143worker2 # [ 47.515286] nix-grpc-daemon[878]: time=2026-09-21T15:08:07.153Z level=INFO msg="Uploading 1 narinfos"4144lb # [ 47.476868] niks3-server[997]: 2026/09/21 15:08:07 INFO Signed narinfos id=3 count=14145lb # [ 47.482203] niks3-server[997]: 2026/09/21 15:08:07 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4146worker2 # [ 47.536464] nix-grpc-daemon[878]: time=2026-09-21T15:08:07.175Z level=INFO msg="Upload complete. (498ms)"4147lb # [ 47.491549] niks3-server[997]: 2026/09/21 15:08:07 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4148lb # [ 47.496073] niks3-server[997]: 2026/09/21 15:08:07 INFO Completed upload id=34149lb # [ 47.499169] niks3-server[997]: 2026/09/21 15:08:07 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4150worker2 # [ 47.545027] nix-grpc-daemon[777]: ts=2026-09-21T15:08:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=6244151worker2 # [ 47.555980] nix-grpc-daemon[777]: ts=2026-09-21T15:08:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384152worker2 # [ 47.562819] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4153worker2 # [ 47.567186] nix-daemon[863]: reaped child process 927, status = succeeded4154client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4155client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4156worker2 # [ 47.579532] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4157client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4158client # farm-b> LOG-farm-b-t14159worker2 # [ 48.824865] nix-daemon[863]: reaped child process 941, status = succeeded4160worker2 # [ 48.999901] nix-daemon[863]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4161worker2 # [ 49.016614] nix-daemon[863]: reaped child process 960, status = succeeded4162lb # [ 48.985737] niks3-server[997]: 2026/09/21 15:08:08 INFO Received uploads request method=POST path=/api/pending_closures4163worker2 # [ 49.044268] nix-grpc-daemon[878]: time=2026-09-21T15:08:08.681Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4164worker2 # [ 49.046340] nix-grpc-daemon[878]: time=2026-09-21T15:08:08.681Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4165lb # [ 49.019206] niks3-server[997]: 2026/09/21 15:08:08 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4166worker2 # [ 49.083241] nix-grpc-daemon[878]: time=2026-09-21T15:08:08.721Z level=INFO msg="Uploading 1 narinfos"4167lb # [ 49.038235] niks3-server[997]: 2026/09/21 15:08:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4168lb # [ 49.047596] niks3-server[997]: 2026/09/21 15:08:08 INFO Signed narinfos id=4 count=14169lb # [ 49.053362] niks3-server[997]: 2026/09/21 15:08:08 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4170lb # [ 49.059407] niks3-server[997]: 2026/09/21 15:08:08 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4171lb # [ 49.069080] niks3-server[997]: 2026/09/21 15:08:08 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4172lb # [ 49.074202] niks3-server[997]: 2026/09/21 15:08:08 INFO Completed upload id=44173worker2 # [ 49.119840] nix-grpc-daemon[878]: time=2026-09-21T15:08:08.758Z level=INFO msg="Upload complete. (330ms)"4174lb # [ 49.076643] niks3-server[997]: 2026/09/21 15:08:08 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4175worker2 # [ 49.141203] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14176worker2 # [ 49.156342] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04177worker2 # [ 49.162035] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14178worker2 # [ 49.167634] nix-daemon[863]: reaped child process 939, status = succeeded4179worker2 # [ 49.183052] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384180worker2 # [ 49.199414] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384181worker2 # [ 49.213413] nix-grpc-daemon[878]: time=2026-09-21T15:08:08.852Z level=INFO msg="All 2 paths already cached"4182client # copying 1 paths to 'lb:50051'...4183worker2 # [ 49.222027] nix-grpc-daemon[777]: ts=2026-09-21T15:08:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384184client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4185worker2 # [ 49.247218] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4186worker2 # [ 49.446551] nix-daemon[863]: accepted connection from pid 966, user nix-grpc-daemon (trusted)4187worker2 # [ 49.463544] nix-daemon[863]: reaped child process 969, status = succeeded4188worker2 # [ 49.638641] nix-daemon[863]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4189worker2 # [ 49.654436] nix-daemon[863]: reaped child process 974, status = succeeded4190lb # [ 49.620809] niks3-server[997]: 2026/09/21 15:08:09 INFO Received uploads request method=POST path=/api/pending_closures4191worker2 # [ 49.676066] nix-grpc-daemon[878]: time=2026-09-21T15:08:09.314Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4192worker2 # [ 49.677517] nix-grpc-daemon[878]: time=2026-09-21T15:08:09.315Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4193lb # [ 49.651597] niks3-server[997]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4194worker2 # [ 49.709734] nix-grpc-daemon[878]: time=2026-09-21T15:08:09.348Z level=INFO msg="Uploading 1 narinfos"4195lb # [ 49.667808] niks3-server[997]: 2026/09/21 15:08:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4196lb # [ 49.678318] niks3-server[997]: 2026/09/21 15:08:09 INFO Signed narinfos id=5 count=14197worker2 # [ 49.733370] nix-grpc-daemon[878]: time=2026-09-21T15:08:09.372Z level=INFO msg="Upload complete. (468ms)"4198lb # [ 49.687928] niks3-server[997]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4199lb # [ 49.693807] niks3-server[997]: 2026/09/21 15:08:09 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4200worker2 # [ 49.740969] nix-grpc-daemon[777]: ts=2026-09-21T15:08:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=7524201lb # [ 49.698700] niks3-server[997]: 2026/09/21 15:08:09 INFO Completed upload id=54202lb # [ 49.700892] niks3-server[997]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4203worker2 # [ 49.754041] nix-grpc-daemon[777]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384204worker2 # [ 49.759388] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4205worker2 # [ 49.763770] nix-daemon[863]: reaped child process 964, status = succeeded4206client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4207worker2 # [ 49.777103] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4208client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4209client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4210client # farm-top> LOG-farm-top-t14211worker2 # [ 50.800851] nix-daemon[863]: reaped child process 978, status = succeeded4212worker2 # [ 50.958150] nix-daemon[863]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4213worker2 # [ 50.974664] nix-daemon[863]: reaped child process 997, status = succeeded4214lb # [ 50.941963] niks3-server[997]: 2026/09/21 15:08:10 INFO Received uploads request method=POST path=/api/pending_closures4215worker2 # [ 50.999137] nix-grpc-daemon[878]: time=2026-09-21T15:08:10.636Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4216worker2 # [ 51.000712] nix-grpc-daemon[878]: time=2026-09-21T15:08:10.636Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4217lb # [ 50.980465] niks3-server[997]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4218lb # [ 50.992578] niks3-server[997]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4219worker2 # [ 51.052625] nix-grpc-daemon[878]: time=2026-09-21T15:08:10.691Z level=INFO msg="Uploading 1 narinfos"4220lb # [ 51.002120] niks3-server[997]: 2026/09/21 15:08:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4221lb # [ 51.011864] niks3-server[997]: 2026/09/21 15:08:10 INFO Signed narinfos id=6 count=14222lb # [ 51.017127] niks3-server[997]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4223lb # [ 51.024292] niks3-server[997]: 2026/09/21 15:08:10 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4224lb # [ 51.032299] niks3-server[997]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4225lb # [ 51.037239] niks3-server[997]: 2026/09/21 15:08:10 INFO Completed upload id=64226worker2 # [ 51.082664] nix-grpc-daemon[878]: time=2026-09-21T15:08:10.721Z level=INFO msg="Upload complete. (297ms)"4227worker2 # [ 51.097101] nix-grpc-daemon[777]: ts=2026-09-21T15:08:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14228worker2 # [ 51.111348] nix-daemon[863]: reaped child process 976, status = succeeded4229client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.58 seconds)4230client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4231lb # [ 51.157114] envoy[914]: conn peer=[2001:db8:1::1]:35452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=30474232lb # [ 51.162571] envoy[914]: conn peer=[2001:db8:1::1]:50992 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=724233lb # [ 51.168306] envoy[914]: conn peer=[2001:db8:1::1]:50998 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114234lb # [ 51.173697] envoy[914]: conn peer=[2001:db8:1::1]:51002 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124235lb # [ 51.178905] envoy[914]: conn peer=[2001:db8:1::1]:51014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=4101 ms=73134236client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4237client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4238client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.04 seconds)4239client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4240client # copying 3 paths...4241client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4242client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4243client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4244client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4245(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.27 seconds)4246subtest: repeat is answered Cached by the scheduler without building4247worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4248worker1 # finding garbage collector roots...4249worker1 # deleting specified paths...4250worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4251worker1 # deleting unused links...4252worker1 # note: hard linking is currently saving 0.0 KiB4253worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4254worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4255worker2 # finding garbage collector roots...4256worker2 # deleting specified paths...4257worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4258worker2 # deleting unused links...4259worker2 # note: hard linking is currently saving 0.0 KiB4260worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4261client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24262worker1 # [ 52.639338] nix-grpc-daemon[776]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324263client # don't know how to build these paths:4264client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4265worker1 # [ 52.649883] nix-grpc-daemon[776]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324266worker2 # [ 52.722684] nix-grpc-daemon[777]: ts=2026-09-21T15:08:12Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14267client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.34 seconds)4268client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4269client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4270lb: must succeed: curl -sf localhost:9901/clusters4271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4272lb: must succeed: curl -sf localhost:9901/clusters4273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4274worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4275worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4276(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.44 seconds)4277subtest: inputs already in the cache are not sent to the scheduler4278lb: must succeed: curl -sf localhost:9901/clusters4279lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4280lb: must succeed: curl -sf localhost:9901/clusters4281lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4282worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4283worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4284client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24285client # don't know how to build these paths:4286client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4287worker1 # [ 53.693381] nix-grpc-daemon[776]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324288worker1 # [ 53.700174] nix-grpc-daemon[776]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324289worker2 # [ 53.761385] nix-grpc-daemon[777]: ts=2026-09-21T15:08:13Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04290worker2 # [ 53.767417] nix-grpc-daemon[777]: ts=2026-09-21T15:08:13Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14291worker2 # [ 53.816783] nix-grpc-daemon[777]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384292worker2 # [ 53.832684] nix-grpc-daemon[777]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384293worker2 # [ 53.848377] nix-grpc-daemon[878]: time=2026-09-21T15:08:13.486Z level=INFO msg="All 2 paths already cached"4294client # copying 1 paths to 'lb:50051'...4295client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4296worker2 # [ 53.858107] nix-grpc-daemon[777]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384297worker2 # [ 53.876312] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4298worker2 # [ 54.089950] nix-daemon[863]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4299worker2 # [ 54.107999] nix-daemon[863]: reaped child process 1029, status = succeeded4300worker2 # [ 54.282027] nix-daemon[863]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4301worker2 # [ 54.298703] nix-daemon[863]: reaped child process 1034, status = succeeded4302lb # [ 54.266915] niks3-server[997]: 2026/09/21 15:08:13 INFO Received uploads request method=POST path=/api/pending_closures4303worker2 # [ 54.321932] nix-grpc-daemon[878]: time=2026-09-21T15:08:13.960Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4304worker2 # [ 54.323360] nix-grpc-daemon[878]: time=2026-09-21T15:08:13.960Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4305lb # [ 54.294808] niks3-server[997]: 2026/09/21 15:08:13 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4306worker2 # [ 54.353199] nix-grpc-daemon[878]: time=2026-09-21T15:08:13.992Z level=INFO msg="Uploading 1 narinfos"4307lb # [ 54.311363] niks3-server[997]: 2026/09/21 15:08:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4308lb # [ 54.321166] niks3-server[997]: 2026/09/21 15:08:13 INFO Signed narinfos id=7 count=14309lb # [ 54.325921] niks3-server[997]: 2026/09/21 15:08:13 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4310lb # [ 54.334390] niks3-server[997]: 2026/09/21 15:08:13 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4311lb # [ 54.338946] niks3-server[997]: 2026/09/21 15:08:13 INFO Completed upload id=74312worker2 # [ 54.384513] nix-grpc-daemon[878]: time=2026-09-21T15:08:14.023Z level=INFO msg="Upload complete. (490ms)"4313lb # [ 54.341629] niks3-server[997]: 2026/09/21 15:08:13 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4314worker2 # [ 54.394369] nix-grpc-daemon[777]: ts=2026-09-21T15:08:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=7524315worker2 # [ 54.406461] nix-grpc-daemon[777]: ts=2026-09-21T15:08:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384316worker2 # [ 54.412286] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4317worker2 # [ 54.416916] nix-daemon[863]: reaped child process 1024, status = succeeded4318client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4319client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4320worker2 # [ 54.429833] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4321client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4322client # farm-top> LOG-farm-top-t1b4323worker2 # [ 55.650453] nix-daemon[863]: reaped child process 1038, status = succeeded4324worker2 # [ 55.808765] nix-daemon[863]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4325worker2 # [ 55.827429] nix-daemon[863]: reaped child process 1057, status = succeeded4326lb # [ 55.797638] niks3-server[997]: 2026/09/21 15:08:15 INFO Received uploads request method=POST path=/api/pending_closures4327worker2 # [ 55.854790] nix-grpc-daemon[878]: time=2026-09-21T15:08:15.492Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4328worker2 # [ 55.858208] nix-grpc-daemon[878]: time=2026-09-21T15:08:15.492Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4329lb # [ 55.839305] niks3-server[997]: 2026/09/21 15:08:15 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4330lb # [ 55.851825] niks3-server[997]: 2026/09/21 15:08:15 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4331worker2 # [ 55.907693] nix-grpc-daemon[878]: time=2026-09-21T15:08:15.546Z level=INFO msg="Uploading 1 narinfos"4332lb # [ 55.862006] niks3-server[997]: 2026/09/21 15:08:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4333lb # [ 55.866823] niks3-server[997]: 2026/09/21 15:08:15 INFO Signed narinfos id=8 count=14334lb # [ 55.871544] niks3-server[997]: 2026/09/21 15:08:15 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4335lb # [ 55.880193] niks3-server[997]: 2026/09/21 15:08:15 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4336lb # [ 55.886876] niks3-server[997]: 2026/09/21 15:08:15 INFO Completed upload id=84337worker2 # [ 55.932139] nix-grpc-daemon[878]: time=2026-09-21T15:08:15.571Z level=INFO msg="Upload complete. (293ms)"4338lb # [ 55.889921] niks3-server[997]: 2026/09/21 15:08:15 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4339worker2 # [ 55.944293] nix-grpc-daemon[777]: ts=2026-09-21T15:08:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14340worker2 # [ 55.956907] nix-daemon[863]: reaped child process 1036, status = succeeded4341client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.51 seconds)4342client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4343client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4344lb: must succeed: curl -sf localhost:9901/clusters4345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4346lb: must succeed: curl -sf localhost:9901/clusters4347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4348worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4349worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4350(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.16 seconds)4351subtest: two clients wanting the same drv build it once4352client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4353client # [ 56.477608] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4354client # Running as unit: dedup1.service; invocation ID: 4ec63b4a6ce3487a9db5ce8186ef37384355client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4356client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4357worker2 # [ 56.781704] nix-grpc-daemon[777]: ts=2026-09-21T15:08:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:475384358client # [ 56.745829] nix[882]: don't know how to build these paths:4359client # [ 56.747334] nix[882]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4360worker2 # [ 56.801075] nix-grpc-daemon[777]: ts=2026-09-21T15:08:16Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04361worker2 # [ 56.807448] nix-grpc-daemon[777]: ts=2026-09-21T15:08:16Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14362worker1 # [ 56.771022] nix-grpc-daemon[776]: ts=2026-09-21T15:08:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324363worker1 # [ 56.788723] nix-grpc-daemon[776]: ts=2026-09-21T15:08:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324364client # [ 56.800085] nix[882]: copying 1 paths to 'lb:50051'...4365client # [ 56.802710] nix[882]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4366worker1 # [ 56.802876] nix-grpc-daemon[776]: ts=2026-09-21T15:08:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324367worker1 # [ 56.819355] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4368worker1 # [ 56.850373] nix-grpc-daemon[776]: ts=2026-09-21T15:08:16Z level=info event=niks3_push_started pid=8694369worker1 # [ 57.120155] nix-grpc-daemon[869]: time=2026-09-21T15:08:16.771Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4370worker1 # [ 57.308726] nix-daemon[841]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4371worker1 # [ 57.328598] nix-daemon[841]: reaped child process 885, status = succeeded4372worker1 # [ 57.498618] nix-daemon[841]: accepted connection from pid 887, user nix-grpc-daemon (trusted)4373worker1 # [ 57.514149] nix-daemon[841]: reaped child process 890, status = succeeded4374lb # [ 57.525463] niks3-server[997]: 2026/09/21 15:08:17 INFO Received uploads request method=POST path=/api/pending_closures4375worker1 # [ 57.536965] nix-grpc-daemon[869]: time=2026-09-21T15:08:17.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4376worker1 # [ 57.540232] nix-grpc-daemon[869]: time=2026-09-21T15:08:17.191Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4377lb # [ 57.560715] niks3-server[997]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4378lb # [ 57.573995] niks3-server[997]: 2026/09/21 15:08:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4379worker1 # [ 57.582797] nix-grpc-daemon[869]: time=2026-09-21T15:08:17.234Z level=INFO msg="Uploading 1 narinfos"4380lb # [ 57.584308] niks3-server[997]: 2026/09/21 15:08:17 INFO Signed narinfos id=9 count=14381lb # [ 57.592848] niks3-server[997]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4382client # [ 57.597416] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4383lb # [ 57.605688] niks3-server[997]: 2026/09/21 15:08:17 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4384worker1 # [ 57.613506] nix-grpc-daemon[869]: time=2026-09-21T15:08:17.265Z level=INFO msg="Upload complete. (485ms)"4385client # Running as unit: dedup2.service; invocation ID: 3df6ee2b7141404383282e6ec6886ce44386lb # [ 57.610699] niks3-server[997]: 2026/09/21 15:08:17 INFO Completed upload id=94387lb # [ 57.614104] niks3-server[997]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4388worker1 # [ 57.620114] nix-grpc-daemon[776]: ts=2026-09-21T15:08:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=4324389client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4390worker1 # [ 57.629029] nix-grpc-daemon[776]: ts=2026-09-21T15:08:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324391worker1 # [ 57.645103] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4392worker1 # [ 57.649644] nix-daemon[841]: reaped child process 867, status = succeeded4393client # [ 57.664527] nix[882]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4394worker1 # [ 57.668723] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4395client # [ 57.986764] nix[898]: don't know how to build these paths:4396client # [ 57.987929] nix[898]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4397worker2 # [ 58.030319] nix-grpc-daemon[777]: ts=2026-09-21T15:08:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:475384398worker2 # [ 58.043410] nix-grpc-daemon[777]: ts=2026-09-21T15:08:17Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04399client # [ 58.009253] nix[898]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4400worker1 # [ 58.009281] nix-grpc-daemon[776]: ts=2026-09-21T15:08:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324401client: must succeed: sleep 24402client: (finished: must succeed: sleep 2, in 2.06 seconds)4403lb # [ 61.154085] envoy[914]: conn peer=[2001:db8:1::1]:47372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=624404lb # [ 61.158981] envoy[914]: conn peer=[2001:db8:1::1]:47380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4202 tx=1824 ms=22204405client: waiting for success: ! systemctl is-active dedup1 dedup24406worker1 # [ 61.319152] nix-daemon[841]: reaped child process 898, status = succeeded4407worker1 # [ 61.498945] nix-daemon[841]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4408worker1 # [ 61.515120] nix-daemon[841]: reaped child process 927, status = succeeded4409lb # [ 61.525332] niks3-server[997]: 2026/09/21 15:08:21 INFO Received uploads request method=POST path=/api/pending_closures4410worker1 # [ 61.538824] nix-grpc-daemon[869]: time=2026-09-21T15:08:21.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4411worker1 # [ 61.541093] nix-grpc-daemon[869]: time=2026-09-21T15:08:21.190Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4412lb # [ 61.561938] niks3-server[997]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4413lb # [ 61.584148] niks3-server[997]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4414lb # [ 61.593824] niks3-server[997]: 2026/09/21 15:08:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4415lb # [ 61.601140] niks3-server[997]: 2026/09/21 15:08:21 INFO Signed narinfos id=10 count=14416worker1 # [ 61.604204] nix-grpc-daemon[869]: time=2026-09-21T15:08:21.254Z level=INFO msg="Uploading 1 narinfos"4417lb # [ 61.604848] niks3-server[997]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4418lb # [ 61.620156] niks3-server[997]: 2026/09/21 15:08:21 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4419worker1 # [ 61.629850] nix-grpc-daemon[869]: time=2026-09-21T15:08:21.282Z level=INFO msg="Upload complete. (339ms)"4420lb # [ 61.628840] niks3-server[997]: 2026/09/21 15:08:21 INFO Completed upload id=104421lb # [ 61.631984] niks3-server[997]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4422worker1 # [ 61.645147] nix-grpc-daemon[776]: ts=2026-09-21T15:08:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14423worker1 # [ 61.660569] nix-grpc-daemon[776]: ts=2026-09-21T15:08:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14424worker1 # [ 61.668508] nix-daemon[841]: reaped child process 895, status = succeeded4425client # [ 61.694301] systemd[1]: dedup2.service: Deactivated successfully.4426client # [ 61.700337] systemd[1]: dedup2.service: Consumed 222ms CPU time over 4.102s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4427client # [ 61.738806] systemd[1]: dedup1.service: Deactivated successfully.4428client # [ 61.741133] systemd[1]: dedup1.service: Consumed 170ms CPU time over 5.263s wall clock time, 39.4M memory peak, 7.3K incoming IP traffic, 8.6K outgoing IP traffic.4429client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4430client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4431client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4432worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4433worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4434worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4435worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4436worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4437worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4438worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4439worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4440(finished: subtest: two clients wanting the same drv build it once, in 6.28 seconds)4441subtest: build hook: nix-daemon with builders = grpc://lb4442client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4443client # [ 62.909622] systemd[1]: Started Nix Daemon.4444client # [ 63.098203] nix-daemon[940]: accepted connection from pid 937, user root (trusted)4445worker1 # [ 63.316490] nix-grpc-daemon[776]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324446worker1 # [ 63.322736] nix-grpc-daemon[776]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324447worker2 # [ 63.385395] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04448worker2 # [ 63.391986] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14449worker2 # [ 63.437930] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384450worker2 # [ 63.453501] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384451worker2 # [ 63.468567] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384452worker2 # [ 63.483505] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4453worker2 # [ 63.690826] nix-daemon[863]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)4454worker2 # [ 63.708261] nix-daemon[863]: reaped child process 1096, status = succeeded4455worker2 # [ 63.885469] nix-daemon[863]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4456worker2 # [ 63.901489] nix-daemon[863]: reaped child process 1101, status = succeeded4457lb # [ 63.871096] niks3-server[997]: 2026/09/21 15:08:23 INFO Received uploads request method=POST path=/api/pending_closures4458worker2 # [ 63.924437] nix-grpc-daemon[878]: time=2026-09-21T15:08:23.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4459worker2 # [ 63.926268] nix-grpc-daemon[878]: time=2026-09-21T15:08:23.563Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4460lb # [ 63.904641] niks3-server[997]: 2026/09/21 15:08:23 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4461worker2 # [ 63.958307] nix-grpc-daemon[878]: time=2026-09-21T15:08:23.597Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 63.917635] niks3-server[997]: 2026/09/21 15:08:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4463lb # [ 63.928886] niks3-server[997]: 2026/09/21 15:08:23 INFO Signed narinfos id=11 count=14464lb # [ 63.936937] niks3-server[997]: 2026/09/21 15:08:23 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4465worker2 # [ 63.987328] nix-grpc-daemon[878]: time=2026-09-21T15:08:23.626Z level=INFO msg="Upload complete. (487ms)"4466lb # [ 63.941958] niks3-server[997]: 2026/09/21 15:08:23 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4467lb # [ 63.947461] niks3-server[997]: 2026/09/21 15:08:23 INFO Completed upload id=114468lb # [ 63.951280] niks3-server[997]: 2026/09/21 15:08:23 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4469worker2 # [ 63.996925] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=5124470worker2 # [ 64.004262] nix-grpc-daemon[777]: ts=2026-09-21T15:08:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384471worker2 # [ 64.016001] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4472worker2 # [ 64.018110] nix-daemon[863]: reaped child process 1091, status = succeeded4473worker2 # [ 64.029025] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4474worker2 # [ 65.476795] nix-daemon[863]: reaped child process 1105, status = succeeded4475worker2 # [ 65.645418] nix-daemon[863]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)4476worker2 # [ 65.663871] nix-daemon[863]: reaped child process 1124, status = succeeded4477lb # [ 65.631256] niks3-server[997]: 2026/09/21 15:08:25 INFO Received uploads request method=POST path=/api/pending_closures4478worker2 # [ 65.688465] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.324Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4479worker2 # [ 65.689954] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.324Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4480lb # [ 65.670128] niks3-server[997]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4481lb # [ 65.682577] niks3-server[997]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4482worker2 # [ 65.740033] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.378Z level=INFO msg="Uploading 1 narinfos"4483lb # [ 65.694456] niks3-server[997]: 2026/09/21 15:08:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4484lb # [ 65.702157] niks3-server[997]: 2026/09/21 15:08:25 INFO Signed narinfos id=12 count=14485lb # [ 65.707217] niks3-server[997]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4486lb # [ 65.714313] niks3-server[997]: 2026/09/21 15:08:25 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4487lb # [ 65.720552] niks3-server[997]: 2026/09/21 15:08:25 INFO Completed upload id=124488worker2 # [ 65.766152] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.405Z level=INFO msg="Upload complete. (325ms)"4489lb # [ 65.723270] niks3-server[997]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4490worker2 # [ 65.775997] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14491worker2 # [ 65.793030] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384492worker2 # [ 65.797805] nix-daemon[863]: reaped child process 1103, status = succeeded4493worker1 # [ 65.767482] nix-grpc-daemon[776]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324494worker1 # [ 65.788781] nix-grpc-daemon[776]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4495worker1 # [ 65.809069] nix-grpc-daemon[776]: ts=2026-09-21T15:08:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1284496worker2 # [ 66.168890] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:475384497worker2 # [ 66.176113] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384498worker2 # [ 66.182402] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.820Z level=INFO msg="All 1 paths already cached"4499worker2 # [ 66.194117] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04500worker2 # [ 66.202596] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14501worker2 # [ 66.210505] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384502worker2 # [ 66.217768] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384503worker2 # [ 66.228539] nix-grpc-daemon[878]: time=2026-09-21T15:08:25.867Z level=INFO msg="All 1 paths already cached"4504worker2 # [ 66.237161] nix-grpc-daemon[777]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384505worker2 # [ 66.251908] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4506worker2 # [ 66.444908] nix-daemon[863]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4507worker2 # [ 66.461546] nix-daemon[863]: reaped child process 1133, status = succeeded4508worker2 # [ 66.639114] nix-daemon[863]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4509worker2 # [ 66.653539] nix-daemon[863]: reaped child process 1138, status = succeeded4510lb # [ 66.623771] niks3-server[997]: 2026/09/21 15:08:26 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 66.678086] nix-grpc-daemon[878]: time=2026-09-21T15:08:26.316Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4512worker2 # [ 66.679512] nix-grpc-daemon[878]: time=2026-09-21T15:08:26.316Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4513lb # [ 66.650919] niks3-server[997]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4514lb # [ 66.665886] niks3-server[997]: 2026/09/21 15:08:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4515worker2 # [ 66.718230] nix-grpc-daemon[878]: time=2026-09-21T15:08:26.356Z level=INFO msg="Uploading 1 narinfos"4516lb # [ 66.676280] niks3-server[997]: 2026/09/21 15:08:26 INFO Signed narinfos id=13 count=14517lb # [ 66.681561] niks3-server[997]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4518lb # [ 66.688627] niks3-server[997]: 2026/09/21 15:08:26 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4519worker2 # [ 66.739814] nix-grpc-daemon[878]: time=2026-09-21T15:08:26.378Z level=INFO msg="Upload complete. (472ms)"4520lb # [ 66.694522] niks3-server[997]: 2026/09/21 15:08:26 INFO Completed upload id=134521lb # [ 66.697188] niks3-server[997]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4522worker2 # [ 66.745620] nix-grpc-daemon[777]: ts=2026-09-21T15:08:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=6244523worker2 # [ 66.758167] nix-grpc-daemon[777]: ts=2026-09-21T15:08:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384524worker2 # [ 66.764157] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4525worker2 # [ 66.768925] nix-daemon[863]: reaped child process 1128, status = succeeded4526worker2 # [ 66.781886] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4527worker2 # [ 68.223020] nix-daemon[863]: reaped child process 1142, status = succeeded4528worker2 # [ 68.413749] nix-daemon[863]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4529worker2 # [ 68.430332] nix-daemon[863]: reaped child process 1161, status = succeeded4530lb # [ 68.397997] niks3-server[997]: 2026/09/21 15:08:28 INFO Received uploads request method=POST path=/api/pending_closures4531worker2 # [ 68.452413] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.091Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4532worker2 # [ 68.454916] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.093Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4533lb # [ 68.434221] niks3-server[997]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4534lb # [ 68.444887] niks3-server[997]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4535worker2 # [ 68.503765] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.142Z level=INFO msg="Uploading 1 narinfos"4536lb # [ 68.459637] niks3-server[997]: 2026/09/21 15:08:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4537lb # [ 68.468237] niks3-server[997]: 2026/09/21 15:08:28 INFO Signed narinfos id=14 count=14538lb # [ 68.470917] niks3-server[997]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4539worker2 # [ 68.526793] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.165Z level=INFO msg="Upload complete. (334ms)"4540lb # [ 68.482625] niks3-server[997]: 2026/09/21 15:08:28 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4541lb # [ 68.488830] niks3-server[997]: 2026/09/21 15:08:28 INFO Completed upload id=144542lb # [ 68.491772] niks3-server[997]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4543worker2 # [ 68.540378] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14544worker2 # [ 68.556070] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384545worker2 # [ 68.560940] nix-daemon[863]: reaped child process 1140, status = succeeded4546worker1 # [ 68.530364] nix-grpc-daemon[776]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324547worker1 # [ 68.543853] nix-grpc-daemon[776]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4548worker1 # [ 68.559816] nix-grpc-daemon[776]: ts=2026-09-21T15:08:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1764549worker1 # [ 68.876425] nix-grpc-daemon[776]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324550worker2 # [ 68.922941] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384551worker2 # [ 68.932369] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.571Z level=INFO msg="All 2 paths already cached"4552worker2 # [ 68.949294] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04553worker2 # [ 68.956257] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14554worker2 # [ 68.993486] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384555worker2 # [ 69.008636] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384556worker2 # [ 69.022399] nix-grpc-daemon[878]: time=2026-09-21T15:08:28.661Z level=INFO msg="All 2 paths already cached"4557worker2 # [ 69.031336] nix-grpc-daemon[777]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384558worker2 # [ 69.045166] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4559worker2 # [ 69.250637] nix-daemon[863]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4560worker2 # [ 69.268290] nix-daemon[863]: reaped child process 1170, status = succeeded4561worker2 # [ 69.441317] nix-daemon[863]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4562worker2 # [ 69.456993] nix-daemon[863]: reaped child process 1175, status = succeeded4563lb # [ 69.424987] niks3-server[997]: 2026/09/21 15:08:29 INFO Received uploads request method=POST path=/api/pending_closures4564worker2 # [ 69.480380] nix-grpc-daemon[878]: time=2026-09-21T15:08:29.119Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4565worker2 # [ 69.482254] nix-grpc-daemon[878]: time=2026-09-21T15:08:29.121Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4566lb # [ 69.457247] niks3-server[997]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4567worker2 # [ 69.513213] nix-grpc-daemon[878]: time=2026-09-21T15:08:29.152Z level=INFO msg="Uploading 1 narinfos"4568lb # [ 69.472355] niks3-server[997]: 2026/09/21 15:08:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4569lb # [ 69.480881] niks3-server[997]: 2026/09/21 15:08:29 INFO Signed narinfos id=15 count=14570lb # [ 69.483312] niks3-server[997]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4571lb # [ 69.491263] niks3-server[997]: 2026/09/21 15:08:29 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4572lb # [ 69.497404] niks3-server[997]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4573worker2 # [ 69.548293] nix-grpc-daemon[878]: time=2026-09-21T15:08:29.187Z level=INFO msg="Upload complete. (486ms)"4574lb # [ 69.502741] niks3-server[997]: 2026/09/21 15:08:29 INFO Completed upload id=154575worker2 # [ 69.557219] nix-grpc-daemon[777]: ts=2026-09-21T15:08:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=7524576worker2 # [ 69.569933] nix-grpc-daemon[777]: ts=2026-09-21T15:08:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384577worker2 # [ 69.575923] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4578worker2 # [ 69.580811] nix-daemon[863]: reaped child process 1165, status = succeeded4579worker2 # [ 69.593657] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4580worker2 # [ 70.815768] nix-daemon[863]: reaped child process 1179, status = succeeded4581worker2 # [ 70.973552] nix-daemon[863]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4582worker2 # [ 70.991713] nix-daemon[863]: reaped child process 1198, status = succeeded4583lb # [ 70.959609] niks3-server[997]: 2026/09/21 15:08:30 INFO Received uploads request method=POST path=/api/pending_closures4584worker2 # [ 71.016382] nix-grpc-daemon[878]: time=2026-09-21T15:08:30.653Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4585worker2 # [ 71.017806] nix-grpc-daemon[878]: time=2026-09-21T15:08:30.653Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4586lb # [ 70.997521] niks3-server[997]: 2026/09/21 15:08:30 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4587lb # [ 71.008396] niks3-server[997]: 2026/09/21 15:08:30 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4588lb # [ 71.020586] niks3-server[997]: 2026/09/21 15:08:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4589worker2 # [ 71.072307] nix-grpc-daemon[878]: time=2026-09-21T15:08:30.710Z level=INFO msg="Uploading 1 narinfos"4590lb # [ 71.026541] niks3-server[997]: 2026/09/21 15:08:30 INFO Signed narinfos id=16 count=14591lb # [ 71.030440] niks3-server[997]: 2026/09/21 15:08:30 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4592lb # [ 71.040169] niks3-server[997]: 2026/09/21 15:08:30 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4593lb # [ 71.045249] niks3-server[997]: 2026/09/21 15:08:30 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4594lb # [ 71.049990] niks3-server[997]: 2026/09/21 15:08:30 INFO Completed upload id=164595worker2 # [ 71.095570] nix-grpc-daemon[878]: time=2026-09-21T15:08:30.734Z level=INFO msg="Upload complete. (292ms)"4596worker2 # [ 71.107717] nix-grpc-daemon[777]: ts=2026-09-21T15:08:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14597worker2 # [ 71.122488] nix-grpc-daemon[777]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384598worker2 # [ 71.126316] nix-daemon[863]: reaped child process 1177, status = succeeded4599worker2 # [ 71.129985] nix-grpc-daemon[777]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:475384600worker2 # [ 71.134447] nix-grpc-daemon[777]: ts=2026-09-21T15:08:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_out=2324601lb # [ 71.154655] envoy[914]: conn peer=[2001:db8:1::1]:47404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=36724602lb # [ 71.159624] envoy[914]: conn peer=[2001:db8:1::1]:47390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3022 tx=1444 ms=49724603lb # [ 71.164460] envoy[914]: conn peer=[2001:db8:1::1]:49346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1740 ms=25534604lb # [ 71.169925] envoy[914]: conn peer=[2001:db8:1::1]:49360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1104605lb # [ 71.174571] envoy[914]: conn peer=[2001:db8:1::1]:49374 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3904 tx=1950 ms=24934606lb # [ 71.179310] envoy[914]: conn peer=[2001:db8:1::1]:49384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=934607lb # [ 71.183981] envoy[914]: conn peer=[2001:db8:1::1]:49398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4356 tx=2182 ms=22744608lb # [ 71.188617] envoy[914]: conn peer=[2001:db8:1::1]:41324 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=684609client # [ 71.221213] nix-daemon[940]: reaped child process 942, status = succeeded4610client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.56 seconds)4611client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4612client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4613client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4614client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4615client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4616client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4617(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.69 seconds)4618subtest: an input only one worker has still reaches the builder4619client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4620client # this derivation will be built:4621client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4622client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4623client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.43 seconds)4624client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4625client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4626worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4627worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4628worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4629worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4630client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24631client # [ 72.503043] nix-daemon[940]: accepted connection from pid 1023, user root (trusted)4632client # this derivation will be built:4633client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4634client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4635worker2 # [ 72.759946] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:475384636worker2 # [ 72.768589] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384637client # copying 1 paths...4638client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4639worker2 # [ 72.780728] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384640worker2 # [ 72.796901] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4641worker2 # [ 73.005762] nix-daemon[863]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)4642worker2 # [ 73.024953] nix-daemon[863]: reaped child process 1208, status = succeeded4643lb # [ 72.994367] niks3-server[997]: 2026/09/21 15:08:32 INFO Received uploads request method=POST path=/api/pending_closures4644worker2 # [ 73.049041] nix-grpc-daemon[878]: time=2026-09-21T15:08:32.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4645worker2 # [ 73.050489] nix-grpc-daemon[878]: time=2026-09-21T15:08:32.687Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4646lb # [ 73.020413] niks3-server[997]: 2026/09/21 15:08:32 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4647lb # [ 73.036002] niks3-server[997]: 2026/09/21 15:08:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4648worker2 # [ 73.086730] nix-grpc-daemon[878]: time=2026-09-21T15:08:32.725Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 73.047881] niks3-server[997]: 2026/09/21 15:08:32 INFO Signed narinfos id=17 count=14650lb # [ 73.054633] niks3-server[997]: 2026/09/21 15:08:32 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4651lb # [ 73.066301] niks3-server[997]: 2026/09/21 15:08:32 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4652lb # [ 73.072381] niks3-server[997]: 2026/09/21 15:08:32 INFO Completed upload id=174653worker2 # [ 73.117831] nix-grpc-daemon[878]: time=2026-09-21T15:08:32.756Z level=INFO msg="Upload complete. (304ms)"4654lb # [ 73.075899] niks3-server[997]: 2026/09/21 15:08:32 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4655worker2 # [ 73.128305] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=0 paths=1 nar_bytes_in=1204656worker2 # [ 73.142259] nix-daemon[863]: reaped child process 1203, status = succeeded4657worker2 # [ 73.147155] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04658worker2 # [ 73.152189] nix-grpc-daemon[777]: ts=2026-09-21T15:08:32Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14659worker1 # [ 73.158896] nix-grpc-daemon[776]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324660worker1 # [ 73.170621] nix-grpc-daemon[776]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324661client # copying 2 paths to 'lb:50051'...4662client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4663worker1 # [ 73.187109] nix-grpc-daemon[776]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324664client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4665worker1 # [ 73.205363] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4666worker1 # [ 73.409038] nix-daemon[841]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4667worker1 # [ 73.427467] nix-daemon[841]: reaped child process 950, status = succeeded4668worker1 # [ 73.596500] nix-daemon[841]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4669worker1 # [ 73.614310] nix-daemon[841]: reaped child process 955, status = succeeded4670lb # [ 73.624304] niks3-server[997]: 2026/09/21 15:08:33 INFO Received uploads request method=POST path=/api/pending_closures4671lb # [ 73.633277] niks3-server[997]: 2026/09/21 15:08:33 INFO Received uploads request method=POST path=/api/pending_closures4672worker1 # [ 73.645856] nix-grpc-daemon[869]: time=2026-09-21T15:08:33.297Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4673worker1 # [ 73.647339] nix-grpc-daemon[869]: time=2026-09-21T15:08:33.297Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4674worker1 # [ 73.649213] nix-grpc-daemon[869]: time=2026-09-21T15:08:33.297Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4675lb # [ 73.663953] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4676lb # [ 73.678856] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4677lb # [ 73.693770] niks3-server[997]: 2026/09/21 15:08:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4678worker1 # [ 73.709595] nix-grpc-daemon[869]: time=2026-09-21T15:08:33.361Z level=INFO msg="Uploading 3 narinfos"4679lb # [ 73.708267] niks3-server[997]: 2026/09/21 15:08:33 INFO Signed narinfos id=18 count=14680lb # [ 73.719943] niks3-server[997]: 2026/09/21 15:08:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4681lb # [ 73.723565] niks3-server[997]: 2026/09/21 15:08:33 INFO Signed narinfos id=19 count=24682lb # [ 73.736888] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4683lb # [ 73.747967] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4684lb # [ 73.764522] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4685lb # [ 73.774545] niks3-server[997]: 2026/09/21 15:08:33 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4686lb # [ 73.778390] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4687lb # [ 73.781971] niks3-server[997]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4688lb # [ 73.784929] niks3-server[997]: 2026/09/21 15:08:33 INFO Completed upload id=184689lb # [ 73.787559] niks3-server[997]: 2026/09/21 15:08:33 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4690lb # [ 73.790707] niks3-server[997]: 2026/09/21 15:08:33 INFO Completed upload id=194691worker1 # [ 73.793355] nix-grpc-daemon[869]: time=2026-09-21T15:08:33.445Z level=INFO msg="Upload complete. (568ms)"4692worker1 # [ 73.804367] nix-grpc-daemon[776]: ts=2026-09-21T15:08:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=2 nar_bytes_in=9844693worker1 # [ 73.816983] nix-grpc-daemon[776]: ts=2026-09-21T15:08:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324694worker1 # [ 73.831423] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4695worker1 # [ 73.844380] nix-daemon[841]: reaped child process 945, status = succeeded4696client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4697client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4698worker1 # [ 73.858499] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4699client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4700worker1 # [ 74.707371] nix-daemon[841]: reaped child process 959, status = succeeded4701worker1 # [ 74.894345] nix-daemon[841]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4702worker1 # [ 74.911486] nix-daemon[841]: reaped child process 978, status = succeeded4703lb # [ 74.924965] niks3-server[997]: 2026/09/21 15:08:34 INFO Received uploads request method=POST path=/api/pending_closures4704worker1 # [ 74.937356] nix-grpc-daemon[869]: time=2026-09-21T15:08:34.589Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4705lb # [ 74.956685] niks3-server[997]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4706worker1 # [ 74.967900] nix-grpc-daemon[869]: time=2026-09-21T15:08:34.620Z level=INFO msg="Uploading 1 narinfos"4707lb # [ 74.968679] niks3-server[997]: 2026/09/21 15:08:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4708lb # [ 74.977980] niks3-server[997]: 2026/09/21 15:08:34 INFO Signed narinfos id=20 count=14709lb # [ 74.987648] niks3-server[997]: 2026/09/21 15:08:34 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4710lb # [ 74.992551] niks3-server[997]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4711lb # [ 74.996892] niks3-server[997]: 2026/09/21 15:08:34 INFO Completed upload id=204712worker1 # [ 74.999351] nix-grpc-daemon[869]: time=2026-09-21T15:08:34.651Z level=INFO msg="Upload complete. (326ms)"4713lb # [ 74.999959] niks3-server[997]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4714worker1 # [ 75.008926] nix-grpc-daemon[776]: ts=2026-09-21T15:08:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14715worker1 # [ 75.022420] nix-daemon[841]: reaped child process 957, status = succeeded4716client # copying 1 paths...4717worker2 # [ 75.101426] nix-grpc-daemon[777]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384718worker2 # [ 75.121256] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4719client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4720worker1 # [ 75.128108] nix-grpc-daemon[776]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324721worker1 # [ 75.134227] nix-grpc-daemon[776]: ts=2026-09-21T15:08:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1204722client # [ 75.224739] nix-daemon[940]: reaped child process 1026, status = succeeded4723client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.95 seconds)4724client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24725client # [ 75.439188] nix-daemon[940]: accepted connection from pid 1045, user root (trusted)4726client # this derivation will be built:4727client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4728worker2 # [ 75.691125] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:475384729client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4730worker1 # [ 75.651615] nix-grpc-daemon[776]: ts=2026-09-21T15:08:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324731worker1 # [ 75.663652] nix-grpc-daemon[869]: time=2026-09-21T15:08:35.316Z level=INFO msg="All 1 paths already cached"4732worker2 # [ 75.723420] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04733worker2 # [ 75.730737] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14734worker2 # [ 75.739388] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384735worker2 # [ 75.749524] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384736client # copying 1 paths to 'lb:50051'...4737client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4738worker2 # [ 75.768362] nix-grpc-daemon[777]: ts=2026-09-21T15:08:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475384739worker2 # [ 75.784153] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4740worker2 # [ 75.806678] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)4741worker2 # [ 75.835233] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)4742worker2 # [ 75.869758] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 231 ms (attempt 3/5)4743worker2 # [ 76.109425] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 710 ms (attempt 4/5)4744worker2 # [ 76.819106] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4745worker2 # [ 76.825649] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4746worker2 # [ 76.835238] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4747worker2 # [ 77.068595] nix-daemon[863]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)4748worker2 # [ 77.091119] nix-daemon[863]: reaped child process 1225, status = succeeded4749worker2 # [ 77.263544] nix-daemon[863]: accepted connection from pid 1227, user nix-grpc-daemon (trusted)4750worker2 # [ 77.284258] nix-daemon[863]: reaped child process 1230, status = succeeded4751lb # [ 77.251855] niks3-server[997]: 2026/09/21 15:08:36 INFO Received uploads request method=POST path=/api/pending_closures4752worker2 # [ 77.307270] nix-grpc-daemon[878]: time=2026-09-21T15:08:36.945Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4753worker2 # [ 77.308895] nix-grpc-daemon[878]: time=2026-09-21T15:08:36.945Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4754lb # [ 77.282310] niks3-server[997]: 2026/09/21 15:08:36 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4755worker2 # [ 77.339999] nix-grpc-daemon[878]: time=2026-09-21T15:08:36.978Z level=INFO msg="Uploading 1 narinfos"4756lb # [ 77.295322] niks3-server[997]: 2026/09/21 15:08:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4757lb # [ 77.306198] niks3-server[997]: 2026/09/21 15:08:36 INFO Signed narinfos id=21 count=14758lb # [ 77.314865] niks3-server[997]: 2026/09/21 15:08:36 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4759worker2 # [ 77.364282] nix-grpc-daemon[878]: time=2026-09-21T15:08:37.003Z level=INFO msg="Upload complete. (495ms)"4760lb # [ 77.319275] niks3-server[997]: 2026/09/21 15:08:36 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4761lb # [ 77.323725] niks3-server[997]: 2026/09/21 15:08:36 INFO Completed upload id=214762lb # [ 77.327255] niks3-server[997]: 2026/09/21 15:08:36 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4763worker2 # [ 77.373426] nix-grpc-daemon[777]: ts=2026-09-21T15:08:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 paths=1 nar_bytes_in=5684764worker2 # [ 77.385488] nix-grpc-daemon[777]: ts=2026-09-21T15:08:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475384765worker2 # [ 77.390882] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4766worker2 # [ 77.395071] nix-daemon[863]: reaped child process 1215, status = succeeded4767client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4768worker2 # [ 77.407343] nix-daemon[863]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4769client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4770client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4771worker2 # [ 78.453427] nix-daemon[863]: reaped child process 1234, status = succeeded4772worker2 # [ 78.632273] nix-daemon[863]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)4773worker2 # [ 78.647865] nix-daemon[863]: reaped child process 1253, status = succeeded4774lb # [ 78.617824] niks3-server[997]: 2026/09/21 15:08:38 INFO Received uploads request method=POST path=/api/pending_closures4775worker2 # [ 78.674239] nix-grpc-daemon[878]: time=2026-09-21T15:08:38.312Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4776lb # [ 78.648249] niks3-server[997]: 2026/09/21 15:08:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4777worker2 # [ 78.699864] nix-grpc-daemon[878]: time=2026-09-21T15:08:38.338Z level=INFO msg="Uploading 1 narinfos"4778lb # [ 78.660407] niks3-server[997]: 2026/09/21 15:08:38 INFO Signed narinfos id=22 count=14779lb # [ 78.669532] niks3-server[997]: 2026/09/21 15:08:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4780worker2 # [ 78.728741] nix-grpc-daemon[878]: time=2026-09-21T15:08:38.367Z level=INFO msg="Upload complete. (310ms)"4781lb # [ 78.685397] niks3-server[997]: 2026/09/21 15:08:38 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4782lb # [ 78.691879] niks3-server[997]: 2026/09/21 15:08:38 INFO Completed upload id=224783lb # [ 78.694729] niks3-server[997]: 2026/09/21 15:08:38 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4784worker2 # [ 78.740885] nix-grpc-daemon[777]: ts=2026-09-21T15:08:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47538 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14785lb # [ 78.699635] niks3-server[997]: 2026/09/21 15:08:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4786worker2 # [ 78.753024] nix-daemon[863]: reaped child process 1232, status = succeeded4787client # copying 1 paths...4788worker1 # [ 78.745342] nix-grpc-daemon[776]: ts=2026-09-21T15:08:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324789worker1 # [ 78.760396] nix-grpc-daemon[776]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4790client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4791worker1 # [ 78.805526] nix-grpc-daemon[776]: ts=2026-09-21T15:08:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324792worker1 # [ 78.811395] nix-grpc-daemon[776]: ts=2026-09-21T15:08:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1204793client # [ 78.942044] nix-daemon[940]: reaped child process 1048, status = succeeded4794client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.72 seconds)4795client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24796client # [ 79.162283] nix-daemon[940]: accepted connection from pid 1067, user root (trusted)4797client # this derivation will be built:4798client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4799worker2 # [ 79.416532] nix-grpc-daemon[777]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:475384800client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4801worker2 # [ 79.424340] nix-grpc-daemon[777]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475384802worker2 # [ 79.431246] nix-grpc-daemon[878]: time=2026-09-21T15:08:39.069Z level=INFO msg="All 1 paths already cached"4803worker2 # [ 79.443041] nix-grpc-daemon[777]: ts=2026-09-21T15:08:39Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04804worker1 # [ 79.403203] nix-grpc-daemon[776]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324805worker2 # [ 79.449557] nix-grpc-daemon[777]: ts=2026-09-21T15:08:39Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14806worker1 # [ 79.418262] nix-grpc-daemon[776]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324807client # copying 1 paths to 'lb:50051'...4808worker1 # [ 79.434441] nix-grpc-daemon[869]: time=2026-09-21T15:08:39.085Z level=INFO msg="All 1 paths already cached"4809client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4810worker1 # [ 79.439857] nix-grpc-daemon[776]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324811worker1 # [ 79.455392] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4812worker1 # [ 79.662715] nix-daemon[841]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4813worker1 # [ 79.681288] nix-daemon[841]: reaped child process 986, status = succeeded4814worker1 # [ 79.854714] nix-daemon[841]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4815worker1 # [ 79.870962] nix-daemon[841]: reaped child process 991, status = succeeded4816lb # [ 79.883655] niks3-server[997]: 2026/09/21 15:08:39 INFO Received uploads request method=POST path=/api/pending_closures4817worker1 # [ 79.893645] nix-grpc-daemon[869]: time=2026-09-21T15:08:39.545Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4818worker1 # [ 79.895129] nix-grpc-daemon[869]: time=2026-09-21T15:08:39.545Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4819lb # [ 79.909149] niks3-server[997]: 2026/09/21 15:08:39 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4820lb # [ 79.924225] niks3-server[997]: 2026/09/21 15:08:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4821worker1 # [ 79.933454] nix-grpc-daemon[869]: time=2026-09-21T15:08:39.585Z level=INFO msg="Uploading 1 narinfos"4822lb # [ 79.936867] niks3-server[997]: 2026/09/21 15:08:39 INFO Signed narinfos id=23 count=14823lb # [ 79.943131] niks3-server[997]: 2026/09/21 15:08:39 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4824lb # [ 79.950947] niks3-server[997]: 2026/09/21 15:08:39 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4825worker1 # [ 79.961247] nix-grpc-daemon[869]: time=2026-09-21T15:08:39.613Z level=INFO msg="Upload complete. (488ms)"4826lb # [ 79.959069] niks3-server[997]: 2026/09/21 15:08:39 INFO Completed upload id=234827lb # [ 79.963000] niks3-server[997]: 2026/09/21 15:08:39 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4828worker1 # [ 79.968722] nix-grpc-daemon[776]: ts=2026-09-21T15:08:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=5684829worker1 # [ 79.981594] nix-grpc-daemon[776]: ts=2026-09-21T15:08:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324830worker1 # [ 79.987488] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4831worker1 # [ 79.992723] nix-daemon[841]: reaped child process 981, status = succeeded4832worker1 # [ 80.004189] nix-daemon[841]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4833client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4834client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4835lb # [ 81.158051] envoy[914]: conn peer=[2001:db8:1::1]:41336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4216 tx=1983 ms=24344836lb # [ 81.163153] envoy[914]: conn peer=[2001:db8:1::1]:41340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=264837lb # [ 81.168068] envoy[914]: conn peer=[2001:db8:1::1]:41354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3765 tx=2015 ms=32224838lb # [ 81.173699] envoy[914]: conn peer=[2001:db8:1::1]:41366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=714839client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4840worker1 # [ 81.658304] nix-daemon[841]: reaped child process 995, status = succeeded4841worker1 # [ 81.859828] nix-daemon[841]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4842worker1 # [ 81.879849] nix-daemon[841]: reaped child process 1014, status = succeeded4843lb # [ 81.891415] niks3-server[997]: 2026/09/21 15:08:41 INFO Received uploads request method=POST path=/api/pending_closures4844worker1 # [ 81.913938] nix-grpc-daemon[869]: time=2026-09-21T15:08:41.564Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4845lb # [ 81.936615] niks3-server[997]: 2026/09/21 15:08:41 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4846worker1 # [ 81.946986] nix-grpc-daemon[869]: time=2026-09-21T15:08:41.599Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 81.946545] niks3-server[997]: 2026/09/21 15:08:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4848lb # [ 81.957351] niks3-server[997]: 2026/09/21 15:08:41 INFO Signed narinfos id=24 count=14849lb # [ 81.962790] niks3-server[997]: 2026/09/21 15:08:41 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4850lb # [ 81.970543] niks3-server[997]: 2026/09/21 15:08:41 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4851lb # [ 81.978536] niks3-server[997]: 2026/09/21 15:08:41 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4852lb # [ 81.984445] niks3-server[997]: 2026/09/21 15:08:41 INFO Completed upload id=244853worker1 # [ 81.987302] nix-grpc-daemon[869]: time=2026-09-21T15:08:41.639Z level=INFO msg="Upload complete. (369ms)"4854client # copying 1 paths...4855worker1 # [ 82.001886] nix-grpc-daemon[776]: ts=2026-09-21T15:08:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=2 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14856worker2 # [ 82.050614] nix-grpc-daemon[777]: ts=2026-09-21T15:08:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:475384857worker1 # [ 82.024435] nix-daemon[841]: reaped child process 993, status = succeeded4858worker2 # [ 82.068997] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4859client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4860worker1 # [ 82.080441] nix-grpc-daemon[776]: ts=2026-09-21T15:08:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324861worker1 # [ 82.089646] nix-grpc-daemon[776]: ts=2026-09-21T15:08:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1204862client # [ 82.242124] nix-daemon[940]: reaped child process 1070, status = succeeded4863client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.29 seconds)4864(finished: subtest: an input only one worker has still reaches the builder, in 10.88 seconds)4865subtest: upload whose reference the worker lacks is completed from the cache4866client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4867client # this derivation will be built:4868client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4869client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4870client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.58 seconds)4871client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4872client # this derivation will be built:4873client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4874client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4875client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.50 seconds)4876client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4877client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.27 seconds)4878worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4879worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.34 seconds)4880worker2 # [ 84.097412] systemd[1]: Stopping Nix worker-protocol over gRPC...4881worker2 # [ 84.703717] nix-grpc-daemon[777]: ts=2026-09-21T15:08:44Z level=info event=signal_exit4882lb # [ 84.908697] niks3-server[997]: 2026/09/21 15:08:44 INFO lead: released remote=192.168.1.4:380364883worker1 # [ 85.909042] nix-grpc-daemon[776]: ts=2026-09-21T15:08:45Z level=info event=scheduler_restarting addr=lb:500514884worker2 # [ 85.951955] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=scheduler_restarting4885worker2 # [ 85.961251] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:602884886worker2 # [ 85.968804] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=worker_gone addr=192.168.1.3:500514887worker2 # [ 85.973180] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=scheduler_restarting addr=lb:500514888worker2 # [ 85.977730] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:602884889worker2 # [ 85.982632] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=worker_gone addr=192.168.1.4:500514890worker2 # [ 86.015038] nix-daemon[863]: reaped child process 865, status = succeeded4891worker1 # [ 86.108675] nix-grpc-daemon[776]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500514892worker1 # [ 86.113664] nix-grpc-daemon[776]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4893worker2 # [ 86.183729] nix-grpc-daemon[777]: ts=2026-09-21T15:08:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4894worker1 # [ 86.314419] nix-grpc-daemon[776]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500514895worker1 # [ 86.319614] nix-grpc-daemon[776]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4896worker1 # [ 86.520365] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500514897worker1 # [ 86.525226] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4898worker1 # [ 86.726093] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500514899worker1 # [ 86.731204] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4900worker2 # [ 86.790152] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4901worker2 # [ 86.799200] systemd[1]: Stopped Nix worker-protocol over gRPC.4902worker2 # [ 86.804436] systemd[1]: nix-grpc-daemon.service: Consumed 3.776s CPU time over 1min 5.431s wall clock time, 65.9M memory peak, 100.4K incoming IP traffic, 258K outgoing IP traffic.4903worker2 # [ 86.818827] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4904worker2 # [ 86.822876] systemd[1]: Closed Nix worker-protocol over gRPC.4905worker2 # [ 86.825455] systemd[1]: nix-grpc-daemon.socket: Consumed 627us CPU time over 1min 14.917s wall clock time, 256K memory peak, 93.6K incoming IP traffic, 77.4K outgoing IP traffic.4906lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4907lb: must succeed: curl -sf localhost:9901/clusters4908lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4909lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4910client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24911worker1 # [ 86.933993] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500514912worker1 # [ 86.938024] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4913worker1 # [ 87.134537] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500514914worker1 # [ 87.140785] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324915worker1 # [ 87.148216] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4916client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4917worker1 # [ 87.155213] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324918client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4919worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4920worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4921worker2 # [ 87.329738] systemd[1]: Listening on Nix worker-protocol over gRPC.4922worker1 # [ 87.347653] nix-grpc-daemon[776]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500514923worker1 # [ 87.352887] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4924worker1 # [ 87.384471] systemd[1]: Stopping Nix worker-protocol over gRPC...4925worker1 # [ 87.399985] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_take_over4926lb # [ 87.396600] niks3-server[997]: 2026/09/21 15:08:47 INFO lead: acquired remote=192.168.1.3:492984927worker1 # [ 87.553503] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500514928worker1 # [ 87.559192] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=signal_exit4929worker1 # [ 87.563232] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4930lb # [ 87.590369] niks3-server[997]: 2026/09/21 15:08:47 INFO lead: released remote=192.168.1.3:492984931worker1 # [ 87.759941] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500514932worker1 # [ 87.765017] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4933worker2 # [ 87.835463] systemd[1]: Starting scheduler-token.service...4934worker2 # [ 87.942988] systemd[1]: scheduler-token.service: Deactivated successfully.4935worker2 # [ 87.948971] systemd[1]: Finished scheduler-token.service.4936worker2 # [ 87.950537] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 111ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4937worker2 # [ 87.959092] systemd[1]: Starting Nix worker-protocol over gRPC...4938worker1 # [ 87.967526] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500514939worker1 # [ 87.971669] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4940worker2 # [ 88.149574] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4941worker2 # [ 88.152174] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4942worker2 # [ 88.156608] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14943worker2 # [ 88.164077] nix-grpc-daemon[1280]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4944worker2 # [ 88.165553] nix-grpc-daemon[1280]: E0000 00:00:1790003327.802183 1280 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4945worker2 # [ 88.177931] nix-grpc-daemon[1280]: E0000 00:00:1790003327.808709 1280 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4946worker2 # [ 88.183263] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4947worker2 # [ 88.187122] systemd[1]: Started Nix worker-protocol over gRPC.4948lb # [ 88.143626] niks3-server[997]: 2026/09/21 15:08:47 INFO lead: acquired remote=192.168.1.4:450964949worker2 # [ 88.192087] nix-grpc-daemon[1280]: E0000 00:00:1790003327.830780 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4950worker1 # [ 88.168184] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500514951worker1 # [ 88.173161] nix-grpc-daemon[776]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4952worker2 # [ 88.224176] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=scheduler_take_over4953worker2 # [ 88.225397] nix-grpc-daemon[1280]: E0000 00:00:1790003327.849350 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4954worker2 # [ 88.229937] nix-grpc-daemon[1280]: E0000 00:00:1790003327.849496 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4955worker2 # [ 88.232835] nix-grpc-daemon[1280]: E0000 00:00:1790003327.868093 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4956worker2 # [ 88.242391] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500514957worker2 # [ 88.244136] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4958worker1 # [ 88.373909] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514959worker1 # [ 88.378991] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4960worker2 # [ 88.445671] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514961worker2 # [ 88.450752] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4962worker1 # [ 88.579640] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514963worker1 # [ 88.584754] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4964worker1 # [ 88.594418] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=scheduler_restarting4965worker2 # [ 88.651789] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514966worker2 # [ 88.657215] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4967worker1 # [ 88.635547] nix-daemon[841]: reaped child process 843, status = succeeded4968worker1 # [ 88.669053] nix-grpc-daemon[776]: E0000 00:00:1790003328.321246 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4969worker1 # [ 88.682150] nix-grpc-daemon[776]: E0000 00:00:1790003328.334475 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4970worker1 # [ 88.809029] nix-grpc-daemon[776]: ts=2026-09-21T15:08:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4971worker2 # [ 88.858355] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514972worker2 # [ 88.863605] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4973worker2 # [ 89.064658] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514974worker2 # [ 89.069703] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4975worker2 # [ 89.272244] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=scheduler_connected addr=lb:500514976worker2 # [ 89.290138] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=oidc_jwks provider=mock keys=14977worker2 # [ 89.294846] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570604978worker2 # [ 89.300265] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04979worker1 # [ 90.679805] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4980worker1 # [ 90.683044] systemd[1]: Stopped Nix worker-protocol over gRPC.4981worker1 # [ 90.689180] systemd[1]: nix-grpc-daemon.service: Consumed 1.901s CPU time over 1min 9.518s wall clock time, 51.7M memory peak, 56.9K incoming IP traffic, 102.2K outgoing IP traffic.4982worker1 # [ 90.703570] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4983worker1 # [ 90.709076] systemd[1]: Closed Nix worker-protocol over gRPC.4984worker1 # [ 90.710700] systemd[1]: nix-grpc-daemon.socket: Consumed 628us CPU time over 1min 18.835s wall clock time, 256K memory peak, 66K incoming IP traffic, 58.2K outgoing IP traffic.4985lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4986lb: must succeed: curl -sf localhost:9901/clusters4987lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4988lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4989client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24990worker2 # [ 91.116450] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358184991worker2 # [ 91.126356] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)4992worker2 # [ 91.166361] nix-grpc-daemon[1280]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 1/5)4993lb # [ 91.155692] envoy[914]: conn peer=[2001:db8:1::1]:47578 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=754994lb # [ 91.160580] envoy[914]: conn peer=[2001:db8:1::1]:41380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3766 tx=2012 ms=27804995lb # [ 91.165743] envoy[914]: conn peer=[2001:db8:1::4]:47840 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=62036 tx=3440 ms=644274996lb # [ 91.170734] envoy[914]: conn peer=[2001:db8:1::1]:47586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=304997lb # [ 91.176116] envoy[914]: conn peer=[2001:db8:1::3]:54862 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13536 tx=3524 ms=671964998worker2 # [ 91.249725] nix-grpc-daemon[1280]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)4999worker2 # [ 91.260136] nix-grpc-daemon[1280]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 3/5)5000worker2 # [ 91.313525] nix-grpc-daemon[1280]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 4/5)5001worker2 # [ 91.470496] nix-grpc-daemon[1280]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5002worker2 # [ 91.477449] nix-grpc-daemon[1280]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5003client # copying 2 paths...5004client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5005worker2 # [ 91.503677] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358185006client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5007worker2 # [ 91.522411] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5008worker2 # [ 91.551956] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:51Z level=info event=niks3_push_started pid=13405009worker2 # [ 91.769613] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.407Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5010worker2 # [ 91.944707] nix-daemon[863]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5011worker2 # [ 91.962585] nix-daemon[863]: reaped child process 1356, status = succeeded5012lb # [ 91.933598] niks3-server[997]: 2026/09/21 15:08:51 INFO Received uploads request method=POST path=/api/pending_closures5013lb # [ 91.944105] niks3-server[997]: 2026/09/21 15:08:51 INFO Received uploads request method=POST path=/api/pending_closures5014worker2 # [ 91.999696] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.636Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5015worker2 # [ 92.002143] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.636Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5016worker2 # [ 92.004112] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.636Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5017lb # [ 92.016649] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5018lb # [ 92.034476] niks3-server[997]: 2026/09/21 15:08:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5019worker2 # [ 92.088249] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.726Z level=INFO msg="Uploading 3 narinfos"5020lb # [ 92.044888] niks3-server[997]: 2026/09/21 15:08:51 INFO Signed narinfos id=25 count=15021lb # [ 92.046481] niks3-server[997]: 2026/09/21 15:08:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5022lb # [ 92.061795] niks3-server[997]: 2026/09/21 15:08:51 INFO Signed narinfos id=26 count=25023lb # [ 92.066025] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5024lb # [ 92.073051] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5025lb # [ 92.075264] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5026lb # [ 92.087755] niks3-server[997]: 2026/09/21 15:08:51 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5027lb # [ 92.093780] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5028lb # [ 92.096530] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5029lb # [ 92.100220] niks3-server[997]: 2026/09/21 15:08:51 INFO Completed upload id=255030lb # [ 92.101491] niks3-server[997]: 2026/09/21 15:08:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5031lb # [ 92.103531] niks3-server[997]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5032lb # [ 92.106801] niks3-server[997]: 2026/09/21 15:08:51 INFO Completed upload id=265033worker2 # [ 92.152570] nix-grpc-daemon[1340]: time=2026-09-21T15:08:51.791Z level=INFO msg="Upload complete. (379ms)"5034worker2 # [ 92.165444] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=0 paths=2 nar_bytes_in=3125035worker2 # [ 92.183812] nix-daemon[863]: reaped child process 1338, status = succeeded5036client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.36 seconds)5037worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5038worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5039worker1 # [ 92.280919] systemd[1]: Listening on Nix worker-protocol over gRPC.5040worker1 # [ 92.287675] systemd[1]: Starting Nix worker-protocol over gRPC...5041worker1 # [ 92.482762] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5042worker1 # [ 92.486145] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5043worker1 # [ 92.490490] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15044worker1 # [ 92.499401] nix-grpc-daemon[1037]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5045worker1 # [ 92.507481] nix-grpc-daemon[1037]: E0000 00:00:1790003332.149520 1037 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5046worker1 # [ 92.512392] nix-grpc-daemon[1037]: E0000 00:00:1790003332.151740 1037 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5047worker1 # [ 92.516468] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5048lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5049lb: must succeed: curl -sf localhost:9901/clusters5050worker1 # [ 92.528153] systemd[1]: Started Nix worker-protocol over gRPC.5051worker1 # [ 92.529045] nix-grpc-daemon[1037]: E0000 00:00:1790003332.171571 1077 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5052worker1 # [ 92.553876] nix-grpc-daemon[1037]: E0000 00:00:1790003332.190878 1077 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5053worker1 # [ 92.556714] nix-grpc-daemon[1037]: E0000 00:00:1790003332.191059 1077 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5054worker1 # [ 92.559167] nix-grpc-daemon[1037]: E0000 00:00:1790003332.205435 1077 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5055worker1 # [ 92.567805] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:52Z level=info event=scheduler_connected addr=lb:500515056worker2 # [ 92.612118] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570605057worker2 # [ 92.617997] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05058lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5059lb: must succeed: curl -sf localhost:9901/clusters5060lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5061lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5062(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.44 seconds)5063subtest: a niks3 restart does not cost the leader its role5064lb: must succeed: curl -sf localhost:9901/clusters5065lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5066lb: must succeed: curl -sf localhost:9901/clusters5067lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5068lb # [ 93.835745] systemd[1]: Stopping niks3 server...5069lb # [ 93.838037] niks3-server[997]: 2026/09/21 15:08:53 INFO Shutdown signal received, draining in-flight requests timeout=10s5070worker1 # [ 93.844579] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:53Z level=info event=scheduler_lock_lost status=2005071worker2 # [ 93.887485] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:53Z level=info event=scheduler_lock_lost status=2005072lb # [ 93.845785] niks3-server[997]: 2026/09/21 15:08:53 INFO lead: released remote=192.168.1.4:450965073lb # [ 97.091957] systemd[1]: niks3.service: Deactivated successfully.5074lb # [ 97.097328] systemd[1]: Stopped niks3 server.5075lb # [ 97.104216] systemd[1]: niks3.service: Consumed 533ms CPU time over 1min 10.051s wall clock time, 22M memory peak, 70.2K incoming IP traffic, 101.2K outgoing IP traffic.5076lb # [ 97.125001] systemd[1]: Starting niks3 server...5077lb # [ 97.247658] niks3-server[1135]: 2026/09/21 15:08:56 goose: no migrations to run. current version: 202609200000005078lb # [ 97.254158] niks3-server[1135]: 2026/09/21 15:08:56 OK 1_commit_pending_closure.sql (8.74ms)5079lb # [ 97.258862] niks3-server[1135]: 2026/09/21 15:08:56 OK 2_object_stats_trigger.sql (2.62ms)5080lb # [ 97.260493] niks3-server[1135]: 2026/09/21 15:08:56 goose: up to current file version: 25081lb # [ 97.262685] niks3-server[1135]: 2026/09/21 15:08:56 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key5082lb # [ 97.276224] niks3-server[1135]: 2026/09/21 15:08:56 INFO Read proxy enabled — serving cache objects from S35083lb # [ 97.277800] niks3-server[1135]: 2026/09/21 15:08:56 INFO Using socket-activated listener address=0.0.0.0:57515084lb # [ 97.284456] niks3-server[1135]: 2026/09/21 15:08:56 INFO systemd watchdog enabled interval=15s5085lb # [ 97.286006] niks3-server[1135]: 2026/09/21 15:08:56 INFO Starting HTTP server address=0.0.0.0:57515086lb # [ 97.287479] niks3-server[1135]: 2026/09/21 15:08:56 INFO lead: acquired remote=192.168.1.4:534085087lb # [ 97.289967] systemd[1]: Started niks3 server.5088lb: waiting for unit niks3.service5089lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5090lb: must succeed: curl -sf localhost:9901/clusters5091lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5092worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5093worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5094lb: must succeed: curl -sf localhost:9901/clusters5095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5096lb: must succeed: curl -sf localhost:9901/clusters5097lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5098worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5099worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5100client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&25101worker1 # [ 97.899801] nix-grpc-daemon[1037]: ts=2026-09-21T15:08:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:497605102worker1 # [ 97.915852] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5103client # don't know how to build these paths:5104client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5105worker2 # [ 97.982486] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358185106worker2 # [ 98.026331] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05107worker2 # [ 98.033537] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15108worker2 # [ 98.076708] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185109worker2 # [ 98.091066] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358185110client # copying 1 paths to 'lb:50051'...5111client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5112worker2 # [ 98.105492] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358185113worker2 # [ 98.119908] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5114worker2 # [ 98.334687] nix-daemon[863]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5115worker2 # [ 98.354127] nix-daemon[863]: reaped child process 1378, status = succeeded5116worker2 # [ 98.529459] nix-daemon[863]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5117worker2 # [ 98.545798] nix-daemon[863]: reaped child process 1383, status = succeeded5118lb # [ 98.512938] niks3-server[1135]: 2026/09/21 15:08:58 INFO Received uploads request method=POST path=/api/pending_closures5119worker2 # [ 98.573759] nix-grpc-daemon[1340]: time=2026-09-21T15:08:58.212Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5120worker2 # [ 98.575225] nix-grpc-daemon[1340]: time=2026-09-21T15:08:58.212Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5121lb # [ 98.558648] niks3-server[1135]: 2026/09/21 15:08:58 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5122lb # [ 98.576392] niks3-server[1135]: 2026/09/21 15:08:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5123lb # [ 98.584944] niks3-server[1135]: 2026/09/21 15:08:58 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5124lb # [ 98.595217] niks3-server[1135]: 2026/09/21 15:08:58 INFO Signed narinfos id=27 count=15125worker2 # [ 98.643600] nix-grpc-daemon[1340]: time=2026-09-21T15:08:58.282Z level=INFO msg="Uploading 1 narinfos"5126lb # [ 98.603848] niks3-server[1135]: 2026/09/21 15:08:58 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5127lb # [ 98.610648] niks3-server[1135]: 2026/09/21 15:08:58 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5128lb # [ 98.619524] niks3-server[1135]: 2026/09/21 15:08:58 INFO Completed upload id=275129worker2 # [ 98.665648] nix-grpc-daemon[1340]: time=2026-09-21T15:08:58.304Z level=INFO msg="Upload complete. (527ms)"5130worker2 # [ 98.676319] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=0 paths=1 nar_bytes_in=5445131worker2 # [ 98.692602] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185132worker2 # [ 98.700165] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5133worker2 # [ 98.705709] nix-daemon[863]: reaped child process 1373, status = succeeded5134client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5135client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5136worker2 # [ 98.721259] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5137client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5138client # farm-a> LOG-farm-a-after-niks3-restart5139worker2 # [ 99.562044] nix-daemon[863]: reaped child process 1387, status = succeeded5140worker2 # [ 99.737849] nix-daemon[863]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5141worker2 # [ 99.757262] nix-daemon[863]: reaped child process 1406, status = succeeded5142lb # [ 99.724694] niks3-server[1135]: 2026/09/21 15:08:59 INFO Received uploads request method=POST path=/api/pending_closures5143worker2 # [ 99.786195] nix-grpc-daemon[1340]: time=2026-09-21T15:08:59.424Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5144worker2 # [ 99.790239] nix-grpc-daemon[1340]: time=2026-09-21T15:08:59.427Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5145lb # [ 99.766688] niks3-server[1135]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5146lb # [ 99.775725] niks3-server[1135]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5147worker2 # [ 99.830339] nix-grpc-daemon[1340]: time=2026-09-21T15:08:59.468Z level=INFO msg="Uploading 1 narinfos"5148lb # [ 99.786225] niks3-server[1135]: 2026/09/21 15:08:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5149lb # [ 99.792166] niks3-server[1135]: 2026/09/21 15:08:59 INFO Signed narinfos id=28 count=15150lb # [ 99.794012] niks3-server[1135]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5151lb # [ 99.806812] niks3-server[1135]: 2026/09/21 15:08:59 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5152lb # [ 99.811764] niks3-server[1135]: 2026/09/21 15:08:59 INFO Completed upload id=285153worker2 # [ 99.856778] nix-grpc-daemon[1340]: time=2026-09-21T15:08:59.495Z level=INFO msg="Upload complete. (324ms)"5154lb # [ 99.813901] niks3-server[1135]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5155worker2 # [ 99.876969] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15156worker2 # [ 99.888567] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05157worker2 # [ 99.893854] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15158worker2 # [ 99.897686] nix-daemon[863]: reaped child process 1385, status = succeeded5159worker2 # [ 99.921116] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185160worker2 # [ 99.935763] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358185161worker2 # [ 99.950192] nix-grpc-daemon[1340]: time=2026-09-21T15:08:59.589Z level=INFO msg="All 1 paths already cached"5162client # copying 1 paths to 'lb:50051'...5163client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5164worker2 # [ 99.960106] nix-grpc-daemon[1280]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358185165worker2 # [ 99.973935] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5166worker2 # [ 100.170228] nix-daemon[863]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5167worker2 # [ 100.188183] nix-daemon[863]: reaped child process 1415, status = succeeded5168worker2 # [ 100.362470] nix-daemon[863]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5169worker2 # [ 100.378470] nix-daemon[863]: reaped child process 1420, status = succeeded5170lb # [ 100.344474] niks3-server[1135]: 2026/09/21 15:08:59 INFO Received uploads request method=POST path=/api/pending_closures5171worker2 # [ 100.399604] nix-grpc-daemon[1340]: time=2026-09-21T15:09:00.038Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5172worker2 # [ 100.401273] nix-grpc-daemon[1340]: time=2026-09-21T15:09:00.038Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5173lb # [ 100.371752] niks3-server[1135]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5174worker2 # [ 100.438157] nix-grpc-daemon[1340]: time=2026-09-21T15:09:00.076Z level=INFO msg="Uploading 1 narinfos"5175lb # [ 100.384833] niks3-server[1135]: 2026/09/21 15:09:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5176lb # [ 100.396158] niks3-server[1135]: 2026/09/21 15:09:00 INFO Signed narinfos id=29 count=15177lb # [ 100.401773] niks3-server[1135]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5178lb # [ 100.409815] niks3-server[1135]: 2026/09/21 15:09:00 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5179lb # [ 100.415915] niks3-server[1135]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5180lb # [ 100.421358] niks3-server[1135]: 2026/09/21 15:09:00 INFO Completed upload id=295181worker2 # [ 100.466781] nix-grpc-daemon[1340]: time=2026-09-21T15:09:00.105Z level=INFO msg="Upload complete. (476ms)"5182worker2 # [ 100.474248] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=0 paths=1 nar_bytes_in=6565183worker2 # [ 100.484693] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185184worker2 # [ 100.494888] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5185worker2 # [ 100.499503] nix-daemon[863]: reaped child process 1410, status = succeeded5186client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5187worker2 # [ 100.512192] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5188client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5189lb # [ 101.160221] envoy[914]: conn peer=[2001:db8:1::1]:40106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=10655190client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5191client # farm-b> LOG-farm-b-after-niks3-restart5192worker2 # [ 101.957585] nix-daemon[863]: reaped child process 1424, status = succeeded5193worker2 # [ 102.126889] nix-daemon[863]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5194worker2 # [ 102.145698] nix-daemon[863]: reaped child process 1443, status = succeeded5195lb # [ 102.114532] niks3-server[1135]: 2026/09/21 15:09:01 INFO Received uploads request method=POST path=/api/pending_closures5196worker2 # [ 102.170873] nix-grpc-daemon[1340]: time=2026-09-21T15:09:01.809Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5197worker2 # [ 102.173269] nix-grpc-daemon[1340]: time=2026-09-21T15:09:01.811Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5198lb # [ 102.150369] niks3-server[1135]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5199worker2 # [ 102.219001] nix-grpc-daemon[1340]: time=2026-09-21T15:09:01.857Z level=INFO msg="Uploading 1 narinfos"5200lb # [ 102.167902] niks3-server[1135]: 2026/09/21 15:09:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5201lb # [ 102.178983] niks3-server[1135]: 2026/09/21 15:09:01 INFO Signed narinfos id=30 count=15202lb # [ 102.184826] niks3-server[1135]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5203lb # [ 102.195174] niks3-server[1135]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5204lb # [ 102.198788] niks3-server[1135]: 2026/09/21 15:09:01 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5205lb # [ 102.202997] niks3-server[1135]: 2026/09/21 15:09:01 INFO Completed upload id=305206worker2 # [ 102.248456] nix-grpc-daemon[1340]: time=2026-09-21T15:09:01.887Z level=INFO msg="Upload complete. (325ms)"5207lb # [ 102.205615] niks3-server[1135]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5208worker2 # [ 102.263300] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15209worker2 # [ 102.277284] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05210worker2 # [ 102.281048] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15211worker2 # [ 102.284865] nix-daemon[863]: reaped child process 1422, status = succeeded5212worker2 # [ 102.309640] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185213worker2 # [ 102.323632] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358185214worker2 # [ 102.337884] nix-grpc-daemon[1340]: time=2026-09-21T15:09:01.976Z level=INFO msg="All 2 paths already cached"5215client # copying 1 paths to 'lb:50051'...5216client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5217worker2 # [ 102.348398] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358185218worker2 # [ 102.363313] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5219worker2 # [ 102.563505] nix-daemon[863]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)5220worker2 # [ 102.579755] nix-daemon[863]: reaped child process 1452, status = succeeded5221worker2 # [ 102.750775] nix-daemon[863]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)5222worker2 # [ 102.766466] nix-daemon[863]: reaped child process 1457, status = succeeded5223lb # [ 102.736504] niks3-server[1135]: 2026/09/21 15:09:02 INFO Received uploads request method=POST path=/api/pending_closures5224worker2 # [ 102.791572] nix-grpc-daemon[1340]: time=2026-09-21T15:09:02.430Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5225worker2 # [ 102.793277] nix-grpc-daemon[1340]: time=2026-09-21T15:09:02.430Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5226lb # [ 102.765316] niks3-server[1135]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5227lb # [ 102.777704] niks3-server[1135]: 2026/09/21 15:09:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5228worker2 # [ 102.830668] nix-grpc-daemon[1340]: time=2026-09-21T15:09:02.469Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 102.787378] niks3-server[1135]: 2026/09/21 15:09:02 INFO Signed narinfos id=31 count=15230lb # [ 102.793460] niks3-server[1135]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5231lb # [ 102.802339] niks3-server[1135]: 2026/09/21 15:09:02 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5232lb # [ 102.809300] niks3-server[1135]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5233lb # [ 102.813478] niks3-server[1135]: 2026/09/21 15:09:02 INFO Completed upload id=315234worker2 # [ 102.859042] nix-grpc-daemon[1340]: time=2026-09-21T15:09:02.498Z level=INFO msg="Upload complete. (477ms)"5235worker2 # [ 102.866879] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=0 paths=1 nar_bytes_in=7845236worker2 # [ 102.876655] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358185237worker2 # [ 102.886502] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5238worker2 # [ 102.890745] nix-daemon[863]: reaped child process 1447, status = succeeded5239client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5240worker2 # [ 102.901453] nix-daemon[863]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5241client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5242client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5243client # farm-top> LOG-farm-top-after-niks3-restart5244worker2 # [ 104.136155] nix-daemon[863]: reaped child process 1461, status = succeeded5245worker2 # [ 104.305786] nix-daemon[863]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)5246worker2 # [ 104.322411] nix-daemon[863]: reaped child process 1480, status = succeeded5247lb # [ 104.289731] niks3-server[1135]: 2026/09/21 15:09:03 INFO Received uploads request method=POST path=/api/pending_closures5248worker2 # [ 104.347352] nix-grpc-daemon[1340]: time=2026-09-21T15:09:03.984Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5249worker2 # [ 104.349130] nix-grpc-daemon[1340]: time=2026-09-21T15:09:03.984Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5250lb # [ 104.317683] niks3-server[1135]: 2026/09/21 15:09:03 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5251lb # [ 104.335193] niks3-server[1135]: 2026/09/21 15:09:03 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5252lb # [ 104.350418] niks3-server[1135]: 2026/09/21 15:09:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5253worker2 # [ 104.402355] nix-grpc-daemon[1340]: time=2026-09-21T15:09:04.040Z level=INFO msg="Uploading 1 narinfos"5254lb # [ 104.362998] niks3-server[1135]: 2026/09/21 15:09:03 INFO Signed narinfos id=32 count=15255lb # [ 104.370045] niks3-server[1135]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5256lb # [ 104.379445] niks3-server[1135]: 2026/09/21 15:09:04 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5257lb # [ 104.384797] niks3-server[1135]: 2026/09/21 15:09:04 INFO Completed upload id=325258worker2 # [ 104.429701] nix-grpc-daemon[1340]: time=2026-09-21T15:09:04.068Z level=INFO msg="Upload complete. (316ms)"5259lb # [ 104.387720] niks3-server[1135]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5260worker2 # [ 104.445149] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35818 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15261worker2 # [ 104.460537] nix-daemon[863]: reaped child process 1459, status = succeeded5262client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 6.81 seconds)5263client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath5264client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.24 seconds)5265(finished: subtest: a niks3 restart does not cost the leader its role, in 11.00 seconds)5266subtest: scheduler down: the other node takes the lock and keeps it5267lb: must succeed: curl -sf localhost:9901/clusters5268lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5269lb: must succeed: curl -sf localhost:9901/clusters5270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5271lb: must succeed: curl -sf localhost:9901/clusters5272lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5273lb: must succeed: curl -sf localhost:9901/clusters5274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5275worker2 # [ 105.045057] systemd[1]: Stopping Nix worker-protocol over gRPC...5276worker2 # [ 105.183287] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:04Z level=info event=signal_exit5277lb # [ 106.069320] niks3-server[1135]: 2026/09/21 15:09:05 INFO lead: released remote=192.168.1.4:534085278worker1 # [ 107.072387] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:06Z level=info event=scheduler_restarting addr=lb:500515279worker2 # [ 107.118474] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=scheduler_restarting5280worker2 # [ 107.122799] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=scheduler_restarting addr=lb:500515281worker2 # [ 107.127448] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570605282worker2 # [ 107.132300] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=worker_gone addr=192.168.1.4:500515283worker2 # [ 107.137113] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570605284worker2 # [ 107.142908] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=worker_gone addr=192.168.1.3:500515285worker2 # [ 107.165009] nix-daemon[863]: reaped child process 1332, status = succeeded5286worker2 # [ 107.209490] nix-grpc-daemon[1280]: E0000 00:00:1790003346.848339 1325 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5287worker2 # [ 107.238672] nix-grpc-daemon[1280]: E0000 00:00:1790003346.877489 1325 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5288worker1 # [ 107.271509] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:06Z level=info event=scheduler_connected addr=lb:500515289worker1 # [ 107.276970] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker1 # [ 107.285700] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:06Z level=info event=scheduler_take_over5291lb # [ 107.285743] niks3-server[1135]: 2026/09/21 15:09:06 INFO lead: acquired remote=192.168.1.3:605005292worker2 # [ 107.333684] nix-grpc-daemon[1280]: ts=2026-09-21T15:09:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5293worker1 # [ 107.477612] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=scheduler_connected addr=lb:500515294worker1 # [ 107.482843] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5295worker2 # [ 107.553350] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5296worker2 # [ 107.561917] systemd[1]: Stopped Nix worker-protocol over gRPC.5297worker2 # [ 107.567986] systemd[1]: nix-grpc-daemon.service: Consumed 1.544s CPU time over 19.590s wall clock time, 36.1M memory peak, 47.1K incoming IP traffic, 81.8K outgoing IP traffic.5298worker2 # [ 107.580412] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5299worker2 # [ 107.583395] systemd[1]: Closed Nix worker-protocol over gRPC.5300worker2 # [ 107.586057] systemd[1]: nix-grpc-daemon.socket: Consumed 777us CPU time over 20.256s wall clock time, 256K memory peak, 40.8K incoming IP traffic, 31K outgoing IP traffic.5301worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5302worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5303worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5304worker1 # [ 107.683320] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=scheduler_connected addr=lb:500515305worker1 # [ 107.687980] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5307worker1 # [ 107.887158] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=scheduler_connected addr=lb:500515308worker1 # [ 107.896583] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:464385309worker1 # [ 107.902070] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05310worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5311worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5312client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25313worker1 # [ 109.037883] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:497605314client # don't know how to build these paths:5315client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5316worker1 # [ 109.047578] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497605317worker1 # [ 109.069210] nix-grpc-daemon[1037]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5318worker1 # [ 109.094711] nix-grpc-daemon[1037]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 2/5)5319worker1 # [ 109.142179] nix-grpc-daemon[1037]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 291 ms (attempt 3/5)5320worker1 # [ 109.440924] nix-grpc-daemon[1037]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 767 ms (attempt 4/5)5321worker1 # [ 110.210208] nix-grpc-daemon[1037]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5322worker1 # [ 110.215043] nix-grpc-daemon[1037]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5323worker1 # [ 110.248159] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:09Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05324worker1 # [ 110.255229] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:09Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15325worker1 # [ 110.296081] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605326worker1 # [ 110.311856] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497605327client # copying 1 paths to 'lb:50051'...5328client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5329worker1 # [ 110.327613] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:497605330worker1 # [ 110.343306] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5331worker1 # [ 110.372288] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:10Z level=info event=niks3_push_started pid=11135332worker1 # [ 110.598347] nix-grpc-daemon[1113]: time=2026-09-21T15:09:10.250Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5333worker1 # [ 110.769372] nix-daemon[841]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5334worker1 # [ 110.785716] nix-daemon[841]: reaped child process 1129, status = succeeded5335worker1 # [ 110.956548] nix-daemon[841]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)5336worker1 # [ 110.971524] nix-daemon[841]: reaped child process 1134, status = succeeded5337lb # [ 110.984907] niks3-server[1135]: 2026/09/21 15:09:10 INFO Received uploads request method=POST path=/api/pending_closures5338worker1 # [ 110.998413] nix-grpc-daemon[1113]: time=2026-09-21T15:09:10.649Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5339worker1 # [ 110.999870] nix-grpc-daemon[1113]: time=2026-09-21T15:09:10.649Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5340lb # [ 111.020306] niks3-server[1135]: 2026/09/21 15:09:10 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5341worker1 # [ 111.044699] nix-grpc-daemon[1113]: time=2026-09-21T15:09:10.696Z level=INFO msg="Uploading 1 narinfos"5342lb # [ 111.034264] niks3-server[1135]: 2026/09/21 15:09:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5343lb # [ 111.052495] niks3-server[1135]: 2026/09/21 15:09:10 INFO Signed narinfos id=33 count=15344lb # [ 111.068918] niks3-server[1135]: 2026/09/21 15:09:10 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5345worker1 # [ 111.078516] nix-grpc-daemon[1113]: time=2026-09-21T15:09:10.730Z level=INFO msg="Upload complete. (476ms)"5346lb # [ 111.084343] niks3-server[1135]: 2026/09/21 15:09:10 INFO Completed upload id=335347worker1 # [ 111.088289] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=0 paths=1 nar_bytes_in=5205348worker1 # [ 111.102444] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605349worker1 # [ 111.109447] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5350worker1 # [ 111.114853] nix-daemon[841]: reaped child process 1111, status = succeeded5351client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5352worker1 # [ 111.125799] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5353client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5354lb # [ 111.158355] envoy[914]: conn peer=[2001:db8:1::1]:40110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9697 tx=4086 ms=65165355lb # [ 111.163740] envoy[914]: conn peer=[2001:db8:1::4]:46134 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=190825356client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5357client # farm-a> LOG-farm-a-on-standby5358worker1 # [ 112.176607] nix-daemon[841]: reaped child process 1138, status = succeeded5359worker1 # [ 112.353144] nix-daemon[841]: accepted connection from pid 1154, user nix-grpc-daemon (trusted)5360worker1 # [ 112.369852] nix-daemon[841]: reaped child process 1157, status = succeeded5361lb # [ 112.382043] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received uploads request method=POST path=/api/pending_closures5362worker1 # [ 112.400450] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.049Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5363worker1 # [ 112.402007] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.049Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5364lb # [ 112.432399] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5365worker1 # [ 112.449050] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.100Z level=INFO msg="Uploading 1 narinfos"5366lb # [ 112.445386] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5367lb # [ 112.456221] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5368lb # [ 112.459436] niks3-server[1135]: 2026/09/21 15:09:12 INFO Signed narinfos id=34 count=15369lb # [ 112.471755] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5370lb # [ 112.476932] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5371lb # [ 112.480844] niks3-server[1135]: 2026/09/21 15:09:12 INFO Completed upload id=345372worker1 # [ 112.483727] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.136Z level=INFO msg="Upload complete. (345ms)"5373lb # [ 112.484512] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5374worker1 # [ 112.511598] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15375worker1 # [ 112.524559] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05376worker1 # [ 112.527861] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15377worker1 # [ 112.532673] nix-daemon[841]: reaped child process 1136, status = succeeded5378worker1 # [ 112.554826] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605379worker1 # [ 112.569131] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497605380worker1 # [ 112.583738] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.236Z level=INFO msg="All 1 paths already cached"5381client # copying 1 paths to 'lb:50051'...5382client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5383worker1 # [ 112.593870] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:497605384worker1 # [ 112.607790] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5385worker1 # [ 112.808374] nix-daemon[841]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5386worker1 # [ 112.824702] nix-daemon[841]: reaped child process 1166, status = succeeded5387worker1 # [ 112.994953] nix-daemon[841]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)5388worker1 # [ 113.010607] nix-daemon[841]: reaped child process 1171, status = succeeded5389lb # [ 113.019797] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received uploads request method=POST path=/api/pending_closures5390worker1 # [ 113.032055] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.684Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5391worker1 # [ 113.033540] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.684Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5392lb # [ 113.052847] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5393lb # [ 113.064957] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5394worker1 # [ 113.075428] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.727Z level=INFO msg="Uploading 1 narinfos"5395lb # [ 113.072879] niks3-server[1135]: 2026/09/21 15:09:12 INFO Signed narinfos id=35 count=15396lb # [ 113.081010] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5397lb # [ 113.089633] niks3-server[1135]: 2026/09/21 15:09:12 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5398lb # [ 113.097003] niks3-server[1135]: 2026/09/21 15:09:12 INFO Completed upload id=355399worker1 # [ 113.099522] nix-grpc-daemon[1113]: time=2026-09-21T15:09:12.751Z level=INFO msg="Upload complete. (475ms)"5400lb # [ 113.099778] niks3-server[1135]: 2026/09/21 15:09:12 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5401worker1 # [ 113.108294] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=0 paths=1 nar_bytes_in=6405402worker1 # [ 113.120611] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605403worker1 # [ 113.127532] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5404worker1 # [ 113.132244] nix-daemon[841]: reaped child process 1161, status = succeeded5405client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5406client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5407worker1 # [ 113.145594] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5408client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5409client # farm-b> LOG-farm-b-on-standby5410worker1 # [ 114.181682] nix-daemon[841]: reaped child process 1175, status = succeeded5411worker1 # [ 114.347479] nix-daemon[841]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5412worker1 # [ 114.363062] nix-daemon[841]: reaped child process 1194, status = succeeded5413lb # [ 114.376654] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received uploads request method=POST path=/api/pending_closures5414worker1 # [ 114.389933] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5415worker1 # [ 114.391551] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.041Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5416lb # [ 114.413745] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5417worker1 # [ 114.424821] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.077Z level=INFO msg="Uploading 1 narinfos"5418lb # [ 114.428381] niks3-server[1135]: 2026/09/21 15:09:14 INFO Signed narinfos id=36 count=15419lb # [ 114.437310] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5420lb # [ 114.452624] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5421lb # [ 114.455939] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5422lb # [ 114.462081] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5423lb # [ 114.466853] niks3-server[1135]: 2026/09/21 15:09:14 INFO Completed upload id=365424worker1 # [ 114.469726] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.122Z level=INFO msg="Upload complete. (314ms)"5425lb # [ 114.469437] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5426worker1 # [ 114.489136] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15427worker1 # [ 114.503693] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05428worker1 # [ 114.508549] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15429worker1 # [ 114.514186] nix-daemon[841]: reaped child process 1173, status = succeeded5430worker1 # [ 114.530743] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605431worker1 # [ 114.545138] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497605432worker1 # [ 114.558499] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.210Z level=INFO msg="All 2 paths already cached"5433client # copying 1 paths to 'lb:50051'...5434client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5435worker1 # [ 114.569219] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:497605436worker1 # [ 114.583151] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5437worker1 # [ 114.780498] nix-daemon[841]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5438worker1 # [ 114.799069] nix-daemon[841]: reaped child process 1203, status = succeeded5439worker1 # [ 114.970902] nix-daemon[841]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5440worker1 # [ 114.987319] nix-daemon[841]: reaped child process 1208, status = succeeded5441lb # [ 114.996654] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received uploads request method=POST path=/api/pending_closures5442worker1 # [ 115.008050] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.660Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5443worker1 # [ 115.009941] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.660Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5444lb # [ 115.027102] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5445lb # [ 115.040551] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5446worker1 # [ 115.049939] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.702Z level=INFO msg="Uploading 1 narinfos"5447lb # [ 115.050486] niks3-server[1135]: 2026/09/21 15:09:14 INFO Signed narinfos id=37 count=15448lb # [ 115.057936] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5449lb # [ 115.067932] niks3-server[1135]: 2026/09/21 15:09:14 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5450lb # [ 115.077311] niks3-server[1135]: 2026/09/21 15:09:14 INFO Completed upload id=375451worker1 # [ 115.079975] nix-grpc-daemon[1113]: time=2026-09-21T15:09:14.732Z level=INFO msg="Upload complete. (480ms)"5452lb # [ 115.080458] niks3-server[1135]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5453worker1 # [ 115.088132] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=0 paths=1 nar_bytes_in=7685454worker1 # [ 115.101990] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605455worker1 # [ 115.108596] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5456worker1 # [ 115.113475] nix-daemon[841]: reaped child process 1198, status = succeeded5457client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5458client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5459worker1 # [ 115.126705] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5460client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5461client # farm-top> LOG-farm-top-on-standby5462worker1 # [ 115.965965] nix-daemon[841]: reaped child process 1212, status = succeeded5463worker1 # [ 116.139497] nix-daemon[841]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5464worker1 # [ 116.158598] nix-daemon[841]: reaped child process 1231, status = succeeded5465lb # [ 116.170204] niks3-server[1135]: 2026/09/21 15:09:15 INFO Received uploads request method=POST path=/api/pending_closures5466worker1 # [ 116.183944] nix-grpc-daemon[1113]: time=2026-09-21T15:09:15.836Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5467worker1 # [ 116.188205] nix-grpc-daemon[1113]: time=2026-09-21T15:09:15.838Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5468lb # [ 116.207170] niks3-server[1135]: 2026/09/21 15:09:15 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5469lb # [ 116.220569] niks3-server[1135]: 2026/09/21 15:09:15 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5470lb # [ 116.236757] niks3-server[1135]: 2026/09/21 15:09:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5471worker1 # [ 116.243862] nix-grpc-daemon[1113]: time=2026-09-21T15:09:15.895Z level=INFO msg="Uploading 1 narinfos"5472lb # [ 116.245583] niks3-server[1135]: 2026/09/21 15:09:15 INFO Signed narinfos id=38 count=15473lb # [ 116.247898] niks3-server[1135]: 2026/09/21 15:09:15 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5474lb # [ 116.256292] niks3-server[1135]: 2026/09/21 15:09:15 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5475lb # [ 116.263444] niks3-server[1135]: 2026/09/21 15:09:15 INFO Completed upload id=385476worker1 # [ 116.265896] nix-grpc-daemon[1113]: time=2026-09-21T15:09:15.918Z level=INFO msg="Upload complete. (329ms)"5477lb # [ 116.266448] niks3-server[1135]: 2026/09/21 15:09:15 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5478worker1 # [ 116.281160] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15479worker1 # [ 116.297652] nix-daemon[841]: reaped child process 1210, status = succeeded5480client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.56 seconds)5481client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5482client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5483worker2 # [ 116.700819] systemd[1]: Listening on Nix worker-protocol over gRPC.5484worker2 # [ 116.706311] systemd[1]: Starting scheduler-token.service...5485worker2 # [ 116.796709] systemd[1]: scheduler-token.service: Deactivated successfully.5486worker2 # [ 116.802728] systemd[1]: Finished scheduler-token.service.5487worker2 # [ 116.804439] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 93ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5488worker2 # [ 116.809594] systemd[1]: Starting Nix worker-protocol over gRPC...5489worker2 # [ 116.986665] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5490worker2 # [ 116.989229] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5491worker2 # [ 116.993282] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15492worker2 # [ 117.000064] nix-grpc-daemon[1501]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5493worker2 # [ 117.001817] nix-grpc-daemon[1501]: E0000 00:00:1790003356.638707 1501 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5494worker2 # [ 117.012738] nix-grpc-daemon[1501]: E0000 00:00:1790003356.645108 1501 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5495worker2 # [ 117.019042] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5496worker2 # [ 117.021351] systemd[1]: Started Nix worker-protocol over gRPC.5497lb: must succeed: curl -sf localhost:9901/clusters5498worker2 # [ 117.024218] nix-grpc-daemon[1501]: E0000 00:00:1790003356.662751 1541 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5499worker2 # [ 117.058592] nix-grpc-daemon[1501]: E0000 00:00:1790003356.681636 1541 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5500worker2 # [ 117.061501] nix-grpc-daemon[1501]: E0000 00:00:1790003356.681829 1541 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5501worker2 # [ 117.064072] nix-grpc-daemon[1501]: E0000 00:00:1790003356.696592 1541 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5502worker2 # [ 117.075152] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:16Z level=info event=scheduler_connected addr=lb:500515503worker1 # [ 117.042629] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:16Z level=info event=oidc_jwks provider=mock keys=15504worker1 # [ 117.043095] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:464385505worker1 # [ 117.052467] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05506lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5507worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5508worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5509lb: must succeed: curl -sf localhost:9901/clusters5510lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5511lb: must succeed: curl -sf localhost:9901/clusters5512lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5513worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5514worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5515(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.67 seconds)5516subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5517client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5518client # [ 117.428750] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5519client # Running as unit: rs.service; invocation ID: 4d4b502c5db840f0ab014800571fdbd15520client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5521worker2 # [ 117.730908] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:332165522worker2 # [ 117.745509] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5523client # [ 117.721848] nix[1194]: don't know how to build these paths:5524client # [ 117.723134] nix[1194]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5525worker1 # [ 117.747901] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:17Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05526worker1 # [ 117.753381] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:17Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15527worker1 # [ 117.791921] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605528worker1 # [ 117.810780] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497605529client # [ 117.824521] nix[1194]: copying 1 paths to 'lb:50051'...5530worker1 # [ 117.826961] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:497605531client # [ 117.828261] nix[1194]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5532worker1 # [ 117.845470] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5533worker1 # [ 118.054920] nix-daemon[841]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5534worker1 # [ 118.075891] nix-daemon[841]: reaped child process 1247, status = succeeded5535worker1 # [ 118.254605] nix-daemon[841]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5536worker1 # [ 118.272425] nix-daemon[841]: reaped child process 1252, status = succeeded5537lb # [ 118.281392] niks3-server[1135]: 2026/09/21 15:09:17 INFO Received uploads request method=POST path=/api/pending_closures5538worker1 # [ 118.296464] nix-grpc-daemon[1113]: time=2026-09-21T15:09:17.948Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5539worker1 # [ 118.297951] nix-grpc-daemon[1113]: time=2026-09-21T15:09:17.948Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5540lb # [ 118.317532] niks3-server[1135]: 2026/09/21 15:09:17 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5541lb # [ 118.334585] niks3-server[1135]: 2026/09/21 15:09:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5542worker1 # [ 118.342629] nix-grpc-daemon[1113]: time=2026-09-21T15:09:17.994Z level=INFO msg="Uploading 1 narinfos"5543lb # [ 118.344338] niks3-server[1135]: 2026/09/21 15:09:17 INFO Signed narinfos id=39 count=15544lb # [ 118.347325] niks3-server[1135]: 2026/09/21 15:09:17 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5545lb # [ 118.363409] niks3-server[1135]: 2026/09/21 15:09:17 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5546lb # [ 118.368067] niks3-server[1135]: 2026/09/21 15:09:18 INFO Completed upload id=395547worker1 # [ 118.370532] nix-grpc-daemon[1113]: time=2026-09-21T15:09:18.022Z level=INFO msg="Upload complete. (507ms)"5548lb # [ 118.372889] niks3-server[1135]: 2026/09/21 15:09:18 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5549worker1 # [ 118.381009] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=0 paths=1 nar_bytes_in=4165550worker1 # [ 118.393901] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497605551worker1 # [ 118.401536] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5552worker1 # [ 118.406628] nix-daemon[841]: reaped child process 1242, status = succeeded5553client # [ 118.415283] nix[1194]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5554worker1 # [ 118.420367] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5555lb: must succeed: curl -sf localhost:9901/clusters5556lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5557lb: must succeed: curl -sf localhost:9901/clusters5558lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5559lb: must succeed: curl -sf localhost:9901/clusters5560lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5561lb: must succeed: curl -sf localhost:9901/clusters5562lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5563worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5564worker2 # [ 120.412406] systemd-journald[274]: Received client request to rotate journal, rotating.5565worker2 # [ 120.431763] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/c8958fd3f6ba4cdfa3beac5c35db0716.5566worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5567worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5568worker1 # [ 120.451111] systemd[1]: Stopping Nix worker-protocol over gRPC...5569worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5570client: must succeed: sleep 5.05571worker1 # [ 120.518059] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:20Z level=info event=draining5572lb # [ 121.158364] envoy[914]: conn peer=[2001:db8:1::1]:45830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=4141 ms=72545573client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5574worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5575worker1 # [ 125.870550] nix-daemon[841]: reaped child process 1256, status = succeeded5576worker1 # [ 126.039843] nix-daemon[841]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5577worker1 # [ 126.058051] nix-daemon[841]: reaped child process 1300, status = succeeded5578lb # [ 126.067469] niks3-server[1135]: 2026/09/21 15:09:25 INFO Received uploads request method=POST path=/api/pending_closures5579worker1 # [ 126.080398] nix-grpc-daemon[1113]: time=2026-09-21T15:09:25.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5580worker1 # [ 126.081876] nix-grpc-daemon[1113]: time=2026-09-21T15:09:25.731Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5581lb # [ 126.113448] niks3-server[1135]: 2026/09/21 15:09:25 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5582lb # [ 126.124738] niks3-server[1135]: 2026/09/21 15:09:25 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5583worker1 # [ 126.135591] nix-grpc-daemon[1113]: time=2026-09-21T15:09:25.787Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 126.137596] niks3-server[1135]: 2026/09/21 15:09:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5585lb # [ 126.144680] niks3-server[1135]: 2026/09/21 15:09:25 INFO Signed narinfos id=40 count=15586lb # [ 126.150237] niks3-server[1135]: 2026/09/21 15:09:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5587worker1 # [ 126.157271] nix-grpc-daemon[1113]: time=2026-09-21T15:09:25.809Z level=INFO msg="Upload complete. (319ms)"5588lb # [ 126.155002] niks3-server[1135]: 2026/09/21 15:09:25 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5589lb # [ 126.159239] niks3-server[1135]: 2026/09/21 15:09:25 INFO Completed upload id=405590lb # [ 126.162618] niks3-server[1135]: 2026/09/21 15:09:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5591worker1 # [ 126.170997] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49760 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15592worker1 # [ 126.188457] nix-daemon[841]: reaped child process 1254, status = succeeded5593client # [ 126.214333] systemd[1]: rs.service: Deactivated successfully.5594client # [ 126.218681] systemd[1]: rs.service: Consumed 178ms CPU time over 8.789s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.2K outgoing IP traffic.5595worker1 # [ 126.519112] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:26Z level=info event=signal_exit5596lb # [ 127.097200] niks3-server[1135]: 2026/09/21 15:09:26 INFO lead: released remote=192.168.1.3:605005597worker2 # [ 128.138448] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:27Z level=info event=scheduler_restarting addr=lb:500515598worker1 # [ 128.098895] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=scheduler_restarting5599worker1 # [ 128.103606] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=scheduler_restarting addr=lb:500515600worker1 # [ 128.108432] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:464385601worker1 # [ 128.113554] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=worker_gone addr=192.168.1.4:500515602worker1 # [ 128.118050] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:464385603worker1 # [ 128.122906] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=worker_gone addr=192.168.1.3:500515604worker1 # [ 128.146073] nix-daemon[841]: reaped child process 1091, status = succeeded5605worker2 # [ 128.338697] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:27Z level=info event=scheduler_connected addr=lb:500515606worker2 # [ 128.343926] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5607worker1 # [ 128.316731] nix-grpc-daemon[1037]: ts=2026-09-21T15:09:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5608worker2 # [ 128.545109] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_connected addr=lb:500515609worker2 # [ 128.550437] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5610worker2 # [ 128.751475] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_connected addr=lb:500515611worker2 # [ 128.756924] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5612worker1 # [ 128.754628] nix-grpc-daemon[1037]: E0000 00:00:1790003368.406824 1082 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5613worker1 # [ 128.761305] nix-grpc-daemon[1037]: E0000 00:00:1790003368.407028 1082 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5614worker2 # [ 128.958066] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_connected addr=lb:500515615worker2 # [ 128.963297] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5616worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.27 seconds)5617client: waiting for success: ! systemctl is-active rs5618worker2 # [ 129.166296] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_connected addr=lb:500515619worker2 # [ 129.170439] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5620client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5621client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5622worker1 # [ 129.196333] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5623worker1 # [ 129.199270] systemd[1]: Stopped Nix worker-protocol over gRPC.5624client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5625client: must fail: journalctl -u rs | grep -q 'reconnecting'5626worker1 # [ 129.208863] systemd[1]: nix-grpc-daemon.service: Consumed 1.848s CPU time over 36.906s wall clock time, 37.5M memory peak, 46.7K incoming IP traffic, 86.5K outgoing IP traffic.5627worker1 # [ 129.223920] systemd[1]: Starting Nix worker-protocol over gRPC...5628client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5629lb: must succeed: curl -sf localhost:9901/clusters5630lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5631worker2 # [ 129.367209] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515632worker2 # [ 129.372586] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5633worker1 # [ 129.418846] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5634worker1 # [ 129.421648] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5635worker1 # [ 129.425959] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15636worker1 # [ 129.434909] nix-grpc-daemon[1307]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5637worker1 # [ 129.440639] nix-grpc-daemon[1307]: E0000 00:00:1790003369.085024 1307 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5638worker1 # [ 129.444543] nix-grpc-daemon[1307]: E0000 00:00:1790003369.087236 1307 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5639worker1 # [ 129.447823] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5640worker1 # [ 129.451433] systemd[1]: Started Nix worker-protocol over gRPC.5641lb # [ 129.448619] niks3-server[1135]: 2026/09/21 15:09:29 INFO lead: acquired remote=192.168.1.3:570485642worker1 # [ 129.453127] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_take_over5643worker1 # [ 129.456077] nix-grpc-daemon[1307]: E0000 00:00:1790003369.108088 1346 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5644worker1 # [ 129.488056] nix-grpc-daemon[1307]: E0000 00:00:1790003369.126714 1346 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5645worker1 # [ 129.490319] nix-grpc-daemon[1307]: E0000 00:00:1790003369.126873 1346 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5646worker1 # [ 129.494264] nix-grpc-daemon[1307]: E0000 00:00:1790003369.145539 1346 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5647worker1 # [ 129.506355] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515648worker1 # [ 129.507983] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5649worker2 # [ 129.573532] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515650worker2 # [ 129.578691] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5651worker1 # [ 129.709211] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515652worker1 # [ 129.714091] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5653worker2 # [ 129.779536] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515654worker2 # [ 129.784378] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5655worker1 # [ 129.914781] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515656worker1 # [ 129.919976] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5657worker2 # [ 129.985434] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515658worker2 # [ 129.990336] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5659worker1 # [ 130.120750] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515660worker1 # [ 130.125832] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5661worker2 # [ 130.193117] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515662worker2 # [ 130.197207] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5663lb: must succeed: curl -sf localhost:9901/clusters5664worker1 # [ 130.326541] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=scheduler_connected addr=lb:500515665worker1 # [ 130.335450] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:416205666worker1 # [ 130.340735] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05667worker2 # [ 130.393826] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:30Z level=info event=scheduler_connected addr=lb:500515668worker1 # [ 130.360130] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=info event=oidc_jwks provider=mock keys=15669worker1 # [ 130.365745] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:416205670worker1 # [ 130.371007] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5672worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5673worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5674(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.12 seconds)5675subtest: scheduler restart mid-build: build finishes, no double build5676client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5677client # [ 130.548685] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5678client # Running as unit: mid.service; invocation ID: 0b5ebc1018b74299a671a90ec7073a705679client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5680worker2 # [ 130.842453] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:332165681client # [ 130.806391] nix[1230]: don't know how to build these paths:5682client # [ 130.807337] nix[1230]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5683worker1 # [ 130.821415] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05684worker1 # [ 130.827388] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15685worker1 # [ 130.878647] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471165686worker1 # [ 130.893232] nix-daemon[841]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5687worker1 # [ 130.916932] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)5688worker1 # [ 130.931632] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)5689worker1 # [ 131.012163] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 3/5)5690worker1 # [ 131.153616] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 4/5)5691lb # [ 131.160429] envoy[914]: conn peer=[2001:db8:1::1]:35176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1429 ms=85045692lb # [ 131.160747] envoy[914]: conn peer=[2001:db8:1::3]:38070 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1873 tx=701 ms=357385693worker1 # [ 131.496621] nix-grpc-daemon[1307]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5694worker1 # [ 131.503052] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5695worker1 # [ 131.521265] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471165696client # [ 131.532309] nix[1230]: copying 1 paths to 'lb:50051'...5697client # [ 131.538769] nix[1230]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5698worker1 # [ 131.539025] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471165699worker1 # [ 131.553329] nix-daemon[841]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5700worker1 # [ 131.580642] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:31Z level=info event=niks3_push_started pid=13765701worker1 # [ 131.796709] nix-grpc-daemon[1376]: time=2026-09-21T15:09:31.448Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5702worker1 # [ 131.973596] nix-daemon[841]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5703worker1 # [ 131.989814] nix-daemon[841]: reaped child process 1396, status = succeeded5704worker1 # [ 132.158653] nix-daemon[841]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5705worker1 # [ 132.174258] nix-daemon[841]: reaped child process 1401, status = succeeded5706lb # [ 132.183618] niks3-server[1135]: 2026/09/21 15:09:31 INFO Received uploads request method=POST path=/api/pending_closures5707worker1 # [ 132.196292] nix-grpc-daemon[1376]: time=2026-09-21T15:09:31.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5708worker1 # [ 132.199253] nix-grpc-daemon[1376]: time=2026-09-21T15:09:31.850Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5709lb # [ 132.219398] niks3-server[1135]: 2026/09/21 15:09:31 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5710worker1 # [ 132.241094] nix-grpc-daemon[1376]: time=2026-09-21T15:09:31.893Z level=INFO msg="Uploading 1 narinfos"5711lb # [ 132.232610] niks3-server[1135]: 2026/09/21 15:09:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5712lb # [ 132.244222] niks3-server[1135]: 2026/09/21 15:09:31 INFO Signed narinfos id=41 count=15713lb # [ 132.250843] niks3-server[1135]: 2026/09/21 15:09:31 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5714lb # [ 132.260317] niks3-server[1135]: 2026/09/21 15:09:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5715worker1 # [ 132.271349] nix-grpc-daemon[1376]: time=2026-09-21T15:09:31.923Z level=INFO msg="Upload complete. (469ms)"5716lb # [ 132.271009] niks3-server[1135]: 2026/09/21 15:09:31 INFO Completed upload id=415717lb # [ 132.274133] niks3-server[1135]: 2026/09/21 15:09:31 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5718worker1 # [ 132.280907] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47116 duration_s=0 paths=1 nar_bytes_in=4245719worker1 # [ 132.294638] nix-grpc-daemon[1307]: ts=2026-09-21T15:09:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471165720worker1 # [ 132.301121] nix-daemon[841]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5721worker1 # [ 132.306086] nix-daemon[841]: reaped child process 1374, status = succeeded5722worker1 # [ 132.316793] nix-daemon[841]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5723client # [ 132.316605] nix[1230]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5724lb: must succeed: curl -sf localhost:9901/clusters5725lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5726lb: must succeed: curl -sf localhost:9901/clusters5727lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5728worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5729worker1 # [ 134.569084] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1307 (nix-grpc-daemon) on client request.5730worker1 # [ 134.570657] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5731lb # [ 134.581545] niks3-server[1135]: 2026/09/21 15:09:34 INFO lead: released remote=192.168.1.3:570485732client # [ 134.591299] nix[1230]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5733worker2 # [ 134.634994] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5734client # [ 134.597765] nix[1230]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5735worker1 # [ 134.602828] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5736worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.09 seconds)5737client: must succeed: sleep 5.05738worker1 # [ 134.606348] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5739worker1 # [ 134.610865] systemd[1]: nix-grpc-daemon.service: Consumed 531ms CPU time over 5.377s wall clock time, 34M memory peak, 14.6K incoming IP traffic, 18.7K outgoing IP traffic.5740worker1 # [ 134.624465] nix-daemon[841]: reaped child process 1368, status = succeeded5741worker1 # [ 134.625568] nix-daemon[841]: reaped child process 1403, status = succeeded5742worker1 # [ 134.631853] nix-daemon[1405]: unexpected Nix daemon error: error: interrupted by the user5743worker1 # [ 134.639621] nix-daemon[841]: reaped child process 1405, status = failed with exit code 15744worker1 # [ 134.760595] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5745worker1 # [ 134.768658] systemd[1]: Starting Nix worker-protocol over gRPC...5746worker2 # [ 134.835207] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:34Z level=info event=scheduler_connected addr=lb:500515747worker1 # [ 134.976397] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5748worker1 # [ 134.979024] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5749worker1 # [ 134.983137] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15750worker1 # [ 134.991711] nix-grpc-daemon[1442]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5751worker1 # [ 135.000950] nix-grpc-daemon[1442]: E0000 00:00:1790003374.642072 1442 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5752worker1 # [ 135.003862] nix-grpc-daemon[1442]: E0000 00:00:1790003374.644050 1442 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5753lb # [ 135.002373] niks3-server[1135]: 2026/09/21 15:09:34 INFO lead: acquired remote=192.168.1.3:556985754worker1 # [ 135.008792] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5755worker1 # [ 135.011566] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=scheduler_take_over5756worker1 # [ 135.013583] systemd[1]: Started Nix worker-protocol over gRPC.5757worker1 # [ 135.016054] nix-grpc-daemon[1442]: E0000 00:00:1790003374.668259 1483 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5758worker1 # [ 135.044052] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=oidc_jwks provider=mock keys=15759worker1 # [ 135.045355] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:471305760worker1 # [ 135.047027] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05761worker1 # [ 135.054857] nix-grpc-daemon[1442]: E0000 00:00:1790003374.687160 1483 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5762worker1 # [ 135.057671] nix-grpc-daemon[1442]: E0000 00:00:1790003374.687301 1483 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5763worker1 # [ 135.060215] nix-grpc-daemon[1442]: E0000 00:00:1790003374.706586 1483 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5764worker1 # [ 135.067535] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=scheduler_connected addr=lb:500515765worker1 # [ 135.069444] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:471305766worker1 # [ 135.070962] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05767worker1 # [ 135.099850] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05768worker1 # [ 135.102431] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:34Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15769worker2 # [ 135.189992] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332165770worker2 # [ 135.207103] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)5771worker2 # [ 135.312874] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5)5772worker2 # [ 135.397171] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 230 ms (attempt 3/5)5773worker2 # [ 135.635808] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 747 ms (attempt 4/5)5774worker2 # [ 136.391063] nix-grpc-daemon[1501]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5775worker2 # [ 136.391385] nix-grpc-daemon[1501]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5776worker2 # [ 136.415058] nix-grpc-daemon[1501]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5777worker2 # [ 136.444462] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5778client # [ 136.417632] nix[1230]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5779worker2 # [ 136.465652] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5780client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5781client: waiting for success: ! systemctl is-active mid5782worker2 # [ 140.084838] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:39Z level=info event=niks3_push_started pid=16635783worker2 # [ 140.104463] nix-daemon[863]: reaped child process 1638, status = succeeded5784worker2 # [ 140.287227] nix-grpc-daemon[1663]: time=2026-09-21T15:09:39.925Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5785worker2 # [ 140.460496] nix-daemon[863]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)5786worker2 # [ 140.477396] nix-daemon[863]: reaped child process 1679, status = succeeded5787lb # [ 140.451837] niks3-server[1135]: 2026/09/21 15:09:40 INFO Received uploads request method=POST path=/api/pending_closures5788worker2 # [ 140.508090] nix-grpc-daemon[1663]: time=2026-09-21T15:09:40.145Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5789worker2 # [ 140.509567] nix-grpc-daemon[1663]: time=2026-09-21T15:09:40.145Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5790lb # [ 140.498753] niks3-server[1135]: 2026/09/21 15:09:40 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5791lb # [ 140.514579] niks3-server[1135]: 2026/09/21 15:09:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5792lb # [ 140.524389] niks3-server[1135]: 2026/09/21 15:09:40 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5793lb # [ 140.537312] niks3-server[1135]: 2026/09/21 15:09:40 INFO Signed narinfos id=42 count=15794worker2 # [ 140.584162] nix-grpc-daemon[1663]: time=2026-09-21T15:09:40.221Z level=INFO msg="Uploading 1 narinfos"5795lb # [ 140.555250] niks3-server[1135]: 2026/09/21 15:09:40 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5796lb # [ 140.559404] niks3-server[1135]: 2026/09/21 15:09:40 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5797lb # [ 140.568234] niks3-server[1135]: 2026/09/21 15:09:40 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5798lb # [ 140.571572] niks3-server[1135]: 2026/09/21 15:09:40 INFO Completed upload id=425799worker2 # [ 140.617275] nix-grpc-daemon[1663]: time=2026-09-21T15:09:40.256Z level=INFO msg="Upload complete. (325ms)"5800worker2 # [ 140.628552] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33216 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15801worker2 # [ 140.647940] nix-daemon[863]: reaped child process 1636, status = succeeded5802client # [ 140.628846] systemd[1]: mid.service: Deactivated successfully.5803client # [ 140.633039] systemd[1]: mid.service: Consumed 183ms CPU time over 10.084s wall clock time, 41.2M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5804client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5805client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5806client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5807(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.68 seconds)5808subtest: interrupting the client stops the build on the worker5809client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5810lb # [ 141.160802] envoy[914]: conn peer=[2001:db8:1::3]:50838 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=411 ms=50925811lb # [ 141.165753] envoy[914]: conn peer=[2001:db8:1::1]:42112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1851 ms=97995812client # [ 141.202776] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5813client # Running as unit: intr.service; invocation ID: 402d71717d004b489782516ae4ae44925814client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5815worker1 # [ 141.440981] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584765816worker1 # [ 141.456514] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5817client # [ 141.472601] nix[1266]: don't know how to build these paths:5818client # [ 141.473647] nix[1266]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5819worker1 # [ 141.487516] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05820worker1 # [ 141.494266] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15821worker1 # [ 141.538659] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584765822worker1 # [ 141.555667] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)5823worker1 # [ 141.657212] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)5824worker1 # [ 141.807648] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 3/5)5825worker1 # [ 141.961225] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 4/5)5826worker1 # [ 142.067260] nix-grpc-daemon[1442]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5827worker1 # [ 142.072646] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5828worker1 # [ 142.091390] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584765829client # [ 142.104147] nix[1266]: copying 1 paths to 'lb:50051'...5830worker1 # [ 142.106703] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584765831client # [ 142.108185] nix[1266]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5832worker1 # [ 142.123285] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5833worker1 # [ 142.151668] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:41Z level=info event=niks3_push_started pid=15185834worker1 # [ 142.362939] nix-grpc-daemon[1518]: time=2026-09-21T15:09:42.014Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5835worker1 # [ 142.545234] nix-daemon[841]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5836worker1 # [ 142.561516] nix-daemon[841]: reaped child process 1538, status = succeeded5837worker1 # [ 142.733067] nix-daemon[841]: accepted connection from pid 1540, user nix-grpc-daemon (trusted)5838worker1 # [ 142.750269] nix-daemon[841]: reaped child process 1543, status = succeeded5839lb # [ 142.762178] niks3-server[1135]: 2026/09/21 15:09:42 INFO Received uploads request method=POST path=/api/pending_closures5840worker1 # [ 142.777110] nix-grpc-daemon[1518]: time=2026-09-21T15:09:42.428Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5841worker1 # [ 142.778606] nix-grpc-daemon[1518]: time=2026-09-21T15:09:42.428Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5842lb # [ 142.800128] niks3-server[1135]: 2026/09/21 15:09:42 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5843lb # [ 142.811717] niks3-server[1135]: 2026/09/21 15:09:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5844worker1 # [ 142.821065] nix-grpc-daemon[1518]: time=2026-09-21T15:09:42.473Z level=INFO msg="Uploading 1 narinfos"5845lb # [ 142.824839] niks3-server[1135]: 2026/09/21 15:09:42 INFO Signed narinfos id=43 count=15846lb # [ 142.832182] niks3-server[1135]: 2026/09/21 15:09:42 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5847lb # [ 142.841533] niks3-server[1135]: 2026/09/21 15:09:42 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5848lb # [ 142.849192] niks3-server[1135]: 2026/09/21 15:09:42 INFO Completed upload id=435849worker1 # [ 142.852043] nix-grpc-daemon[1518]: time=2026-09-21T15:09:42.504Z level=INFO msg="Upload complete. (485ms)"5850lb # [ 142.851954] niks3-server[1135]: 2026/09/21 15:09:42 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5851worker1 # [ 142.860804] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=4245852worker1 # [ 142.875200] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584765853worker1 # [ 142.884234] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5854worker1 # [ 142.888384] nix-daemon[841]: reaped child process 1516, status = succeeded5855client # [ 142.898357] nix[1266]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5856worker1 # [ 142.902628] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5857client: must succeed: systemctl kill -s INT intr5858worker1 # [ 144.904211] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:44Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5859client # [ 144.901175] systemd[1]: intr.service: Sent signal SIGINT to main process 1266 (nix) on client request.5860client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5861client # [ 144.909072] nix[1266]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5862client # [ 144.913813] nix[1266]: error: interrupted by the user5863worker1 # [ 144.942033] nix-daemon[1547]: unexpected Nix daemon error: error: interrupted by the user5864worker1 # [ 144.945978] nix-daemon[841]: reaped child process 1545, status = succeeded5865worker1 # [ 144.953685] nix-daemon[841]: reaped child process 1547, status = failed with exit code 15866client # [ 144.956593] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5867client # [ 144.959380] systemd[1]: intr.service: Failed with result 'exit-code'.5868client # [ 144.962965] systemd[1]: intr.service: Consumed 159ms CPU time over 3.759s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5869(finished: subtest: interrupting the client stops the build on the worker, in 3.89 seconds)5870subtest: a deploy mid-build drains: the build finishes, the new generation takes over5871client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5872client # [ 145.118893] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5873client # Running as unit: sw.service; invocation ID: b931d8d7b3a94c3da43dc1b884afe03a5874client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.08 seconds)5875worker1 # [ 145.371488] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584765876client # [ 145.378563] nix[1286]: don't know how to build these paths:5877client # [ 145.380432] nix[1286]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5878worker1 # [ 145.391426] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:45Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05879worker1 # [ 145.397434] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:45Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15880worker2 # [ 145.483204] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332165881worker2 # [ 145.499074] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332165882client # [ 145.468246] nix[1286]: copying 1 paths to 'lb:50051'...5883worker2 # [ 145.513750] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332165884client # [ 145.472157] nix[1286]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5885worker2 # [ 145.531159] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5886worker2 # [ 145.748686] nix-daemon[863]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5887worker2 # [ 145.768163] nix-daemon[863]: reaped child process 1710, status = succeeded5888worker2 # [ 145.941981] nix-daemon[863]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)5889worker2 # [ 145.959227] nix-daemon[863]: reaped child process 1715, status = succeeded5890lb # [ 145.930135] niks3-server[1135]: 2026/09/21 15:09:45 INFO Received uploads request method=POST path=/api/pending_closures5891worker2 # [ 145.985176] nix-grpc-daemon[1663]: time=2026-09-21T15:09:45.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5892worker2 # [ 145.986616] nix-grpc-daemon[1663]: time=2026-09-21T15:09:45.623Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5893lb # [ 145.963815] niks3-server[1135]: 2026/09/21 15:09:45 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5894worker2 # [ 146.022095] nix-grpc-daemon[1663]: time=2026-09-21T15:09:45.660Z level=INFO msg="Uploading 1 narinfos"5895lb # [ 145.979862] niks3-server[1135]: 2026/09/21 15:09:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5896lb # [ 145.991140] niks3-server[1135]: 2026/09/21 15:09:45 INFO Signed narinfos id=44 count=15897lb # [ 145.999599] niks3-server[1135]: 2026/09/21 15:09:45 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5898lb # [ 146.005410] niks3-server[1135]: 2026/09/21 15:09:45 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5899worker2 # [ 146.055095] nix-grpc-daemon[1663]: time=2026-09-21T15:09:45.694Z level=INFO msg="Upload complete. (507ms)"5900lb # [ 146.009719] niks3-server[1135]: 2026/09/21 15:09:45 INFO Completed upload id=445901lb # [ 146.012972] niks3-server[1135]: 2026/09/21 15:09:45 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5902worker2 # [ 146.060163] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33216 duration_s=0 paths=1 nar_bytes_in=4165903worker2 # [ 146.064192] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332165904worker2 # [ 146.070582] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5905worker2 # [ 146.073476] nix-daemon[863]: reaped child process 1705, status = succeeded5906worker2 # [ 146.081498] nix-daemon[863]: accepted connection from pid 1501, user nix-grpc-daemon (trusted)5907client # [ 146.035693] nix[1286]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5908worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25909worker2 # Checking switch inhibitors... done5910worker2 # [ 148.635573] nixos[1751]: switching to system configuration /nix/store/c2b90fzp30xrd06a8zngfc0zn3vf7fxm-nixos-system-worker2-test5911worker2 # [ 148.638642] systemd[1]: Stopped target Remote File Systems.5912worker2 # [ 148.640838] systemd[1]: Stopped target Local File Systems.5913worker2 # activating the configuration...5914worker2 # setting up /etc...5915worker2 # [ 149.401889] systemd[1]: Reload requested from client PID 1751 ('.switch-to-conf') (unit backdoor.service)...5916worker2 # [ 149.403672] systemd[1]: Reloading...5917worker2 # [ 150.308950] systemd[1]: Reloading finished in 902 ms.5918worker2 # restarting sysinit-reactivation.target5919worker2 # [ 150.390087] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5920worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5921worker2 # [ 150.546707] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5922worker2 # [ 150.547911] systemd[1]: Reached target Reactivate sysinit units.5923worker2 # [ 150.552571] systemd[1]: Reloading D-Bus System Message Bus...5924worker2 # [ 150.562642] dbus-broker-launch[498]: Caught SIGHUP, trigger reload.5925worker2 # [ 150.566113] systemd[1]: Reloading Nix worker-protocol over gRPC...5926worker2 # [ 150.578531] dbus-broker-launch[498]: Looking up NSS user entry for 'systemd-timesync'...5927worker2 # [ 150.580849] dbus-broker-launch[498]: NSS returned no entry for 'systemd-timesync'5928worker2 # [ 150.582523] dbus-broker-launch[498]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5929worker2 # [ 150.610399] systemd[1]: Reloaded Nix worker-protocol over gRPC.5930worker2 # [ 150.632596] systemd[1]: Reloaded D-Bus System Message Bus.5931worker2 # [ 150.637446] systemd[1]: Reached target Local File Systems.5932worker2 # [ 150.644711] systemd[1]: Reached target Remote File Systems.5933worker2 # [ 150.649371] systemd[1]: Starting linger-users.service...5934worker2 # [ 150.654500] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5935worker2 # [ 150.666177] systemd[1]: Starting Create SUID/SGID Wrappers...5936worker2 # [ 150.667013] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5937worker2 # [ 150.677712] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675938worker2 # [ 150.679862] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5939worker2 # [ 150.685284] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5940worker2 # [ 150.686531] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5941worker2 # [ 150.687668] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5942worker2 # [ 150.694696] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5943worker2 # [ 150.734888] systemd[1]: linger-users.service: Deactivated successfully.5944worker2 # [ 150.736423] systemd[1]: Finished linger-users.service.5945worker2 # [ 151.022686] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:50Z level=info event=draining5946worker2 # [ 151.192180] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5947worker2 # [ 151.193211] systemd[1]: Finished Create SUID/SGID Wrappers.5948lb # [ 151.162041] envoy[914]: conn peer=[2001:db8:1::1]:46990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=827 ms=34985949worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5950worker2 # [ 151.448520] nixos[1751]: finished switching to system configuration /nix/store/c2b90fzp30xrd06a8zngfc0zn3vf7fxm-nixos-system-worker2-test5951worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.65 seconds)5952worker2: must succeed: pgrep -f 'read -t [9]0 x'5953worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5954client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25955worker1 # [ 151.737976] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584765956client # don't know how to build these paths:5957client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5958worker1 # [ 151.750954] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584765959worker1 # [ 151.778003] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05960worker1 # [ 151.785097] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15961worker1 # [ 151.822632] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584765962worker1 # [ 151.838214] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584765963client # copying 1 paths to 'lb:50051'...5964worker1 # [ 151.854467] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584765965client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5966worker1 # [ 151.877668] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5967worker1 # [ 152.078938] nix-daemon[841]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)5968worker1 # [ 152.096658] nix-daemon[841]: reaped child process 1601, status = succeeded5969worker1 # [ 152.269053] nix-daemon[841]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5970worker1 # [ 152.284260] nix-daemon[841]: reaped child process 1606, status = succeeded5971lb # [ 152.298067] niks3-server[1135]: 2026/09/21 15:09:51 INFO Received uploads request method=POST path=/api/pending_closures5972worker1 # [ 152.311628] nix-grpc-daemon[1518]: time=2026-09-21T15:09:51.963Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5973worker1 # [ 152.313214] nix-grpc-daemon[1518]: time=2026-09-21T15:09:51.963Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5974lb # [ 152.328842] niks3-server[1135]: 2026/09/21 15:09:51 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5975lb # [ 152.344793] niks3-server[1135]: 2026/09/21 15:09:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5976worker1 # [ 152.353023] nix-grpc-daemon[1518]: time=2026-09-21T15:09:52.005Z level=INFO msg="Uploading 1 narinfos"5977lb # [ 152.355184] niks3-server[1135]: 2026/09/21 15:09:51 INFO Signed narinfos id=45 count=15978lb # [ 152.360404] niks3-server[1135]: 2026/09/21 15:09:51 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5979lb # [ 152.368876] niks3-server[1135]: 2026/09/21 15:09:52 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5980lb # [ 152.375552] niks3-server[1135]: 2026/09/21 15:09:52 INFO Completed upload id=455981worker1 # [ 152.378384] nix-grpc-daemon[1518]: time=2026-09-21T15:09:52.030Z level=INFO msg="Upload complete. (483ms)"5982lb # [ 152.378486] niks3-server[1135]: 2026/09/21 15:09:52 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5983worker1 # [ 152.386232] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=5285984worker1 # [ 152.400574] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584765985worker1 # [ 152.411037] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5986worker1 # [ 152.413474] nix-daemon[841]: reaped child process 1596, status = succeeded5987client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5988client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5989worker1 # [ 152.427991] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5990client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5991client # farm-a> LOG-farm-a-during-drain5992worker1 # [ 153.668756] nix-daemon[841]: reaped child process 1610, status = succeeded5993worker1 # [ 153.848134] nix-daemon[841]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)5994worker1 # [ 153.863079] nix-daemon[841]: reaped child process 1629, status = succeeded5995lb # [ 153.878171] niks3-server[1135]: 2026/09/21 15:09:53 INFO Received uploads request method=POST path=/api/pending_closures5996worker1 # [ 153.891647] nix-grpc-daemon[1518]: time=2026-09-21T15:09:53.540Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5997worker1 # [ 153.894006] nix-grpc-daemon[1518]: time=2026-09-21T15:09:53.541Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5998lb # [ 153.921965] niks3-server[1135]: 2026/09/21 15:09:53 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5999lb # [ 153.935844] niks3-server[1135]: 2026/09/21 15:09:53 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6000worker1 # [ 153.952153] nix-grpc-daemon[1518]: time=2026-09-21T15:09:53.603Z level=INFO msg="Uploading 1 narinfos"6001lb # [ 153.949708] niks3-server[1135]: 2026/09/21 15:09:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6002lb # [ 153.960162] niks3-server[1135]: 2026/09/21 15:09:53 INFO Signed narinfos id=46 count=16003lb # [ 153.963168] niks3-server[1135]: 2026/09/21 15:09:53 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6004lb # [ 153.973636] niks3-server[1135]: 2026/09/21 15:09:53 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6005lb # [ 153.980255] niks3-server[1135]: 2026/09/21 15:09:53 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6006lb # [ 153.985788] niks3-server[1135]: 2026/09/21 15:09:53 INFO Completed upload id=466007worker1 # [ 153.988613] nix-grpc-daemon[1518]: time=2026-09-21T15:09:53.641Z level=INFO msg="Upload complete. (350ms)"6008worker1 # [ 154.009235] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16009worker1 # [ 154.023833] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06010worker1 # [ 154.028724] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16011worker1 # [ 154.033752] nix-daemon[841]: reaped child process 1608, status = succeeded6012worker1 # [ 154.056792] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766013worker1 # [ 154.071561] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766014worker1 # [ 154.085359] nix-grpc-daemon[1518]: time=2026-09-21T15:09:53.737Z level=INFO msg="All 1 paths already cached"6015client # copying 1 paths to 'lb:50051'...6016worker1 # [ 154.095314] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766017client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6018worker1 # [ 154.118017] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6019worker1 # [ 154.316789] nix-daemon[841]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6020worker1 # [ 154.332439] nix-daemon[841]: reaped child process 1638, status = succeeded6021worker1 # [ 154.504861] nix-daemon[841]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6022worker1 # [ 154.520681] nix-daemon[841]: reaped child process 1643, status = succeeded6023lb # [ 154.530961] niks3-server[1135]: 2026/09/21 15:09:54 INFO Received uploads request method=POST path=/api/pending_closures6024worker1 # [ 154.543608] nix-grpc-daemon[1518]: time=2026-09-21T15:09:54.195Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6025worker1 # [ 154.545258] nix-grpc-daemon[1518]: time=2026-09-21T15:09:54.195Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6026lb # [ 154.562938] niks3-server[1135]: 2026/09/21 15:09:54 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6027lb # [ 154.573440] niks3-server[1135]: 2026/09/21 15:09:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6028worker1 # [ 154.583794] nix-grpc-daemon[1518]: time=2026-09-21T15:09:54.235Z level=INFO msg="Uploading 1 narinfos"6029lb # [ 154.584355] niks3-server[1135]: 2026/09/21 15:09:54 INFO Signed narinfos id=47 count=16030lb # [ 154.590138] niks3-server[1135]: 2026/09/21 15:09:54 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6031lb # [ 154.599850] niks3-server[1135]: 2026/09/21 15:09:54 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6032worker1 # [ 154.607058] nix-grpc-daemon[1518]: time=2026-09-21T15:09:54.259Z level=INFO msg="Upload complete. (472ms)"6033lb # [ 154.605285] niks3-server[1135]: 2026/09/21 15:09:54 INFO Completed upload id=476034lb # [ 154.607897] niks3-server[1135]: 2026/09/21 15:09:54 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6035worker1 # [ 154.616273] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=6406036worker1 # [ 154.626058] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766037worker1 # [ 154.635175] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6038worker1 # [ 154.637950] nix-daemon[841]: reaped child process 1633, status = succeeded6039client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6040client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6041worker1 # [ 154.651298] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6042client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6043client # farm-b> LOG-farm-b-during-drain6044worker1 # [ 155.898007] nix-daemon[841]: reaped child process 1647, status = succeeded6045worker1 # [ 156.081898] nix-daemon[841]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)6046worker1 # [ 156.098419] nix-daemon[841]: reaped child process 1666, status = succeeded6047lb # [ 156.114156] niks3-server[1135]: 2026/09/21 15:09:55 INFO Received uploads request method=POST path=/api/pending_closures6048worker1 # [ 156.125782] nix-grpc-daemon[1518]: time=2026-09-21T15:09:55.777Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6049worker1 # [ 156.127644] nix-grpc-daemon[1518]: time=2026-09-21T15:09:55.777Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6050lb # [ 156.147862] niks3-server[1135]: 2026/09/21 15:09:55 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6051lb # [ 156.161093] niks3-server[1135]: 2026/09/21 15:09:55 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6052lb # [ 156.169094] niks3-server[1135]: 2026/09/21 15:09:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6053worker1 # [ 156.177787] nix-grpc-daemon[1518]: time=2026-09-21T15:09:55.829Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 156.177916] niks3-server[1135]: 2026/09/21 15:09:55 INFO Signed narinfos id=48 count=16055lb # [ 156.183500] niks3-server[1135]: 2026/09/21 15:09:55 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6056lb # [ 156.196132] niks3-server[1135]: 2026/09/21 15:09:55 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6057lb # [ 156.202798] niks3-server[1135]: 2026/09/21 15:09:55 INFO Completed upload id=486058worker1 # [ 156.205021] nix-grpc-daemon[1518]: time=2026-09-21T15:09:55.857Z level=INFO msg="Upload complete. (341ms)"6059lb # [ 156.205919] niks3-server[1135]: 2026/09/21 15:09:55 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6060worker1 # [ 156.224416] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16061worker1 # [ 156.240373] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06062worker1 # [ 156.245337] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16063worker1 # [ 156.249696] nix-daemon[841]: reaped child process 1645, status = succeeded6064worker1 # [ 156.266703] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766065worker1 # [ 156.281761] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766066worker1 # [ 156.295548] nix-grpc-daemon[1518]: time=2026-09-21T15:09:55.947Z level=INFO msg="All 2 paths already cached"6067client # copying 1 paths to 'lb:50051'...6068worker1 # [ 156.304466] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766069client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6070worker1 # [ 156.328833] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6071worker1 # [ 156.527307] nix-daemon[841]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6072worker1 # [ 156.542991] nix-daemon[841]: reaped child process 1675, status = succeeded6073worker1 # [ 156.719863] nix-daemon[841]: accepted connection from pid 1677, user nix-grpc-daemon (trusted)6074worker1 # [ 156.734112] nix-daemon[841]: reaped child process 1680, status = succeeded6075lb # [ 156.746492] niks3-server[1135]: 2026/09/21 15:09:56 INFO Received uploads request method=POST path=/api/pending_closures6076worker1 # [ 156.758332] nix-grpc-daemon[1518]: time=2026-09-21T15:09:56.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6077worker1 # [ 156.759861] nix-grpc-daemon[1518]: time=2026-09-21T15:09:56.410Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6078lb # [ 156.776255] niks3-server[1135]: 2026/09/21 15:09:56 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6079worker1 # [ 156.789021] nix-grpc-daemon[1518]: time=2026-09-21T15:09:56.441Z level=INFO msg="Uploading 1 narinfos"6080lb # [ 156.790198] niks3-server[1135]: 2026/09/21 15:09:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6081lb # [ 156.800893] niks3-server[1135]: 2026/09/21 15:09:56 INFO Signed narinfos id=49 count=16082lb # [ 156.806133] niks3-server[1135]: 2026/09/21 15:09:56 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6083worker1 # [ 156.819050] nix-grpc-daemon[1518]: time=2026-09-21T15:09:56.471Z level=INFO msg="Upload complete. (472ms)"6084lb # [ 156.816655] niks3-server[1135]: 2026/09/21 15:09:56 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6085lb # [ 156.821933] niks3-server[1135]: 2026/09/21 15:09:56 INFO Completed upload id=496086worker1 # [ 156.828193] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=7686087lb # [ 156.825547] niks3-server[1135]: 2026/09/21 15:09:56 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6088worker1 # [ 156.835876] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766089worker1 # [ 156.844005] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6090worker1 # [ 156.846127] nix-daemon[841]: reaped child process 1670, status = succeeded6091client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6092worker1 # [ 156.856424] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6093client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6094client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6095client # farm-top> LOG-farm-top-during-drain6096worker1 # [ 157.675905] nix-daemon[841]: reaped child process 1684, status = succeeded6097worker1 # [ 157.834357] nix-daemon[841]: accepted connection from pid 1700, user nix-grpc-daemon (trusted)6098worker1 # [ 157.850937] nix-daemon[841]: reaped child process 1703, status = succeeded6099lb # [ 157.865321] niks3-server[1135]: 2026/09/21 15:09:57 INFO Received uploads request method=POST path=/api/pending_closures6100worker1 # [ 157.882520] nix-grpc-daemon[1518]: time=2026-09-21T15:09:57.533Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6101worker1 # [ 157.883994] nix-grpc-daemon[1518]: time=2026-09-21T15:09:57.533Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6102lb # [ 157.898711] niks3-server[1135]: 2026/09/21 15:09:57 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6103lb # [ 157.913799] niks3-server[1135]: 2026/09/21 15:09:57 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6104lb # [ 157.927651] niks3-server[1135]: 2026/09/21 15:09:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6105worker1 # [ 157.935516] nix-grpc-daemon[1518]: time=2026-09-21T15:09:57.587Z level=INFO msg="Uploading 1 narinfos"6106lb # [ 157.937906] niks3-server[1135]: 2026/09/21 15:09:57 INFO Signed narinfos id=50 count=16107lb # [ 157.943147] niks3-server[1135]: 2026/09/21 15:09:57 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6108lb # [ 157.949178] niks3-server[1135]: 2026/09/21 15:09:57 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6109lb # [ 157.956086] niks3-server[1135]: 2026/09/21 15:09:57 INFO Completed upload id=506110worker1 # [ 157.958912] nix-grpc-daemon[1518]: time=2026-09-21T15:09:57.611Z level=INFO msg="Upload complete. (292ms)"6111lb # [ 157.959545] niks3-server[1135]: 2026/09/21 15:09:57 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6112worker1 # [ 157.967611] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16113worker1 # [ 157.985464] nix-daemon[841]: reaped child process 1682, status = succeeded6114client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.55 seconds)6115client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6116client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6117worker2: must succeed: pkill -f 'read -t [9]0 x'6118worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6119client: waiting for success: ! systemctl is-active sw6120worker2 # [ 158.536263] nix-daemon[863]: reaped child process 1719, status = succeeded6121worker2 # [ 158.724543] nix-daemon[863]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6122worker2 # [ 158.741669] nix-daemon[863]: reaped child process 1894, status = succeeded6123lb # [ 158.709540] niks3-server[1135]: 2026/09/21 15:09:58 INFO Received uploads request method=POST path=/api/pending_closures6124worker2 # [ 158.768769] nix-grpc-daemon[1663]: time=2026-09-21T15:09:58.406Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6125worker2 # [ 158.770242] nix-grpc-daemon[1663]: time=2026-09-21T15:09:58.406Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6126lb # [ 158.747318] niks3-server[1135]: 2026/09/21 15:09:58 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6127lb # [ 158.765143] niks3-server[1135]: 2026/09/21 15:09:58 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6128lb # [ 158.772641] niks3-server[1135]: 2026/09/21 15:09:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6129worker2 # [ 158.825349] nix-grpc-daemon[1663]: time=2026-09-21T15:09:58.463Z level=INFO msg="Uploading 1 narinfos"6130lb # [ 158.778766] niks3-server[1135]: 2026/09/21 15:09:58 INFO Signed narinfos id=51 count=16131lb # [ 158.784767] niks3-server[1135]: 2026/09/21 15:09:58 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6132lb # [ 158.794063] niks3-server[1135]: 2026/09/21 15:09:58 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6133lb # [ 158.804286] niks3-server[1135]: 2026/09/21 15:09:58 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6134lb # [ 158.810397] niks3-server[1135]: 2026/09/21 15:09:58 INFO Completed upload id=516135worker2 # [ 158.856082] nix-grpc-daemon[1663]: time=2026-09-21T15:09:58.494Z level=INFO msg="Upload complete. (351ms)"6136worker2 # [ 158.869054] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33216 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16137worker2 # [ 158.888131] nix-daemon[863]: reaped child process 1717, status = succeeded6138client # [ 158.867278] systemd[1]: sw.service: Deactivated successfully.6139client # [ 158.872247] systemd[1]: sw.service: Consumed 176ms CPU time over 13.751s wall clock time, 39.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6140worker2 # [ 159.024770] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:58Z level=info event=signal_exit6141client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)6142client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6143client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6144worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6145worker2 # [ 159.640988] systemd[1]: Starting scheduler-token.service...6146worker2 # [ 159.736557] systemd[1]: scheduler-token.service: Deactivated successfully.6147worker2 # [ 159.740863] systemd[1]: Finished scheduler-token.service.6148worker2 # [ 159.742252] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 100ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6149worker2 # [ 160.264854] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:59Z level=info event=scheduler_restarting6150worker1 # [ 160.222185] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:471306151worker1 # [ 160.227071] nix-grpc-daemon[1442]: ts=2026-09-21T15:09:59Z level=info event=worker_gone addr=192.168.1.4:500516152worker2 # [ 160.289524] nix-grpc-daemon[1501]: ts=2026-09-21T15:09:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6153worker2 # [ 160.315032] nix-daemon[863]: reaped child process 1560, status = succeeded6154worker2 # [ 160.652436] nix-grpc-daemon[1501]: E0000 00:00:1790003400.290990 1546 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6155worker2 # [ 160.659035] nix-grpc-daemon[1501]: E0000 00:00:1790003400.291248 1546 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6156lb # [ 161.169313] envoy[914]: conn peer=[2001:db8:1::1]:42492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4102 ms=62516157lb # [ 161.174316] envoy[914]: conn peer=[2001:db8:1::1]:46996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1428 ms=134676158lb # [ 161.179196] envoy[914]: conn peer=[2001:db8:1::4]:48678 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=10665 tx=839 ms=432006159worker2 # [ 161.479013] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6160worker2 # [ 161.484990] systemd[1]: nix-grpc-daemon.service: Consumed 932ms CPU time over 44.667s wall clock time, 33.6M memory peak, 23.7K incoming IP traffic, 41.6K outgoing IP traffic.6161worker2 # [ 161.504781] systemd[1]: Starting Nix worker-protocol over gRPC...6162worker2 # [ 161.700327] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6163worker2 # [ 161.703789] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6164worker2 # [ 161.707971] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16165worker2 # [ 161.716073] nix-grpc-daemon[1911]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6166worker2 # [ 161.717536] nix-grpc-daemon[1911]: E0000 00:00:1790003401.353619 1911 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6167worker2 # [ 161.726733] nix-grpc-daemon[1911]: E0000 00:00:1790003401.359823 1911 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6168worker2 # [ 161.731036] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6169worker2 # [ 161.734416] systemd[1]: Started Nix worker-protocol over gRPC.6170worker2 # [ 161.740112] nix-grpc-daemon[1911]: E0000 00:00:1790003401.376572 1952 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6171worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.20 seconds)6172lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6173lb: must succeed: curl -sf localhost:9901/clusters6174worker2 # [ 161.768051] nix-grpc-daemon[1911]: E0000 00:00:1790003401.395091 1952 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6175worker2 # [ 161.770352] nix-grpc-daemon[1911]: E0000 00:00:1790003401.395274 1952 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6176worker2 # [ 161.776752] nix-grpc-daemon[1911]: E0000 00:00:1790003401.414686 1952 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6177worker2 # [ 161.794932] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:01Z level=info event=scheduler_connected addr=lb:500516178worker1 # [ 161.755808] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:471306179worker1 # [ 161.761714] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6181lb: must succeed: curl -sf localhost:9901/clusters6182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6183lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)6184lb: must succeed: curl -sf localhost:9901/clusters6185lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6186worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6187worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6188(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.97 seconds)6189subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6190client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16191worker1 # [ 163.297964] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584766192worker1 # [ 163.306230] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766193worker1 # [ 163.349204] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06194worker1 # [ 163.356506] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16195worker1 # [ 163.362435] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06196worker1 # [ 163.367623] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06197worker1 # [ 163.395226] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766198worker1 # [ 163.410398] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766199worker1 # [ 163.425012] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766200worker1 # [ 163.439855] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6201worker1 # [ 163.649208] nix-daemon[841]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6202worker1 # [ 163.668147] nix-daemon[841]: reaped child process 1715, status = succeeded6203worker1 # [ 163.844240] nix-daemon[841]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6204worker1 # [ 163.863180] nix-daemon[841]: reaped child process 1720, status = succeeded6205lb # [ 163.872816] niks3-server[1135]: 2026/09/21 15:10:03 INFO Received uploads request method=POST path=/api/pending_closures6206worker1 # [ 163.885575] nix-grpc-daemon[1518]: time=2026-09-21T15:10:03.537Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6207worker1 # [ 163.887075] nix-grpc-daemon[1518]: time=2026-09-21T15:10:03.537Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6208worker1 # [ 163.914918] nix-grpc-daemon[1518]: time=2026-09-21T15:10:03.567Z level=INFO msg="Uploading 1 narinfos"6209lb # [ 163.902449] niks3-server[1135]: 2026/09/21 15:10:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6210lb # [ 163.917450] niks3-server[1135]: 2026/09/21 15:10:03 INFO Signed narinfos id=52 count=16211lb # [ 163.925753] niks3-server[1135]: 2026/09/21 15:10:03 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6212lb # [ 163.937253] niks3-server[1135]: 2026/09/21 15:10:03 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6213lb # [ 163.946994] niks3-server[1135]: 2026/09/21 15:10:03 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6214lb # [ 163.955227] niks3-server[1135]: 2026/09/21 15:10:03 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6215lb # [ 163.960055] niks3-server[1135]: 2026/09/21 15:10:03 INFO Completed upload id=526216worker1 # [ 163.962416] nix-grpc-daemon[1518]: time=2026-09-21T15:10:03.614Z level=INFO msg="Upload complete. (505ms)"6217worker1 # [ 163.970811] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=5126218worker1 # [ 163.984302] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766219worker1 # [ 163.991509] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6220worker1 # [ 163.996591] nix-daemon[841]: reaped child process 1710, status = succeeded6221worker1 # [ 164.010159] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6222worker1 # [ 165.056844] nix-daemon[841]: reaped child process 1724, status = succeeded6223worker1 # [ 165.231569] nix-daemon[841]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6224worker1 # [ 165.246605] nix-daemon[841]: reaped child process 1743, status = succeeded6225lb # [ 165.261280] niks3-server[1135]: 2026/09/21 15:10:04 INFO Received uploads request method=POST path=/api/pending_closures6226worker1 # [ 165.275035] nix-grpc-daemon[1518]: time=2026-09-21T15:10:04.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6227worker1 # [ 165.276665] nix-grpc-daemon[1518]: time=2026-09-21T15:10:04.924Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6228lb # [ 165.294004] niks3-server[1135]: 2026/09/21 15:10:04 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6229lb # [ 165.306260] niks3-server[1135]: 2026/09/21 15:10:04 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6230lb # [ 165.318402] niks3-server[1135]: 2026/09/21 15:10:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6231worker1 # [ 165.326157] nix-grpc-daemon[1518]: time=2026-09-21T15:10:04.978Z level=INFO msg="Uploading 1 narinfos"6232lb # [ 165.325813] niks3-server[1135]: 2026/09/21 15:10:04 INFO Signed narinfos id=53 count=16233lb # [ 165.332396] niks3-server[1135]: 2026/09/21 15:10:04 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6234lb # [ 165.340093] niks3-server[1135]: 2026/09/21 15:10:04 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6235lb # [ 165.350113] niks3-server[1135]: 2026/09/21 15:10:04 INFO Completed upload id=536236worker1 # [ 165.353110] nix-grpc-daemon[1518]: time=2026-09-21T15:10:05.005Z level=INFO msg="Upload complete. (332ms)"6237lb # [ 165.353278] niks3-server[1135]: 2026/09/21 15:10:04 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6238worker1 # [ 165.371963] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16239worker1 # [ 165.388280] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16240worker1 # [ 165.399499] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766241worker1 # [ 165.404909] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06242worker1 # [ 165.410481] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766243worker1 # [ 165.415463] nix-daemon[841]: reaped child process 1722, status = succeeded6244worker1 # [ 165.425843] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766245worker1 # [ 165.432264] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6246worker1 # [ 165.614917] nix-daemon[841]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6247worker1 # [ 165.632506] nix-daemon[841]: reaped child process 1752, status = succeeded6248worker1 # [ 165.804782] nix-daemon[841]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6249worker1 # [ 165.821428] nix-daemon[841]: reaped child process 1757, status = succeeded6250lb # [ 165.831773] niks3-server[1135]: 2026/09/21 15:10:05 INFO Received uploads request method=POST path=/api/pending_closures6251worker1 # [ 165.843323] nix-grpc-daemon[1518]: time=2026-09-21T15:10:05.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6252worker1 # [ 165.844958] nix-grpc-daemon[1518]: time=2026-09-21T15:10:05.495Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6253lb # [ 165.858167] niks3-server[1135]: 2026/09/21 15:10:05 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6254worker1 # [ 165.874152] nix-grpc-daemon[1518]: time=2026-09-21T15:10:05.526Z level=INFO msg="Uploading 1 narinfos"6255lb # [ 165.875059] niks3-server[1135]: 2026/09/21 15:10:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6256lb # [ 165.885967] niks3-server[1135]: 2026/09/21 15:10:05 INFO Signed narinfos id=54 count=16257lb # [ 165.891325] niks3-server[1135]: 2026/09/21 15:10:05 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6258lb # [ 165.899905] niks3-server[1135]: 2026/09/21 15:10:05 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6259worker1 # [ 165.907021] nix-grpc-daemon[1518]: time=2026-09-21T15:10:05.559Z level=INFO msg="Upload complete. (468ms)"6260lb # [ 165.904996] niks3-server[1135]: 2026/09/21 15:10:05 INFO Completed upload id=546261lb # [ 165.908707] niks3-server[1135]: 2026/09/21 15:10:05 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6262worker1 # [ 165.916529] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=5126263worker1 # [ 165.926390] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766264worker1 # [ 165.932325] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6265worker1 # [ 165.938106] nix-daemon[841]: reaped child process 1747, status = succeeded6266worker1 # [ 165.950632] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6267worker1 # [ 166.989450] nix-daemon[841]: reaped child process 1761, status = succeeded6268worker1 # [ 167.156713] nix-daemon[841]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)6269worker1 # [ 167.175524] nix-daemon[841]: reaped child process 1780, status = succeeded6270lb # [ 167.185342] niks3-server[1135]: 2026/09/21 15:10:06 INFO Received uploads request method=POST path=/api/pending_closures6271worker1 # [ 167.200493] nix-grpc-daemon[1518]: time=2026-09-21T15:10:06.849Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6272worker1 # [ 167.202240] nix-grpc-daemon[1518]: time=2026-09-21T15:10:06.849Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6273lb # [ 167.229825] niks3-server[1135]: 2026/09/21 15:10:06 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6274worker1 # [ 167.246621] nix-grpc-daemon[1518]: time=2026-09-21T15:10:06.898Z level=INFO msg="Uploading 1 narinfos"6275lb # [ 167.247578] niks3-server[1135]: 2026/09/21 15:10:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6276lb # [ 167.256287] niks3-server[1135]: 2026/09/21 15:10:06 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6277lb # [ 167.259806] niks3-server[1135]: 2026/09/21 15:10:06 INFO Signed narinfos id=55 count=16278lb # [ 167.263446] niks3-server[1135]: 2026/09/21 15:10:06 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6279lb # [ 167.270580] niks3-server[1135]: 2026/09/21 15:10:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6280lb # [ 167.274181] niks3-server[1135]: 2026/09/21 15:10:06 INFO Completed upload id=556281worker1 # [ 167.276503] nix-grpc-daemon[1518]: time=2026-09-21T15:10:06.928Z level=INFO msg="Upload complete. (316ms)"6282lb # [ 167.276258] niks3-server[1135]: 2026/09/21 15:10:06 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6283worker1 # [ 167.293082] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16284worker1 # [ 167.308376] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:06Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16285worker1 # [ 167.313466] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:06Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06286worker1 # [ 167.317063] nix-daemon[841]: reaped child process 1759, status = succeeded6287worker1 # [ 167.330898] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766288worker1 # [ 167.345835] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766289worker1 # [ 167.360169] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766290worker1 # [ 167.374238] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6291worker1 # [ 167.573733] nix-daemon[841]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6292worker1 # [ 167.591376] nix-daemon[841]: reaped child process 1789, status = succeeded6293worker1 # [ 167.763877] nix-daemon[841]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6294worker1 # [ 167.783747] nix-daemon[841]: reaped child process 1794, status = succeeded6295lb # [ 167.792929] niks3-server[1135]: 2026/09/21 15:10:07 INFO Received uploads request method=POST path=/api/pending_closures6296worker1 # [ 167.804784] nix-grpc-daemon[1518]: time=2026-09-21T15:10:07.456Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6297worker1 # [ 167.806322] nix-grpc-daemon[1518]: time=2026-09-21T15:10:07.456Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6298lb # [ 167.819246] niks3-server[1135]: 2026/09/21 15:10:07 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6299lb # [ 167.833637] niks3-server[1135]: 2026/09/21 15:10:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6300worker1 # [ 167.842521] nix-grpc-daemon[1518]: time=2026-09-21T15:10:07.494Z level=INFO msg="Uploading 1 narinfos"6301lb # [ 167.846383] niks3-server[1135]: 2026/09/21 15:10:07 INFO Signed narinfos id=56 count=16302lb # [ 167.852380] niks3-server[1135]: 2026/09/21 15:10:07 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6303lb # [ 167.859153] niks3-server[1135]: 2026/09/21 15:10:07 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6304lb # [ 167.865369] niks3-server[1135]: 2026/09/21 15:10:07 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6305lb # [ 167.870630] niks3-server[1135]: 2026/09/21 15:10:07 INFO Completed upload id=566306worker1 # [ 167.873375] nix-grpc-daemon[1518]: time=2026-09-21T15:10:07.525Z level=INFO msg="Upload complete. (484ms)"6307worker1 # [ 167.880153] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=5126308worker1 # [ 167.894444] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766309worker1 # [ 167.900741] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6310worker1 # [ 167.905418] nix-daemon[841]: reaped child process 1784, status = succeeded6311worker1 # [ 167.918142] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6312worker1 # [ 169.165769] nix-daemon[841]: reaped child process 1798, status = succeeded6313worker1 # [ 169.352118] nix-daemon[841]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6314worker1 # [ 169.370554] nix-daemon[841]: reaped child process 1817, status = succeeded6315lb # [ 169.381439] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received uploads request method=POST path=/api/pending_closures6316worker1 # [ 169.395404] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.046Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6317worker1 # [ 169.397862] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.046Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6318lb # [ 169.423675] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6319worker1 # [ 169.438708] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.090Z level=INFO msg="Uploading 1 narinfos"6320lb # [ 169.438954] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6321lb # [ 169.451182] niks3-server[1135]: 2026/09/21 15:10:09 INFO Signed narinfos id=57 count=16322lb # [ 169.456134] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6323worker1 # [ 169.468936] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.121Z level=INFO msg="Upload complete. (338ms)"6324lb # [ 169.466732] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6325lb # [ 169.470856] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6326lb # [ 169.476387] niks3-server[1135]: 2026/09/21 15:10:09 INFO Completed upload id=576327lb # [ 169.479399] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6328worker1 # [ 169.483042] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16329worker1 # [ 169.507100] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16330worker1 # [ 169.513164] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766331worker1 # [ 169.517413] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06332worker1 # [ 169.522741] nix-daemon[841]: reaped child process 1796, status = succeeded6333worker1 # [ 169.539159] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766334worker1 # [ 169.553865] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.206Z level=INFO msg="All 1 paths already cached"6335worker1 # [ 169.563049] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766336worker1 # [ 169.577300] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6337worker1 # [ 169.773578] nix-daemon[841]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6338worker1 # [ 169.791287] nix-daemon[841]: reaped child process 1826, status = succeeded6339worker1 # [ 169.965531] nix-daemon[841]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6340worker1 # [ 169.982811] nix-daemon[841]: reaped child process 1831, status = succeeded6341lb # [ 169.991462] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received uploads request method=POST path=/api/pending_closures6342worker1 # [ 170.004667] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.656Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6343worker1 # [ 170.006204] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.656Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6344lb # [ 170.019129] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6345worker1 # [ 170.034344] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.686Z level=INFO msg="Uploading 1 narinfos"6346lb # [ 170.034476] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6347lb # [ 170.044986] niks3-server[1135]: 2026/09/21 15:10:09 INFO Signed narinfos id=58 count=16348lb # [ 170.050208] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6349worker1 # [ 170.060109] nix-grpc-daemon[1518]: time=2026-09-21T15:10:09.712Z level=INFO msg="Upload complete. (467ms)"6350lb # [ 170.057154] niks3-server[1135]: 2026/09/21 15:10:09 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6351lb # [ 170.063367] niks3-server[1135]: 2026/09/21 15:10:09 INFO Completed upload id=586352worker1 # [ 170.068106] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=6246353lb # [ 170.067575] niks3-server[1135]: 2026/09/21 15:10:09 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6354worker1 # [ 170.078712] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766355worker1 # [ 170.085221] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6356worker1 # [ 170.089777] nix-daemon[841]: reaped child process 1821, status = succeeded6357worker1 # [ 170.100156] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6358worker1 # [ 171.332594] nix-daemon[841]: reaped child process 1835, status = succeeded6359worker1 # [ 171.500904] nix-daemon[841]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6360worker1 # [ 171.521066] nix-daemon[841]: reaped child process 1854, status = succeeded6361lb # [ 171.530581] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received uploads request method=POST path=/api/pending_closures6362worker1 # [ 171.544450] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.194Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6363worker1 # [ 171.546084] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.194Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6364lb # [ 171.572875] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6365lb # [ 171.583553] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6366worker1 # [ 171.596415] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.248Z level=INFO msg="Uploading 1 narinfos"6367lb # [ 171.595801] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6368lb # [ 171.603829] niks3-server[1135]: 2026/09/21 15:10:11 INFO Signed narinfos id=59 count=16369lb # [ 171.606844] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6370lb # [ 171.614278] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6371lb # [ 171.619092] niks3-server[1135]: 2026/09/21 15:10:11 INFO Completed upload id=596372worker1 # [ 171.621560] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.274Z level=INFO msg="Upload complete. (310ms)"6373lb # [ 171.622096] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6374worker1 # [ 171.638297] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16375worker1 # [ 171.656720] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16376worker1 # [ 171.660279] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766377worker1 # [ 171.663120] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06378worker1 # [ 171.667518] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766379worker1 # [ 171.671559] nix-daemon[841]: reaped child process 1833, status = succeeded6380worker1 # [ 171.674408] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.326Z level=INFO msg="All 1 paths already cached"6381worker1 # [ 171.681730] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766382worker1 # [ 171.692120] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6383worker1 # [ 171.884166] nix-daemon[841]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6384worker1 # [ 171.907202] nix-daemon[841]: reaped child process 1863, status = succeeded6385worker1 # [ 172.084417] nix-daemon[841]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6386worker1 # [ 172.101655] nix-daemon[841]: reaped child process 1868, status = succeeded6387lb # [ 172.110975] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received uploads request method=POST path=/api/pending_closures6388worker1 # [ 172.122776] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.774Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6389worker1 # [ 172.124341] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.775Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6390lb # [ 172.137700] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6391worker1 # [ 172.153979] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.806Z level=INFO msg="Uploading 1 narinfos"6392lb # [ 172.153445] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6393lb # [ 172.163544] niks3-server[1135]: 2026/09/21 15:10:11 INFO Signed narinfos id=60 count=16394lb # [ 172.173267] niks3-server[1135]: 2026/09/21 15:10:11 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6395worker1 # [ 172.181200] nix-grpc-daemon[1518]: time=2026-09-21T15:10:11.833Z level=INFO msg="Upload complete. (477ms)"6396lb # [ 172.178585] niks3-server[1135]: 2026/09/21 15:10:11 INFO Completed upload id=606397lb # [ 172.182816] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6398worker1 # [ 172.188229] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=6246399lb # [ 172.187531] niks3-server[1135]: 2026/09/21 15:10:11 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6400worker1 # [ 172.199476] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766401worker1 # [ 172.204945] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6402worker1 # [ 172.209479] nix-daemon[841]: reaped child process 1858, status = succeeded6403worker1 # [ 172.221025] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6404worker1 # [ 173.867385] nix-daemon[841]: reaped child process 1872, status = succeeded6405worker1 # [ 174.058552] nix-daemon[841]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6406worker1 # [ 174.075193] nix-daemon[841]: reaped child process 1891, status = succeeded6407lb # [ 174.084897] niks3-server[1135]: 2026/09/21 15:10:13 INFO Received uploads request method=POST path=/api/pending_closures6408worker1 # [ 174.098523] nix-grpc-daemon[1518]: time=2026-09-21T15:10:13.749Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6409worker1 # [ 174.100784] nix-grpc-daemon[1518]: time=2026-09-21T15:10:13.749Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6410lb # [ 174.115900] niks3-server[1135]: 2026/09/21 15:10:13 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6411lb # [ 174.132743] niks3-server[1135]: 2026/09/21 15:10:13 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6412lb # [ 174.144109] niks3-server[1135]: 2026/09/21 15:10:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6413worker1 # [ 174.152413] nix-grpc-daemon[1518]: time=2026-09-21T15:10:13.804Z level=INFO msg="Uploading 1 narinfos"6414lb # [ 174.149259] niks3-server[1135]: 2026/09/21 15:10:13 INFO Signed narinfos id=61 count=16415lb # [ 174.155418] niks3-server[1135]: 2026/09/21 15:10:13 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6416lb # [ 174.166370] niks3-server[1135]: 2026/09/21 15:10:13 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6417worker1 # [ 174.176391] nix-grpc-daemon[1518]: time=2026-09-21T15:10:13.828Z level=INFO msg="Upload complete. (344ms)"6418lb # [ 174.175104] niks3-server[1135]: 2026/09/21 15:10:13 INFO Completed upload id=616419lb # [ 174.178703] niks3-server[1135]: 2026/09/21 15:10:13 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6420worker1 # [ 174.193945] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16421worker1 # [ 174.212942] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16422worker1 # [ 174.229233] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766423worker1 # [ 174.237831] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06424worker1 # [ 174.251588] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766425worker1 # [ 174.256942] nix-grpc-daemon[1518]: time=2026-09-21T15:10:13.876Z level=INFO msg="All 1 paths already cached"6426worker1 # [ 174.261532] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6427worker1 # [ 174.264501] nix-daemon[841]: reaped child process 1870, status = succeeded6428worker1 # [ 174.266622] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766429worker1 # [ 174.445835] nix-daemon[841]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6430worker1 # [ 174.462974] nix-daemon[841]: reaped child process 1900, status = succeeded6431worker1 # [ 174.632269] nix-daemon[841]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6432worker1 # [ 174.650824] nix-daemon[841]: reaped child process 1905, status = succeeded6433lb # [ 174.659101] niks3-server[1135]: 2026/09/21 15:10:14 INFO Received uploads request method=POST path=/api/pending_closures6434worker1 # [ 174.671720] nix-grpc-daemon[1518]: time=2026-09-21T15:10:14.323Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6435worker1 # [ 174.673342] nix-grpc-daemon[1518]: time=2026-09-21T15:10:14.323Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6436worker1 # [ 174.700368] nix-grpc-daemon[1518]: time=2026-09-21T15:10:14.352Z level=INFO msg="Uploading 1 narinfos"6437lb # [ 174.690324] niks3-server[1135]: 2026/09/21 15:10:14 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6438lb # [ 174.706101] niks3-server[1135]: 2026/09/21 15:10:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6439lb # [ 174.715518] niks3-server[1135]: 2026/09/21 15:10:14 INFO Signed narinfos id=62 count=16440lb # [ 174.723301] niks3-server[1135]: 2026/09/21 15:10:14 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6441worker1 # [ 174.731325] nix-grpc-daemon[1518]: time=2026-09-21T15:10:14.383Z level=INFO msg="Upload complete. (470ms)"6442lb # [ 174.730863] niks3-server[1135]: 2026/09/21 15:10:14 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6443lb # [ 174.737109] niks3-server[1135]: 2026/09/21 15:10:14 INFO Completed upload id=626444worker1 # [ 174.739733] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=6246445lb # [ 174.741000] niks3-server[1135]: 2026/09/21 15:10:14 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6446worker1 # [ 174.751023] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766447worker1 # [ 174.757084] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6448worker1 # [ 174.761711] nix-daemon[841]: reaped child process 1895, status = succeeded6449worker1 # [ 174.774712] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6450worker1 # [ 176.016772] nix-daemon[841]: reaped child process 1909, status = succeeded6451worker1 # [ 176.198741] nix-daemon[841]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6452worker1 # [ 176.216298] nix-daemon[841]: reaped child process 1928, status = succeeded6453lb # [ 176.231327] niks3-server[1135]: 2026/09/21 15:10:15 INFO Received uploads request method=POST path=/api/pending_closures6454worker1 # [ 176.240370] nix-grpc-daemon[1518]: time=2026-09-21T15:10:15.892Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6455worker1 # [ 176.243576] nix-grpc-daemon[1518]: time=2026-09-21T15:10:15.894Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6456lb # [ 176.264383] niks3-server[1135]: 2026/09/21 15:10:15 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6457lb # [ 176.276447] niks3-server[1135]: 2026/09/21 15:10:15 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6458worker1 # [ 176.290851] nix-grpc-daemon[1518]: time=2026-09-21T15:10:15.943Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 176.289428] niks3-server[1135]: 2026/09/21 15:10:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6460lb # [ 176.298066] niks3-server[1135]: 2026/09/21 15:10:15 INFO Signed narinfos id=63 count=16461lb # [ 176.301687] niks3-server[1135]: 2026/09/21 15:10:15 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6462lb # [ 176.310088] niks3-server[1135]: 2026/09/21 15:10:15 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6463lb # [ 176.316207] niks3-server[1135]: 2026/09/21 15:10:15 INFO Completed upload id=636464worker1 # [ 176.318386] nix-grpc-daemon[1518]: time=2026-09-21T15:10:15.970Z level=INFO msg="Upload complete. (333ms)"6465lb # [ 176.318873] niks3-server[1135]: 2026/09/21 15:10:15 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6466worker1 # [ 176.336867] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16467worker1 # [ 176.352506] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:15Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16468worker1 # [ 176.360458] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766469worker1 # [ 176.366233] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:15Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06470worker1 # [ 176.371566] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766471worker1 # [ 176.376365] nix-daemon[841]: reaped child process 1907, status = succeeded6472worker1 # [ 176.378338] nix-grpc-daemon[1518]: time=2026-09-21T15:10:16.028Z level=INFO msg="All 2 paths already cached"6473worker1 # [ 176.384376] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766474worker1 # [ 176.391429] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6475worker1 # [ 176.573827] nix-daemon[841]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6476worker1 # [ 176.591127] nix-daemon[841]: reaped child process 1937, status = succeeded6477worker1 # [ 176.760362] nix-daemon[841]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6478worker1 # [ 176.774952] nix-daemon[841]: reaped child process 1942, status = succeeded6479lb # [ 176.787174] niks3-server[1135]: 2026/09/21 15:10:16 INFO Received uploads request method=POST path=/api/pending_closures6480worker1 # [ 176.800700] nix-grpc-daemon[1518]: time=2026-09-21T15:10:16.452Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6481worker1 # [ 176.802168] nix-grpc-daemon[1518]: time=2026-09-21T15:10:16.452Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6482lb # [ 176.817885] niks3-server[1135]: 2026/09/21 15:10:16 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6483worker1 # [ 176.834034] nix-grpc-daemon[1518]: time=2026-09-21T15:10:16.486Z level=INFO msg="Uploading 1 narinfos"6484lb # [ 176.834008] niks3-server[1135]: 2026/09/21 15:10:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6485lb # [ 176.844227] niks3-server[1135]: 2026/09/21 15:10:16 INFO Signed narinfos id=64 count=16486lb # [ 176.854335] niks3-server[1135]: 2026/09/21 15:10:16 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6487lb # [ 176.860237] niks3-server[1135]: 2026/09/21 15:10:16 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6488worker1 # [ 176.867276] nix-grpc-daemon[1518]: time=2026-09-21T15:10:16.519Z level=INFO msg="Upload complete. (464ms)"6489lb # [ 176.864570] niks3-server[1135]: 2026/09/21 15:10:16 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6490lb # [ 176.868470] niks3-server[1135]: 2026/09/21 15:10:16 INFO Completed upload id=646491worker1 # [ 176.877078] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=7526492worker1 # [ 176.887681] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766493worker1 # [ 176.896284] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6494worker1 # [ 176.899212] nix-daemon[841]: reaped child process 1932, status = succeeded6495worker1 # [ 176.912197] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6496worker1 # [ 178.558817] nix-daemon[841]: reaped child process 1946, status = succeeded6497worker1 # [ 178.747160] nix-daemon[841]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6498worker1 # [ 178.763583] nix-daemon[841]: reaped child process 1965, status = succeeded6499lb # [ 178.777485] niks3-server[1135]: 2026/09/21 15:10:18 INFO Received uploads request method=POST path=/api/pending_closures6500worker1 # [ 178.791402] nix-grpc-daemon[1518]: time=2026-09-21T15:10:18.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6501worker1 # [ 178.793273] nix-grpc-daemon[1518]: time=2026-09-21T15:10:18.442Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6502lb # [ 178.814812] niks3-server[1135]: 2026/09/21 15:10:18 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6503worker1 # [ 178.829196] nix-grpc-daemon[1518]: time=2026-09-21T15:10:18.481Z level=INFO msg="Uploading 1 narinfos"6504lb # [ 178.825039] niks3-server[1135]: 2026/09/21 15:10:18 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6505lb # [ 178.837712] niks3-server[1135]: 2026/09/21 15:10:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6506lb # [ 178.843230] niks3-server[1135]: 2026/09/21 15:10:18 INFO Signed narinfos id=65 count=16507lb # [ 178.848299] niks3-server[1135]: 2026/09/21 15:10:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6508lb # [ 178.852282] niks3-server[1135]: 2026/09/21 15:10:18 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6509worker1 # [ 178.854526] nix-grpc-daemon[1518]: time=2026-09-21T15:10:18.506Z level=INFO msg="Upload complete. (330ms)"6510lb # [ 178.855529] niks3-server[1135]: 2026/09/21 15:10:18 INFO Completed upload id=656511lb # [ 178.858923] niks3-server[1135]: 2026/09/21 15:10:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6512worker1 # [ 178.866852] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16513worker1 # [ 178.882380] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:18Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16514worker1 # [ 178.886640] nix-daemon[841]: reaped child process 1944, status = succeeded6515worker1 # [ 178.911271] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766516worker1 # [ 178.926493] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766517worker1 # [ 178.941730] nix-grpc-daemon[1518]: time=2026-09-21T15:10:18.594Z level=INFO msg="All 2 paths already cached"6518worker1 # [ 178.950663] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766519worker1 # [ 178.964490] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6520worker1 # [ 179.171025] nix-daemon[841]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6521worker1 # [ 179.188277] nix-daemon[841]: reaped child process 1972, status = succeeded6522worker1 # [ 179.361413] nix-daemon[841]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)6523worker1 # [ 179.378238] nix-daemon[841]: reaped child process 1977, status = succeeded6524lb # [ 179.386828] niks3-server[1135]: 2026/09/21 15:10:19 INFO Received uploads request method=POST path=/api/pending_closures6525worker1 # [ 179.398579] nix-grpc-daemon[1518]: time=2026-09-21T15:10:19.050Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6526worker1 # [ 179.400111] nix-grpc-daemon[1518]: time=2026-09-21T15:10:19.050Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6527lb # [ 179.413780] niks3-server[1135]: 2026/09/21 15:10:19 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6528worker1 # [ 179.427875] nix-grpc-daemon[1518]: time=2026-09-21T15:10:19.080Z level=INFO msg="Uploading 1 narinfos"6529lb # [ 179.428557] niks3-server[1135]: 2026/09/21 15:10:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6530lb # [ 179.439554] niks3-server[1135]: 2026/09/21 15:10:19 INFO Signed narinfos id=66 count=16531lb # [ 179.444418] niks3-server[1135]: 2026/09/21 15:10:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6532lb # [ 179.454203] niks3-server[1135]: 2026/09/21 15:10:19 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6533worker1 # [ 179.462431] nix-grpc-daemon[1518]: time=2026-09-21T15:10:19.114Z level=INFO msg="Upload complete. (481ms)"6534lb # [ 179.459703] niks3-server[1135]: 2026/09/21 15:10:19 INFO Completed upload id=666535lb # [ 179.463845] niks3-server[1135]: 2026/09/21 15:10:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6536worker1 # [ 179.468223] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=7526537worker1 # [ 179.479059] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766538worker1 # [ 179.485261] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6539worker1 # [ 179.490171] nix-daemon[841]: reaped child process 1967, status = succeeded6540worker1 # [ 179.501834] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6541worker1 # [ 180.720994] nix-daemon[841]: reaped child process 1981, status = succeeded6542worker1 # [ 180.875840] nix-daemon[841]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)6543worker1 # [ 180.894438] nix-daemon[841]: reaped child process 2000, status = succeeded6544lb # [ 180.905879] niks3-server[1135]: 2026/09/21 15:10:20 INFO Received uploads request method=POST path=/api/pending_closures6545worker1 # [ 180.916395] nix-grpc-daemon[1518]: time=2026-09-21T15:10:20.568Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6546worker1 # [ 180.917840] nix-grpc-daemon[1518]: time=2026-09-21T15:10:20.568Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6547lb # [ 180.946085] niks3-server[1135]: 2026/09/21 15:10:20 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6548worker1 # [ 180.958566] nix-grpc-daemon[1518]: time=2026-09-21T15:10:20.610Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 180.963523] niks3-server[1135]: 2026/09/21 15:10:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6550lb # [ 180.972300] niks3-server[1135]: 2026/09/21 15:10:20 INFO Signed narinfos id=67 count=16551lb # [ 180.978591] niks3-server[1135]: 2026/09/21 15:10:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6552lb # [ 180.982291] niks3-server[1135]: 2026/09/21 15:10:20 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6553lb # [ 180.986942] niks3-server[1135]: 2026/09/21 15:10:20 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6554worker1 # [ 180.991569] nix-grpc-daemon[1518]: time=2026-09-21T15:10:20.641Z level=INFO msg="Upload complete. (278ms)"6555lb # [ 180.991927] niks3-server[1135]: 2026/09/21 15:10:20 INFO Completed upload id=676556lb # [ 180.994394] niks3-server[1135]: 2026/09/21 15:10:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6557worker1 # [ 181.002961] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16558worker1 # [ 181.014427] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:20Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16559worker1 # [ 181.018523] nix-daemon[841]: reaped child process 1979, status = succeeded6560worker1 # [ 181.046540] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766561worker1 # [ 181.061502] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584766562worker1 # [ 181.075863] nix-grpc-daemon[1518]: time=2026-09-21T15:10:20.728Z level=INFO msg="All 2 paths already cached"6563worker1 # [ 181.084847] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584766564worker1 # [ 181.098251] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6565worker1 # [ 181.297201] nix-daemon[841]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6566worker1 # [ 181.314182] nix-daemon[841]: reaped child process 2007, status = succeeded6567worker1 # [ 181.484681] nix-daemon[841]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)6568worker1 # [ 181.499788] nix-daemon[841]: reaped child process 2012, status = succeeded6569lb # [ 181.512033] niks3-server[1135]: 2026/09/21 15:10:21 INFO Received uploads request method=POST path=/api/pending_closures6570worker1 # [ 181.524211] nix-grpc-daemon[1518]: time=2026-09-21T15:10:21.176Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6571worker1 # [ 181.525696] nix-grpc-daemon[1518]: time=2026-09-21T15:10:21.176Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6572lb # [ 181.543907] niks3-server[1135]: 2026/09/21 15:10:21 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6573worker1 # [ 181.557410] nix-grpc-daemon[1518]: time=2026-09-21T15:10:21.209Z level=INFO msg="Uploading 1 narinfos"6574lb # [ 181.557586] niks3-server[1135]: 2026/09/21 15:10:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6575lb # [ 181.568282] niks3-server[1135]: 2026/09/21 15:10:21 INFO Signed narinfos id=68 count=16576lb # [ 181.573503] niks3-server[1135]: 2026/09/21 15:10:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6577worker1 # [ 181.582383] nix-grpc-daemon[1518]: time=2026-09-21T15:10:21.234Z level=INFO msg="Upload complete. (468ms)"6578lb # [ 181.581472] niks3-server[1135]: 2026/09/21 15:10:21 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6579worker1 # [ 181.588261] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=0 paths=1 nar_bytes_in=7526580lb # [ 181.586193] niks3-server[1135]: 2026/09/21 15:10:21 INFO Completed upload id=686581lb # [ 181.590341] niks3-server[1135]: 2026/09/21 15:10:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6582worker1 # [ 181.602396] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584766583worker1 # [ 181.606548] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6584worker1 # [ 181.615655] nix-daemon[841]: reaped child process 2002, status = succeeded6585worker1 # [ 181.623138] nix-daemon[841]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6586worker1 # [ 183.055082] nix-daemon[841]: reaped child process 2016, status = succeeded6587worker1 # [ 183.225109] nix-daemon[841]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6588worker1 # [ 183.243574] nix-daemon[841]: reaped child process 2035, status = succeeded6589lb # [ 183.254027] niks3-server[1135]: 2026/09/21 15:10:22 INFO Received uploads request method=POST path=/api/pending_closures6590worker1 # [ 183.266768] nix-grpc-daemon[1518]: time=2026-09-21T15:10:22.918Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6591worker1 # [ 183.268450] nix-grpc-daemon[1518]: time=2026-09-21T15:10:22.918Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6592lb # [ 183.287848] niks3-server[1135]: 2026/09/21 15:10:22 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6593lb # [ 183.300614] niks3-server[1135]: 2026/09/21 15:10:22 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6594worker1 # [ 183.314266] nix-grpc-daemon[1518]: time=2026-09-21T15:10:22.966Z level=INFO msg="Uploading 1 narinfos"6595lb # [ 183.313748] niks3-server[1135]: 2026/09/21 15:10:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6596lb # [ 183.321532] niks3-server[1135]: 2026/09/21 15:10:22 INFO Signed narinfos id=69 count=16597lb # [ 183.325267] niks3-server[1135]: 2026/09/21 15:10:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6598worker1 # [ 183.334434] nix-grpc-daemon[1518]: time=2026-09-21T15:10:22.986Z level=INFO msg="Upload complete. (302ms)"6599lb # [ 183.332647] niks3-server[1135]: 2026/09/21 15:10:22 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6600lb # [ 183.337922] niks3-server[1135]: 2026/09/21 15:10:22 INFO Completed upload id=696601lb # [ 183.340570] niks3-server[1135]: 2026/09/21 15:10:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6602worker1 # [ 183.344917] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58476 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16603worker1 # [ 183.360719] nix-daemon[841]: reaped child process 2014, status = succeeded6604client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.40 seconds)6605worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6606worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6607client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6608client # [ 183.570195] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6609client # Running as unit: gpu.service; invocation ID: b377c56c5dcb4b269c6191089b3fdcc46610client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6611lb: must succeed: curl -sf localhost:9901/clusters6612lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6613lb: must succeed: curl -sf localhost:9901/clusters6614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6615worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6616worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6617worker2 # [ 183.871392] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585866618worker2 # [ 183.886776] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6619client # [ 183.860428] nix[1366]: don't know how to build these paths:6620client # [ 183.861610] nix[1366]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6621worker2 # [ 183.909680] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585866622worker2 # [ 183.926415] nix-grpc-daemon[1911]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)6623worker2 # [ 183.955072] nix-grpc-daemon[1911]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)6624worker2 # [ 184.127858] nix-grpc-daemon[1911]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5)6625worker2 # [ 184.530698] nix-grpc-daemon[1911]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 212 ms (attempt 4/5)6626worker2 # [ 184.743431] nix-grpc-daemon[1911]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6627worker2 # [ 184.749948] nix-grpc-daemon[1911]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6628worker1 # [ 184.741364] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:24Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06629client # [ 184.747563] nix[1366]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6630lb: must succeed: curl -sf localhost:9901/clusters6631lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6632lb: must succeed: curl -sf localhost:9901/clusters6633lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6634worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6635worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6636client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6637lb # [ 191.167075] envoy[914]: conn peer=[2001:db8:1::1]:54402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27273 tx=11550 ms=200696638client # [ 214.749076] nix[1366]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6639client # [ 214.758223] nix[1366]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6640client # [ 214.803481] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6641client # [ 214.813789] systemd[1]: gpu.service: Failed with result 'exit-code'.6642client # [ 214.817107] systemd[1]: gpu.service: Consumed 181ms CPU time over 31.241s wall clock time, 35.4M memory peak, 4K written to disk, 5.4K incoming IP traffic, 5.1K outgoing IP traffic.6643client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 30.36 seconds)6644client: must succeed: journalctl -u gpu -o cat6645client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6646lb: must succeed: curl -sf localhost:9901/clusters6647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6648lb: must succeed: curl -sf localhost:9901/clusters6649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6650worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6651worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6652(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.61 seconds)6653subtest: low disk drains a worker and builds go to the other6654worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6655worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.09 seconds)6656worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6657worker1 # [ 217.039740] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:56Z level=info event=unhealthy reason=min_free6658worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6659client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26660client # don't know how to build these paths:6661client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6662worker1 # [ 218.109659] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584766663worker2 # [ 218.158641] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585866664worker1 # [ 218.143307] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:57Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06665worker1 # [ 218.149461] nix-grpc-daemon[1442]: ts=2026-09-21T15:10:57Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16666worker2 # [ 218.234117] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866667worker2 # [ 218.250418] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585866668client # copying 1 paths to 'lb:50051'...6669client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6670worker2 # [ 218.266528] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585866671worker2 # [ 218.282517] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6672worker2 # [ 218.311011] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:57Z level=info event=niks3_push_started pid=19716673worker2 # [ 218.531525] nix-grpc-daemon[1971]: time=2026-09-21T15:10:58.170Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6674worker2 # [ 218.711973] nix-daemon[863]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6675worker2 # [ 218.729395] nix-daemon[863]: reaped child process 1987, status = succeeded6676worker2 # [ 218.907844] nix-daemon[863]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6677worker2 # [ 218.924408] nix-daemon[863]: reaped child process 1992, status = succeeded6678lb # [ 218.894193] niks3-server[1135]: 2026/09/21 15:10:58 INFO Received uploads request method=POST path=/api/pending_closures6679worker2 # [ 218.952072] nix-grpc-daemon[1971]: time=2026-09-21T15:10:58.590Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6680worker2 # [ 218.953538] nix-grpc-daemon[1971]: time=2026-09-21T15:10:58.590Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6681lb # [ 218.933085] niks3-server[1135]: 2026/09/21 15:10:58 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6682worker2 # [ 218.994058] nix-grpc-daemon[1971]: time=2026-09-21T15:10:58.632Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 218.947747] niks3-server[1135]: 2026/09/21 15:10:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6684lb # [ 218.959501] niks3-server[1135]: 2026/09/21 15:10:58 INFO Signed narinfos id=70 count=16685lb # [ 218.965840] niks3-server[1135]: 2026/09/21 15:10:58 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6686lb # [ 218.980497] niks3-server[1135]: 2026/09/21 15:10:58 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6687lb # [ 218.986136] niks3-server[1135]: 2026/09/21 15:10:58 INFO Completed upload id=706688worker2 # [ 219.031695] nix-grpc-daemon[1971]: time=2026-09-21T15:10:58.670Z level=INFO msg="Upload complete. (495ms)"6689lb # [ 218.989556] niks3-server[1135]: 2026/09/21 15:10:58 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6690worker2 # [ 219.040358] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=0 paths=1 nar_bytes_in=5126691worker2 # [ 219.058892] nix-grpc-daemon[1911]: ts=2026-09-21T15:10:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866692worker2 # [ 219.065851] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6693worker2 # [ 219.071437] nix-daemon[863]: reaped child process 1969, status = succeeded6694client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6695worker2 # [ 219.084188] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6696client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6697client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6698client # farm-a> LOG-farm-a-drain6699worker2 # [ 220.730898] nix-daemon[863]: reaped child process 1996, status = succeeded6700worker2 # [ 220.920432] nix-daemon[863]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6701worker2 # [ 220.939421] nix-daemon[863]: reaped child process 2015, status = succeeded6702lb # [ 220.913457] niks3-server[1135]: 2026/09/21 15:11:00 INFO Received uploads request method=POST path=/api/pending_closures6703worker2 # [ 220.972223] nix-grpc-daemon[1971]: time=2026-09-21T15:11:00.607Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6704worker2 # [ 220.973774] nix-grpc-daemon[1971]: time=2026-09-21T15:11:00.607Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6705lb # [ 220.952421] niks3-server[1135]: 2026/09/21 15:11:00 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6706lb # [ 220.962833] niks3-server[1135]: 2026/09/21 15:11:00 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6707lb # [ 220.973912] niks3-server[1135]: 2026/09/21 15:11:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6708worker2 # [ 221.025997] nix-grpc-daemon[1971]: time=2026-09-21T15:11:00.663Z level=INFO msg="Uploading 1 narinfos"6709lb # [ 220.979636] niks3-server[1135]: 2026/09/21 15:11:00 INFO Signed narinfos id=71 count=16710lb # [ 220.984375] niks3-server[1135]: 2026/09/21 15:11:00 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6711lb # [ 220.994400] niks3-server[1135]: 2026/09/21 15:11:00 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6712lb # [ 221.002935] niks3-server[1135]: 2026/09/21 15:11:00 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6713lb # [ 221.007826] niks3-server[1135]: 2026/09/21 15:11:00 INFO Completed upload id=716714worker2 # [ 221.052893] nix-grpc-daemon[1971]: time=2026-09-21T15:11:00.691Z level=INFO msg="Upload complete. (354ms)"6715worker2 # [ 221.065093] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=2 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16716worker1 # [ 221.026020] nix-grpc-daemon[1442]: ts=2026-09-21T15:11:00Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06717worker1 # [ 221.032078] nix-grpc-daemon[1442]: ts=2026-09-21T15:11:00Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16718worker2 # [ 221.083295] nix-daemon[863]: reaped child process 1994, status = succeeded6719worker2 # [ 221.113784] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866720worker2 # [ 221.130438] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585866721worker2 # [ 221.148078] nix-grpc-daemon[1971]: time=2026-09-21T15:11:00.786Z level=INFO msg="All 1 paths already cached"6722client # copying 1 paths to 'lb:50051'...6723worker2 # [ 221.157059] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585866724client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6725worker2 # [ 221.175824] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6726lb # [ 221.171963] envoy[914]: conn peer=[2001:db8:1::1]:47622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=914 tx=541 ms=309486727worker2 # [ 221.381541] nix-daemon[863]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6728worker2 # [ 221.401183] nix-daemon[863]: reaped child process 2022, status = succeeded6729worker2 # [ 221.575148] nix-daemon[863]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6730worker2 # [ 221.595358] nix-daemon[863]: reaped child process 2027, status = succeeded6731lb # [ 221.561070] niks3-server[1135]: 2026/09/21 15:11:01 INFO Received uploads request method=POST path=/api/pending_closures6732worker2 # [ 221.614876] nix-grpc-daemon[1971]: time=2026-09-21T15:11:01.253Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6733worker2 # [ 221.617009] nix-grpc-daemon[1971]: time=2026-09-21T15:11:01.255Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6734lb # [ 221.589476] niks3-server[1135]: 2026/09/21 15:11:01 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6735worker2 # [ 221.649837] nix-grpc-daemon[1971]: time=2026-09-21T15:11:01.288Z level=INFO msg="Uploading 1 narinfos"6736lb # [ 221.607609] niks3-server[1135]: 2026/09/21 15:11:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6737lb # [ 221.617966] niks3-server[1135]: 2026/09/21 15:11:01 INFO Signed narinfos id=72 count=16738lb # [ 221.623523] niks3-server[1135]: 2026/09/21 15:11:01 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6739worker2 # [ 221.676973] nix-grpc-daemon[1971]: time=2026-09-21T15:11:01.315Z level=INFO msg="Upload complete. (483ms)"6740lb # [ 221.633074] niks3-server[1135]: 2026/09/21 15:11:01 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6741lb # [ 221.638227] niks3-server[1135]: 2026/09/21 15:11:01 INFO Completed upload id=726742worker2 # [ 221.683084] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=0 paths=1 nar_bytes_in=6326743lb # [ 221.641653] niks3-server[1135]: 2026/09/21 15:11:01 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6744worker2 # [ 221.696256] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866745worker2 # [ 221.702538] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6746worker2 # [ 221.707538] nix-daemon[863]: reaped child process 2017, status = succeeded6747client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6748worker2 # [ 221.717895] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6749client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6750client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6751client # farm-b> LOG-farm-b-drain6752worker2 # [ 222.566657] nix-daemon[863]: reaped child process 2031, status = succeeded6753worker2 # [ 222.751686] nix-daemon[863]: accepted connection from pid 2047, user nix-grpc-daemon (trusted)6754worker2 # [ 222.773017] nix-daemon[863]: reaped child process 2050, status = succeeded6755lb # [ 222.740779] niks3-server[1135]: 2026/09/21 15:11:02 INFO Received uploads request method=POST path=/api/pending_closures6756worker2 # [ 222.798314] nix-grpc-daemon[1971]: time=2026-09-21T15:11:02.436Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6757worker2 # [ 222.799984] nix-grpc-daemon[1971]: time=2026-09-21T15:11:02.436Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6758lb # [ 222.784481] niks3-server[1135]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6759worker2 # [ 222.840699] nix-grpc-daemon[1971]: time=2026-09-21T15:11:02.478Z level=INFO msg="Uploading 1 narinfos"6760lb # [ 222.795128] niks3-server[1135]: 2026/09/21 15:11:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6761lb # [ 222.806552] niks3-server[1135]: 2026/09/21 15:11:02 INFO Signed narinfos id=73 count=16762lb # [ 222.811793] niks3-server[1135]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6763lb # [ 222.819686] niks3-server[1135]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6764lb # [ 222.823158] niks3-server[1135]: 2026/09/21 15:11:02 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6765lb # [ 222.827301] niks3-server[1135]: 2026/09/21 15:11:02 INFO Completed upload id=736766worker2 # [ 222.872444] nix-grpc-daemon[1971]: time=2026-09-21T15:11:02.511Z level=INFO msg="Upload complete. (342ms)"6767lb # [ 222.829946] niks3-server[1135]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6768worker1 # [ 222.853678] nix-grpc-daemon[1442]: ts=2026-09-21T15:11:02Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06769worker2 # [ 222.885286] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16770worker2 # [ 222.904733] nix-daemon[863]: reaped child process 2029, status = succeeded6771worker1 # [ 222.859669] nix-grpc-daemon[1442]: ts=2026-09-21T15:11:02Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16772worker2 # [ 222.938280] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866773worker2 # [ 222.955415] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585866774worker2 # [ 222.970606] nix-grpc-daemon[1971]: time=2026-09-21T15:11:02.609Z level=INFO msg="All 2 paths already cached"6775client # copying 1 paths to 'lb:50051'...6776client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6777worker2 # [ 222.981835] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585866778worker2 # [ 222.996823] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6779worker2 # [ 223.201611] nix-daemon[863]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6780worker2 # [ 223.220783] nix-daemon[863]: reaped child process 2057, status = succeeded6781worker2 # [ 223.397349] nix-daemon[863]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6782worker2 # [ 223.415133] nix-daemon[863]: reaped child process 2062, status = succeeded6783lb # [ 223.385846] niks3-server[1135]: 2026/09/21 15:11:03 INFO Received uploads request method=POST path=/api/pending_closures6784worker2 # [ 223.441262] nix-grpc-daemon[1971]: time=2026-09-21T15:11:03.079Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6785worker2 # [ 223.442777] nix-grpc-daemon[1971]: time=2026-09-21T15:11:03.079Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6786lb # [ 223.417529] niks3-server[1135]: 2026/09/21 15:11:03 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6787lb # [ 223.429910] niks3-server[1135]: 2026/09/21 15:11:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6788worker2 # [ 223.482846] nix-grpc-daemon[1971]: time=2026-09-21T15:11:03.121Z level=INFO msg="Uploading 1 narinfos"6789lb # [ 223.437438] niks3-server[1135]: 2026/09/21 15:11:03 INFO Signed narinfos id=74 count=16790lb # [ 223.444328] niks3-server[1135]: 2026/09/21 15:11:03 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6791lb # [ 223.453794] niks3-server[1135]: 2026/09/21 15:11:03 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6792worker2 # [ 223.504142] nix-grpc-daemon[1971]: time=2026-09-21T15:11:03.143Z level=INFO msg="Upload complete. (490ms)"6793lb # [ 223.460847] niks3-server[1135]: 2026/09/21 15:11:03 INFO Completed upload id=746794lb # [ 223.464445] niks3-server[1135]: 2026/09/21 15:11:03 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6795worker2 # [ 223.512109] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=0 paths=1 nar_bytes_in=7606796worker2 # [ 223.521132] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585866797worker2 # [ 223.532899] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6798worker2 # [ 223.537892] nix-daemon[863]: reaped child process 2052, status = succeeded6799client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6800client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6801worker2 # [ 223.552269] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6802client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6803client # farm-top> LOG-farm-top-drain6804worker2 # [ 224.787815] nix-daemon[863]: reaped child process 2066, status = succeeded6805worker2 # [ 224.960094] nix-daemon[863]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)6806worker2 # [ 224.977069] nix-daemon[863]: reaped child process 2085, status = succeeded6807lb # [ 224.947213] niks3-server[1135]: 2026/09/21 15:11:04 INFO Received uploads request method=POST path=/api/pending_closures6808worker2 # [ 225.003634] nix-grpc-daemon[1971]: time=2026-09-21T15:11:04.640Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6809worker2 # [ 225.005290] nix-grpc-daemon[1971]: time=2026-09-21T15:11:04.640Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6810lb # [ 224.981791] niks3-server[1135]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6811worker2 # [ 225.050641] nix-grpc-daemon[1971]: time=2026-09-21T15:11:04.689Z level=INFO msg="Uploading 1 narinfos"6812lb # [ 224.996304] niks3-server[1135]: 2026/09/21 15:11:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6813lb # [ 225.010451] niks3-server[1135]: 2026/09/21 15:11:04 INFO Signed narinfos id=75 count=16814lb # [ 225.017328] niks3-server[1135]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6815lb # [ 225.023557] niks3-server[1135]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6816lb # [ 225.032520] niks3-server[1135]: 2026/09/21 15:11:04 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6817lb # [ 225.037377] niks3-server[1135]: 2026/09/21 15:11:04 INFO Completed upload id=756818worker2 # [ 225.082894] nix-grpc-daemon[1971]: time=2026-09-21T15:11:04.721Z level=INFO msg="Upload complete. (320ms)"6819lb # [ 225.040210] niks3-server[1135]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6820worker2 # [ 225.093071] nix-grpc-daemon[1911]: ts=2026-09-21T15:11:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58586 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16821worker2 # [ 225.109710] nix-daemon[863]: reaped child process 2064, status = succeeded6822client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.26 seconds)6823client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6824client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6825worker1: must succeed: rm /nix/.rw-store/fill6826worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6827worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6828worker1 # [ 226.044369] nix-grpc-daemon[1442]: ts=2026-09-21T15:11:05Z level=info event=healthy reason=min_free6829worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.13 seconds)6830(finished: subtest: low disk drains a worker and builds go to the other, in 11.09 seconds)6831(finished: run the VM test script, in 227.67 seconds)6832test script finished in 227.96s6833cleanup6834kill QemuMachine (pid 49)6835client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6836kill QemuMachine (pid 50)6837lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6838kill QemuMachine (pid 51)6839worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6840kill QemuMachine (pid 52)6841worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6842(finished: cleanup, in 1.04 seconds)6843additionally exposed symbols:6844 client, lb, worker1, worker2,6845 vlan1,6846 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh