vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #463
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14client # Formatting '/build/vm-state-client/tmp.zLtbM7u2T1', fmt=raw size=107374182415client: QEMU running (pid 50)16worker2 # Formatting '/build/vm-state-worker2/tmp.jB20MmqH5N', fmt=raw size=107374182417lb: QEMU running (pid 49)18worker1: QEMU running (pid 52)19lb # Disk image does not exist, creating the virtualisation disk image...20worker2 # mke2fs 1.47.4 (6-Mar-2025)21client # mke2fs 1.47.4 (6-Mar-2025)22lb # Formatting '/build/vm-state-lb/tmp.bpNfyjCIlc', fmt=raw size=107374182423worker2 # Discarding device blocks: 0/262144 done24client # Discarding device blocks: 0/262144 done25lb # mke2fs 1.47.4 (6-Mar-2025)26worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes27client # Creating filesystem with 262144 4k blocks and 65536 inodes28lb # Discarding device blocks: 0/262144 done29worker2 # Filesystem UUID: 2fabf295-1902-4332-9c33-777d2ced073d30client # Filesystem UUID: 01360881-a3f5-4b9e-87ab-1036e455f3d231lb # Creating filesystem with 262144 4k blocks and 65536 inodes32worker2 # Superblock backups stored on blocks:33client # Superblock backups stored on blocks:34lb # Filesystem UUID: e3ad1f47-6dca-447e-a1a3-ac15f3b96d9835worker2 # 32768, 98304, 163840, 22937636client # 32768, 98304, 163840, 22937637lb # Superblock backups stored on blocks:38worker2 # 39client # 40lb # 32768, 98304, 163840, 22937641worker2 # Allocating group tables: 0/8 done42client # Allocating group tables: 0/8 done43lb # 44worker2 # Writing inode tables: 0/8 done45client # Writing inode tables: 0/8 done46lb # Allocating group tables: 0/8 done47worker2 # Creating journal (8192 blocks): done48client # Creating journal (8192 blocks): done49lb # Writing inode tables: 0/8 done50worker2 # Writing superblocks and filesystem accounting information: 0/8 done51client # Writing superblocks and filesystem accounting information: 0/8 done52lb # Creating journal (8192 blocks): done53worker2 # 54client # 55lb # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # Virtualisation disk image created.57client # Virtualisation disk image created.58lb # 59worker2: QEMU running (pid 51)60lb # Virtualisation disk image created.61worker1 # Disk image does not exist, creating the virtualisation disk image...62worker1 # Formatting '/build/vm-state-worker1/tmp.TmDTawSQ6c', fmt=raw size=107374182463worker1 # mke2fs 1.47.4 (6-Mar-2025)64worker1 # Discarding device blocks: 0/262144 done65worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes66(finished: start all VMs, in 0.44 seconds)67worker1 # Filesystem UUID: 1c652a42-2698-47de-a537-554776af751a68client: waiting for unit network-addresses-eth1.service69worker1 # Superblock backups stored on blocks:70client: waiting for the VM to finish booting71worker1 # 32768, 98304, 163840, 22937672worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681lb # [ 0.000000] KASLR enabled82lb # [ 0.000000] random: crng init done83lb # [ 0.000000] Machine model: linux,dummy-virt84lb # [ 0.000000] efi: UEFI not found.85lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88lb # [ 0.000000] Zone ranges:89lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90lb # [ 0.000000] DMA32 empty91lb # [ 0.000000] Normal empty92lb # [ 0.000000] Device empty93lb # [ 0.000000] Movable zone start for each node94lb # [ 0.000000] Early memory node ranges95lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098lb # [ 0.000000] psci: probing for conduit method from DT.99lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.100lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101lb # [ 0.000000] psci: Trusted OS migration not required102lb # [ 0.000000] psci: SMC Calling Convention v1.1103lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105lb # [ 0.000000] Detected PIPT I-cache on CPU0106lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107lb # [ 0.000000] CPU features: detected: GICv3 CPU interface108lb # [ 0.000000] CPU features: detected: Spectre-v4109lb # [ 0.000000] CPU features: detected: Spectre-BHB110lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112lb # [ 0.000000] alternatives: applying boot alternatives113lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/px4c3ifawqq4xs8a4zhy519i865fkk09-nixos-system-lb-test/init regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration console=ttyAMA0,115200n8 console=tty0114lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration", will be passed to user space.115lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119lb # [ 0.000000] software IO TLB: area num 1.120lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]122lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)123lb # [ 0.000000] Fallback order for Node 0: 0124client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026125lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216126client # [ 0.000000] KASLR enabled127lb # [ 0.000000] Policy zone: DMA128client # [ 0.000000] random: crng init done129lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off130client # [ 0.000000] Machine model: linux,dummy-virt131client # [ 0.000000] efi: UEFI not found.132lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1133lb # [ 0.000000] allocated 3145728 bytes of page_ext134client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT135lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages136client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]137lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups138client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]139lb # [ 0.000000] rcu: Hierarchical RCU implementation.140client # [ 0.000000] Zone ranges:141lb # [ 0.000000] rcu: RCU event tracing is enabled.142client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]143client # [ 0.000000] DMA32 empty144lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.145client # [ 0.000000] Normal empty146client # [ 0.000000] Device empty147lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.148client # [ 0.000000] Movable zone start for each node149lb # [ 0.000000] Rude variant of Tasks RCU enabled.150client # [ 0.000000] Early memory node ranges151lb # [ 0.000000] Tracing variant of Tasks RCU enabled.152client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]153lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.154client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]155lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1156client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000157client # [ 0.000000] psci: probing for conduit method from DT.158lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.159client # [ 0.000000] psci: PSCIv1.3 detected in firmware.160client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs161lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.162client # [ 0.000000] psci: Trusted OS migration not required163lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.164client # [ 0.000000] psci: SMC Calling Convention v1.1165lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0166client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)167lb # [ 0.000000] GICv3: 256 SPIs implemented168lb # [ 0.000000] GICv3: 0 Extended SPIs implemented169client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296170lb # [ 0.000000] Root IRQ handler: gic_handle_irq171client # [ 0.000000] Detected PIPT I-cache on CPU0172lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI173lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0174client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)175client # [ 0.000000] CPU features: detected: GICv3 CPU interface176lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000177client # [ 0.000000] CPU features: detected: Spectre-v4178lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]179client # [ 0.000000] CPU features: detected: Spectre-BHB180worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]181lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)182client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38183client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23184worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026185lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)186worker2 # [ 0.000000] KASLR enabled187client # [ 0.000000] alternatives: applying boot alternatives188worker2 # [ 0.000000] random: crng init done189lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000190worker2 # [ 0.000000] Machine model: linux,dummy-virt191worker2 # [ 0.000000] efi: UEFI not found.192lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000193worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT194lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.195worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]196lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns197worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]198client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test/init regInfo=/nix/store/mfi7s969scingkrqknkvk3lldv44xdhn-closure-info/registration console=ttyAMA0,115200n8 console=tty0199worker2 # [ 0.000000] Zone ranges:200lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).201worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]202worker2 # [ 0.000000] DMA32 empty203lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns204worker2 # [ 0.000000] Normal empty205client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/mfi7s969scingkrqknkvk3lldv44xdhn-closure-info/registration", will be passed to user space.206worker2 # [ 0.000000] Device empty207worker2 # [ 0.000000] Movable zone start for each node208client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes209lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns210worker2 # [ 0.000000] Early memory node ranges211lb # [ 0.000038] arm-pv: using stolen time PV212client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)213worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]214lb # [ 0.000795] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)215client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)216worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]217lb # [ 0.000959] Console: colour dummy device 80x25218worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000219client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB220lb # [ 0.000966] printk: legacy console [tty0] enabled221client # [ 0.000000] software IO TLB: area num 1.222worker2 # [ 0.000000] psci: probing for conduit method from DT.223worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.224client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB225lb # [ 0.001156] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)226worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs227lb # [ 0.001163] pid_max: default: 32768 minimum: 301228client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)229worker2 # [ 0.000000] psci: Trusted OS migration not required230client # [ 0.000000] Fallback order for Node 0: 0231lb # [ 0.001232] LSM: initializing lsm=capability,landlock,yama,bpf,ima232worker2 # [ 0.000000] psci: SMC Calling Convention v1.1233lb # [ 0.001426] landlock: Up and running.234client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216235lb # [ 0.001429] Yama: becoming mindful.236worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)237client # [ 0.000000] Policy zone: DMA238lb # [ 0.002196] LSM support for eBPF active239worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296240client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off241lb # [ 0.002417] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)242worker2 # [ 0.000000] Detected PIPT I-cache on CPU0243client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1244lb # [ 0.002450] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)245client # [ 0.000000] allocated 3145728 bytes of page_ext246worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)247lb # [ 0.003609] cacheinfo: Unable to detect cache hierarchy for CPU 0248client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages249worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface250lb # [ 0.004594] rcu: Hierarchical SRCU implementation.251client # [ 0.000000] ftrace: allocated 294 pages with 4 groups252worker2 # [ 0.000000] CPU features: detected: Spectre-v4253worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]254lb # [ 0.004598] rcu: Max phase no-delay instances is 1000.255client # [ 0.000000] rcu: Hierarchical RCU implementation.256worker2 # [ 0.000000] CPU features: detected: Spectre-BHB257lb # [ 0.005848] fsl-mc MSI: its@8080000 domain created258client # [ 0.000000] rcu: RCU event tracing is enabled.259worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38260lb # [ 0.005947] EFI services will not be available.261worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026262client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.263worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23264worker1 # [ 0.000000] KASLR enabled265lb # [ 0.006014] smp: Bringing up secondary CPUs ...266client # [ 0.000000] Trampoline variant of Tasks RCU enabled.267worker1 # [ 0.000000] random: crng init done268lb # [ 0.006022] smp: Brought up 1 node, 1 CPU269worker2 # [ 0.000000] alternatives: applying boot alternatives270worker1 # [ 0.000000] Machine model: linux,dummy-virt271client # [ 0.000000] Rude variant of Tasks RCU enabled.272lb # [ 0.006025] SMP: Total of 1 processors activated.273worker1 # [ 0.000000] efi: UEFI not found.274client # [ 0.000000] Tracing variant of Tasks RCU enabled.275lb # [ 0.006031] CPU: All CPU(s) started at EL1276worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT277client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.278lb # [ 0.006043] CPU features: detected: Branch Target Identification279worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]280lb # [ 0.006047] CPU features: detected: ARMv8.4 Translation Table Level281client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1282worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]283worker1 # [ 0.000000] Zone ranges:284lb # [ 0.006051] CPU features: detected: Instruction cache invalidation not required for I/D coherence285client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.286worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jzf4xkwa8xmlj8vk00ys4gir3nz7fy8j-nixos-system-worker2-test/init regInfo=/nix/store/8vck4wm0zxi7gd9ibf4y3jhycpzbwnl6-closure-info/registration console=ttyAMA0,115200n8 console=tty0287worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]288worker1 # [ 0.000000] DMA32 empty289lb # [ 0.006054] CPU features: detected: Data cache clean to the PoU not required for I/D coherence290client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.291worker1 # [ 0.000000] Normal empty292worker1 # [ 0.000000] Device empty293lb # [ 0.006058] CPU features: detected: Common not Private translations294worker1 # [ 0.000000] Movable zone start for each node295client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.296worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8vck4wm0zxi7gd9ibf4y3jhycpzbwnl6-closure-info/registration", will be passed to user space.297worker1 # [ 0.000000] Early memory node ranges298client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0299lb # [ 0.006061] CPU features: detected: CRC32 instructions300worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]301client # [ 0.000000] GICv3: 256 SPIs implemented302worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes303client # [ 0.000000] GICv3: 0 Extended SPIs implemented304lb # [ 0.006064] CPU features: detected: Data cache clean to Point of Deep Persistence305worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]306worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)307client # [ 0.000000] Root IRQ handler: gic_handle_irq308worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000309lb # [ 0.006068] CPU features: detected: Data cache clean to Point of Persistence310client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI311worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)312worker1 # [ 0.000000] psci: probing for conduit method from DT.313client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0314lb # [ 0.006071] CPU features: detected: Data independent timing control (DIT)315worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.316lb # [ 0.006074] CPU features: detected: E0PD317worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB318client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000319worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs320worker2 # [ 0.000000] software IO TLB: area num 1.321lb # [ 0.006077] CPU features: detected: Enhanced Counter Virtualization322client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]323worker1 # [ 0.000000] psci: Trusted OS migration not required324worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB325worker1 # [ 0.000000] psci: SMC Calling Convention v1.1326lb # [ 0.006080] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)327client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)328worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)329lb # [ 0.006084] CPU features: detected: Enhanced Virtualization Traps330worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)331worker2 # [ 0.000000] Fallback order for Node 0: 0332lb # [ 0.006087] CPU features: detected: Fine Grained Traps333client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)334worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296335worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216336worker1 # [ 0.000000] Detected PIPT I-cache on CPU0337lb # [ 0.006091] CPU features: detected: Generic authentication (architected QARMA5 algorithm)338worker2 # [ 0.000000] Policy zone: DMA339client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000340lb # [ 0.006097] CPU features: detected: RCpc load-acquire (LDAPR)341worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off342client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000343worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)344lb # [ 0.006100] CPU features: detected: LSE atomic instructions345worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1346client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.347worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface348lb # [ 0.006103] CPU features: detected: Privileged Access Never349worker2 # [ 0.000000] allocated 3145728 bytes of page_ext350worker1 # [ 0.000000] CPU features: detected: Spectre-v4351lb # [ 0.006105] CPU features: detected: PMUv3352worker1 # [ 0.000000] CPU features: detected: Spectre-BHB353worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages354client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns355lb # [ 0.006108] CPU features: detected: RAS Extension Support356worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups357worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38358client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).359lb # [ 0.006111] CPU features: detected: RASv1p1 Extension Support360worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.361worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23362lb # [ 0.006113] CPU features: detected: Random Number Generator363worker2 # [ 0.000000] rcu: RCU event tracing is enabled.364worker1 # [ 0.000000] alternatives: applying boot alternatives365client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns366lb # [ 0.006116] CPU features: detected: Speculation barrier (SB)367worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.368client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns369lb # [ 0.006119] CPU features: detected: Stage-2 Force Write-Back370worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.371client # [ 0.000032] arm-pv: using stolen time PV372worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.373lb # [ 0.006122] CPU features: detected: TLB range maintenance instructions374worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.375lb # [ 0.006127] CPU features: detected: Speculative Store Bypassing Safe (SSBS)376client # [ 0.000541] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)377worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.378lb # [ 0.006169] alternatives: applying system-wide alternatives379client # [ 0.000706] Console: colour dummy device 80x25380client # [ 0.000714] printk: legacy console [tty0] enabled381worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/143xd0r0k7bpy3hb59k6jhcv7f9kn0pn-nixos-system-worker1-test/init regInfo=/nix/store/kyyw7i260gz287j3y494hm7f7ygzcgmp-closure-info/registration console=ttyAMA0,115200n8 console=tty0382worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1383lb # [ 0.009276] CPU features: detected: BBM Level 2 without TLB conflict abort384client # [ 0.000917] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)385worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.386worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kyyw7i260gz287j3y494hm7f7ygzcgmp-closure-info/registration", will be passed to user space.387client # [ 0.000924] pid_max: default: 32768 minimum: 301388lb # [ 0.009463] Memory: 1406560K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)389worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.390client # [ 0.000994] LSM: initializing lsm=capability,landlock,yama,bpf,ima391lb # [ 0.009893] devtmpfs: initialized392worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes393client # [ 0.001213] landlock: Up and running.394client # [ 0.001216] Yama: becoming mindful.395lb # [ 0.011754] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)396worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.397worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)398client # [ 0.001833] LSM support for eBPF active399worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0400lb # [ 0.011778] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).401worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)402worker2 # [ 0.000000] GICv3: 256 SPIs implemented403client # [ 0.002011] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)404lb # [ 0.012060] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL405worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented406client # [ 0.002044] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)407worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq408lb # [ 0.012065] 0 pages in range for non-PLT usage409worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB410lb # [ 0.012066] 508288 pages in range for PLT usage411worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI412client # [ 0.003204] cacheinfo: Unable to detect cache hierarchy for CPU 0413worker1 # [ 0.000000] software IO TLB: area num 1.414worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0415lb # [ 0.012193] pinctrl core: initialized pinctrl subsystem416client # [ 0.003955] rcu: Hierarchical SRCU implementation.417worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB418lb # [ 0.012972] DMI not present or invalid.419client # [ 0.003959] rcu: Max phase no-delay instances is 1000.420worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000421lb # [ 0.016916] NET: Registered PF_NETLINK/PF_ROUTE protocol family422worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)423client # [ 0.005209] fsl-mc MSI: its@8080000 domain created424worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]425worker1 # [ 0.000000] Fallback order for Node 0: 0426client # [ 0.005300] EFI services will not be available.427lb # [ 0.019201] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations428client # [ 0.005367] smp: Bringing up secondary CPUs ...429worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)430worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216431client # [ 0.005376] smp: Brought up 1 node, 1 CPU432lb # [ 0.019395] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations433worker1 # [ 0.000000] Policy zone: DMA434client # [ 0.005379] SMP: Total of 1 processors activated.435worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)436worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off437lb # [ 0.019599] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations438client # [ 0.005384] CPU: All CPU(s) started at EL1439worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000440lb # [ 0.019621] audit: initializing netlink subsys (disabled)441worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1442client # [ 0.005395] CPU features: detected: Branch Target Identification443worker1 # [ 0.000000] allocated 3145728 bytes of page_ext444lb # [ 0.020207] thermal_sys: Registered thermal governor 'fair_share'445worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000446client # [ 0.005399] CPU features: detected: ARMv8.4 Translation Table Level447worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages448lb # [ 0.020209] thermal_sys: Registered thermal governor 'bang_bang'449worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.450worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups451client # [ 0.005403] CPU features: detected: Instruction cache invalidation not required for I/D coherence452lb # [ 0.020213] thermal_sys: Registered thermal governor 'step_wise'453worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.454worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns455lb # [ 0.020218] thermal_sys: Registered thermal governor 'user_space'456worker1 # [ 0.000000] rcu: RCU event tracing is enabled.457client # [ 0.005406] CPU features: detected: Data cache clean to the PoU not required for I/D coherence458worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).459lb # [ 0.020220] thermal_sys: Registered thermal governor 'power_allocator'460worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.461client # [ 0.005410] CPU features: detected: Common not Private translations462worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.463client # [ 0.005413] CPU features: detected: CRC32 instructions464lb # [ 0.020246] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1465worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns466worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.467lb # [ 0.020256] cpuidle: using governor ladder468client # [ 0.005416] CPU features: detected: Data cache clean to Point of Deep Persistence469worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.470lb # [ 0.020262] cpuidle: using governor menu471worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns472client # [ 0.005420] CPU features: detected: Data cache clean to Point of Persistence473worker2 # [ 0.000033] arm-pv: using stolen time PV474worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.475lb # [ 0.020468] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.476client # [ 0.005423] CPU features: detected: Data independent timing control (DIT)477lb # [ 0.020484] ASID allocator initialised with 65536 entries478worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1479client # [ 0.005426] CPU features: detected: E0PD480lb # [ 0.021762] Serial: AMBA PL011 UART driver481worker2 # [ 0.000520] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)482worker2 # [ 0.000683] Console: colour dummy device 80x25483client # [ 0.005428] CPU features: detected: Enhanced Counter Virtualization484worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.485lb # [ 0.027123] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1486worker2 # [ 0.000692] printk: legacy console [tty0] enabled487client # [ 0.005431] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)488lb # [ 0.027285] printk: console [ttyAMA0] enabled489worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490client # [ 0.005435] CPU features: detected: Enhanced Virtualization Traps491worker2 # [ 0.000882] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)492worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.493client # [ 0.005438] CPU features: detected: Fine Grained Traps494worker2 # [ 0.000889] pid_max: default: 32768 minimum: 301495worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0496worker2 # [ 0.000959] LSM: initializing lsm=capability,landlock,yama,bpf,ima497client # [ 0.005442] CPU features: detected: Generic authentication (architected QARMA5 algorithm)498worker1 # [ 0.000000] GICv3: 256 SPIs implemented499worker2 # [ 0.001111] landlock: Up and running.500client # [ 0.005447] CPU features: detected: RCpc load-acquire (LDAPR)501worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented502worker2 # [ 0.001113] Yama: becoming mindful.503worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq504worker2 # [ 0.001590] LSM support for eBPF active505client # [ 0.005450] CPU features: detected: LSE atomic instructions506worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI507client # [ 0.005453] CPU features: detected: Privileged Access Never508worker2 # [ 0.001715] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)509worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0510client # [ 0.005456] CPU features: detected: PMUv3511worker2 # [ 0.001748] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)512worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000513client # [ 0.005458] CPU features: detected: RAS Extension Support514worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]515worker2 # [ 0.002845] cacheinfo: Unable to detect cache hierarchy for CPU 0516client # [ 0.005461] CPU features: detected: RASv1p1 Extension Support517worker2 # [ 0.003635] rcu: Hierarchical SRCU implementation.518client # [ 0.005464] CPU features: detected: Random Number Generator519worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)520worker2 # [ 0.003639] rcu: Max phase no-delay instances is 1000.521client # [ 0.005466] CPU features: detected: Speculation barrier (SB)522worker2 # [ 0.004837] fsl-mc MSI: its@8080000 domain created523client # [ 0.005469] CPU features: detected: Stage-2 Force Write-Back524worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)525worker2 # [ 0.004924] EFI services will not be available.526worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000527worker2 # [ 0.005052] smp: Bringing up secondary CPUs ...528client # [ 0.005472] CPU features: detected: TLB range maintenance instructions529worker2 # [ 0.005060] smp: Brought up 1 node, 1 CPU530lb # [ 0.149032] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages531worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000532client # [ 0.005477] CPU features: detected: Speculative Store Bypassing Safe (SSBS)533worker2 # [ 0.005063] SMP: Total of 1 processors activated.534client # [ 0.005517] alternatives: applying system-wide alternatives535lb # [ 0.149053] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page536worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.537worker2 # [ 0.005069] CPU: All CPU(s) started at EL1538worker2 # [ 0.005080] CPU features: detected: Branch Target Identification539lb # [ 0.149059] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages540client # [ 0.008536] CPU features: detected: BBM Level 2 without TLB conflict abort541worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns542worker2 # [ 0.005085] CPU features: detected: ARMv8.4 Translation Table Level543lb # [ 0.149063] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page544worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).545lb # [ 0.149068] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages546client # [ 0.008769] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)547worker2 # [ 0.005088] CPU features: detected: Instruction cache invalidation not required for I/D coherence548client # [ 0.009165] devtmpfs: initialized549lb # [ 0.149072] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page550worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns551worker2 # [ 0.005092] CPU features: detected: Data cache clean to the PoU not required for I/D coherence552client # [ 0.011093] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)553lb # [ 0.149076] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages554worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns555worker2 # [ 0.005095] CPU features: detected: Common not Private translations556lb # [ 0.149080] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page557client # [ 0.011116] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).558worker1 # [ 0.000028] arm-pv: using stolen time PV559worker2 # [ 0.005098] CPU features: detected: CRC32 instructions560client # [ 0.011311] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL561worker2 # [ 0.005101] CPU features: detected: Data cache clean to Point of Deep Persistence562worker1 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)563client # [ 0.011316] 0 pages in range for non-PLT usage564worker1 # [ 0.000568] Console: colour dummy device 80x25565client # [ 0.011317] 508288 pages in range for PLT usage566worker2 # [ 0.005104] CPU features: detected: Data cache clean to Point of Persistence567lb # [ 0.156310] fbcon: Taking over console568worker1 # [ 0.000575] printk: legacy console [tty0] enabled569client # [ 0.011432] pinctrl core: initialized pinctrl subsystem570worker2 # [ 0.005107] CPU features: detected: Data independent timing control (DIT)571lb # [ 0.156327] ACPI: Interpreter disabled.572client # [ 0.012165] DMI not present or invalid.573worker2 # [ 0.005110] CPU features: detected: E0PD574worker1 # [ 0.000761] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)575lb # [ 0.158153] iommu: Default domain type: Translated576client # [ 0.015570] NET: Registered PF_NETLINK/PF_ROUTE protocol family577worker2 # [ 0.005113] CPU features: detected: Enhanced Counter Virtualization578worker1 # [ 0.000768] pid_max: default: 32768 minimum: 301579lb # [ 0.158163] iommu: DMA domain TLB invalidation policy: strict mode580client # [ 0.017826] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations581worker2 # [ 0.005116] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)582worker1 # [ 0.000850] LSM: initializing lsm=capability,landlock,yama,bpf,ima583lb # [ 0.159813] SCSI subsystem initialized584worker1 # [ 0.000974] landlock: Up and running.585client # [ 0.018008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations586worker2 # [ 0.005119] CPU features: detected: Enhanced Virtualization Traps587worker1 # [ 0.000977] Yama: becoming mindful.588worker1 # [ 0.001397] LSM support for eBPF active589worker2 # [ 0.005122] CPU features: detected: Fine Grained Traps590client # [ 0.018223] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations591client # [ 0.018245] audit: initializing netlink subsys (disabled)592worker1 # [ 0.001527] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)593worker2 # [ 0.005125] CPU features: detected: Generic authentication (architected QARMA5 algorithm)594client # [ 0.018868] thermal_sys: Registered thermal governor 'fair_share'595worker2 # [ 0.005130] CPU features: detected: RCpc load-acquire (LDAPR)596worker1 # [ 0.001564] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)597client # [ 0.018870] thermal_sys: Registered thermal governor 'bang_bang'598worker2 # [ 0.005133] CPU features: detected: LSE atomic instructions599worker1 # [ 0.002719] cacheinfo: Unable to detect cache hierarchy for CPU 0600worker2 # [ 0.005136] CPU features: detected: Privileged Access Never601worker1 # [ 0.003447] rcu: Hierarchical SRCU implementation.602worker2 # [ 0.005139] CPU features: detected: PMUv3603worker1 # [ 0.003451] rcu: Max phase no-delay instances is 1000.604client # [ 0.018875] thermal_sys: Registered thermal governor 'step_wise'605worker1 # [ 0.004621] fsl-mc MSI: its@8080000 domain created606worker2 # [ 0.005142] CPU features: detected: RAS Extension Support607client # [ 0.018880] thermal_sys: Registered thermal governor 'user_space'608worker1 # [ 0.004710] EFI services will not be available.609worker2 # [ 0.005144] CPU features: detected: RASv1p1 Extension Support610client # [ 0.018883] thermal_sys: Registered thermal governor 'power_allocator'611lb # [ 0.165339] usbcore: registered new interface driver usbfs612worker1 # [ 0.004820] smp: Bringing up secondary CPUs ...613worker2 # [ 0.005147] CPU features: detected: Random Number Generator614worker1 # [ 0.004828] smp: Brought up 1 node, 1 CPU615lb # [ 0.165373] usbcore: registered new interface driver hub616client # [ 0.018908] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1617worker2 # [ 0.005150] CPU features: detected: Speculation barrier (SB)618worker1 # [ 0.004831] SMP: Total of 1 processors activated.619lb # [ 0.165391] usbcore: registered new device driver usb620client # [ 0.018917] cpuidle: using governor ladder621worker1 # [ 0.004836] CPU: All CPU(s) started at EL1622worker2 # [ 0.005152] CPU features: detected: Stage-2 Force Write-Back623lb # [ 0.165661] pps_core: LinuxPPS API ver. 1 registered624client # [ 0.018922] cpuidle: using governor menu625worker1 # [ 0.004847] CPU features: detected: Branch Target Identification626worker2 # [ 0.005155] CPU features: detected: TLB range maintenance instructions627client # [ 0.019125] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.628lb # [ 0.165667] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>629worker1 # [ 0.004851] CPU features: detected: ARMv8.4 Translation Table Level630client # [ 0.019141] ASID allocator initialised with 65536 entries631worker2 # [ 0.005160] CPU features: detected: Speculative Store Bypassing Safe (SSBS)632lb # [ 0.165677] PTP clock support registered633lb # [ 0.165724] EDAC MC: Ver: 3.0.0634client # [ 0.020366] Serial: AMBA PL011 UART driver635worker2 # [ 0.005198] alternatives: applying system-wide alternatives636worker1 # [ 0.004854] CPU features: detected: Instruction cache invalidation not required for I/D coherence637lb # [ 0.170181] scmi_core: SCMI protocol bus registered638client # [ 0.025679] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1639worker2 # [ 0.008167] CPU features: detected: BBM Level 2 without TLB conflict abort640worker1 # [ 0.004858] CPU features: detected: Data cache clean to the PoU not required for I/D coherence641client # [ 0.025813] printk: console [ttyAMA0] enabled642lb # [ 0.171144] FPGA manager framework643worker1 # [ 0.004862] CPU features: detected: Common not Private translations644lb # [ 0.172075] vgaarb: loaded645worker1 # [ 0.004865] CPU features: detected: CRC32 instructions646worker2 # [ 0.008348] Memory: 1406540K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)647worker2 # [ 0.008723] devtmpfs: initialized648lb # [ 0.172729] clocksource: Switched to clocksource arch_sys_counter649worker1 # [ 0.004868] CPU features: detected: Data cache clean to Point of Deep Persistence650worker1 # [ 0.004871] CPU features: detected: Data cache clean to Point of Persistence651worker1 # [ 0.004874] CPU features: detected: Data independent timing control (DIT)652worker1 # [ 0.004877] CPU features: detected: E0PD653worker1 # [ 0.004880] CPU features: detected: Enhanced Counter Virtualization654lb # [ 0.176570] VFS: Disk quotas dquot_6.6.0655worker1 # [ 0.004883] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)656lb # [ 0.176608] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)657worker1 # [ 0.004886] CPU features: detected: Enhanced Virtualization Traps658worker1 # [ 0.004889] CPU features: detected: Fine Grained Traps659worker1 # [ 0.004893] CPU features: detected: Generic authentication (architected QARMA5 algorithm)660worker1 # [ 0.004898] CPU features: detected: RCpc load-acquire (LDAPR)661lb # [ 0.180421] netfs: FS-Cache loaded662worker1 # [ 0.004901] CPU features: detected: LSE atomic instructions663client # [ 0.146664] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages664lb # [ 0.180548] pnp: PnP ACPI: disabled665worker1 # [ 0.004904] CPU features: detected: Privileged Access Never666client # [ 0.146682] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page667worker2 # [ 0.010512] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)668worker1 # [ 0.004907] CPU features: detected: PMUv3669client # [ 0.146688] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages670worker1 # [ 0.004909] CPU features: detected: RAS Extension Support671worker2 # [ 0.010535] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).672client # [ 0.146692] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page673worker1 # [ 0.004912] CPU features: detected: RASv1p1 Extension Support674worker2 # [ 0.010717] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL675worker1 # [ 0.004915] CPU features: detected: Random Number Generator676client # [ 0.146697] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages677worker2 # [ 0.010722] 0 pages in range for non-PLT usage678worker1 # [ 0.004917] CPU features: detected: Speculation barrier (SB)679worker2 # [ 0.010723] 508288 pages in range for PLT usage680client # [ 0.146701] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page681lb # [ 0.184585] NET: Registered PF_INET protocol family682worker1 # [ 0.004920] CPU features: detected: Stage-2 Force Write-Back683worker2 # [ 0.010823] pinctrl core: initialized pinctrl subsystem684client # [ 0.146705] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages685worker2 # [ 0.011568] DMI not present or invalid.686worker1 # [ 0.004923] CPU features: detected: TLB range maintenance instructions687client # [ 0.146710] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page688worker2 # [ 0.014834] NET: Registered PF_NETLINK/PF_ROUTE protocol family689worker1 # [ 0.004928] CPU features: detected: Speculative Store Bypassing Safe (SSBS)690worker2 # [ 0.017039] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations691worker1 # [ 0.004963] alternatives: applying system-wide alternatives692worker2 # [ 0.017223] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations693worker1 # [ 0.007851] CPU features: detected: BBM Level 2 without TLB conflict abort694client # [ 0.154085] fbcon: Taking over console695client # [ 0.154115] ACPI: Interpreter disabled.696worker2 # [ 0.017416] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations697lb # [ 0.185461] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)698worker2 # [ 0.017437] audit: initializing netlink subsys (disabled)699client # [ 0.155975] iommu: Default domain type: Translated700worker1 # [ 0.008018] Memory: 1406552K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)701worker1 # [ 0.008413] devtmpfs: initialized702worker2 # [ 0.018034] thermal_sys: Registered thermal governor 'fair_share'703client # [ 0.155984] iommu: DMA domain TLB invalidation policy: strict mode704worker2 # [ 0.018036] thermal_sys: Registered thermal governor 'bang_bang'705worker1 # [ 0.010124] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)706client # [ 0.157618] SCSI subsystem initialized707worker2 # [ 0.018039] thermal_sys: Registered thermal governor 'step_wise'708worker1 # [ 0.010150] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).709worker2 # [ 0.018045] thermal_sys: Registered thermal governor 'user_space'710client # [ 0.158324] usbcore: registered new interface driver usbfs711worker1 # [ 0.010330] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL712client # [ 0.158362] usbcore: registered new interface driver hub713worker2 # [ 0.018047] thermal_sys: Registered thermal governor 'power_allocator'714worker1 # [ 0.010335] 0 pages in range for non-PLT usage715client # [ 0.158379] usbcore: registered new device driver usb716worker1 # [ 0.010336] 508288 pages in range for PLT usage717worker2 # [ 0.018087] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1718client # [ 0.158640] pps_core: LinuxPPS API ver. 1 registered719worker1 # [ 0.010424] pinctrl core: initialized pinctrl subsystem720worker2 # [ 0.018096] cpuidle: using governor ladder721worker1 # [ 0.011207] DMI not present or invalid.722worker2 # [ 0.018102] cpuidle: using governor menu723client # [ 0.158647] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>724worker1 # [ 0.014217] NET: Registered PF_NETLINK/PF_ROUTE protocol family725client # [ 0.158657] PTP clock support registered726worker2 # [ 0.018285] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.727client # [ 0.158706] EDAC MC: Ver: 3.0.0728worker2 # [ 0.018300] ASID allocator initialised with 65536 entries729worker2 # [ 0.019512] Serial: AMBA PL011 UART driver730worker2 # [ 0.024685] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1731worker2 # [ 0.024835] printk: console [ttyAMA0] enabled732worker2 # [ 0.145818] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages733worker2 # [ 0.145837] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page734worker2 # [ 0.145842] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages735worker2 # [ 0.145846] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page736worker2 # [ 0.145851] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages737worker2 # [ 0.145855] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page738client # [ 0.170268] scmi_core: SCMI protocol bus registered739worker1 # [ 0.016583] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations740worker2 # [ 0.145859] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages741client # [ 0.171268] FPGA manager framework742worker1 # [ 0.016774] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations743worker2 # [ 0.145864] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page744client # [ 0.172219] vgaarb: loaded745worker1 # [ 0.016973] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations746worker1 # [ 0.016995] audit: initializing netlink subsys (disabled)747client # [ 0.172856] clocksource: Switched to clocksource arch_sys_counter748worker1 # [ 0.017534] thermal_sys: Registered thermal governor 'fair_share'749client # [ 0.173457] VFS: Disk quotas dquot_6.6.0750worker2 # [ 0.153455] fbcon: Taking over console751worker2 # [ 0.153473] ACPI: Interpreter disabled.752worker1 # [ 0.017536] thermal_sys: Registered thermal governor 'bang_bang'753client # [ 0.173497] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)754worker1 # [ 0.017540] thermal_sys: Registered thermal governor 'step_wise'755worker2 # [ 0.155352] iommu: Default domain type: Translated756client # [ 0.175859] netfs: FS-Cache loaded757worker1 # [ 0.017545] thermal_sys: Registered thermal governor 'user_space'758client # [ 0.175982] pnp: PnP ACPI: disabled759worker2 # [ 0.155362] iommu: DMA domain TLB invalidation policy: strict mode760worker1 # [ 0.017548] thermal_sys: Registered thermal governor 'power_allocator'761worker2 # [ 0.157077] SCSI subsystem initialized762worker1 # [ 0.017570] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1763worker1 # [ 0.017580] cpuidle: using governor ladder764worker1 # [ 0.017586] cpuidle: using governor menu765worker1 # [ 0.017777] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.766worker1 # [ 0.017792] ASID allocator initialised with 65536 entries767worker1 # [ 0.019030] Serial: AMBA PL011 UART driver768worker1 # [ 0.024189] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1769worker1 # [ 0.024305] printk: console [ttyAMA0] enabled770worker2 # [ 0.162348] usbcore: registered new interface driver usbfs771worker1 # [ 0.144579] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages772worker2 # [ 0.162382] usbcore: registered new interface driver hub773worker2 # [ 0.162401] usbcore: registered new device driver usb774worker1 # [ 0.144598] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page775client # [ 0.181892] NET: Registered PF_INET protocol family776worker2 # [ 0.162696] pps_core: LinuxPPS API ver. 1 registered777worker1 # [ 0.144603] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages778client # [ 0.182204] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)779worker1 # [ 0.144607] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page780worker2 # [ 0.162702] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>781worker2 # [ 0.162713] PTP clock support registered782worker1 # [ 0.144611] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages783worker2 # [ 0.162760] EDAC MC: Ver: 3.0.0784worker1 # [ 0.144615] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page785lb # [ 0.214769] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)786worker2 # [ 0.167426] scmi_core: SCMI protocol bus registered787worker1 # [ 0.144620] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages788lb # [ 0.214826] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)789worker2 # [ 0.168386] FPGA manager framework790worker1 # [ 0.144624] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page791worker2 # [ 0.169314] vgaarb: loaded792lb # [ 0.214860] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)793worker2 # [ 0.169966] clocksource: Switched to clocksource arch_sys_counter794lb # [ 0.214948] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)795lb # [ 0.215091] TCP: Hash tables configured (established 16384 bind 16384)796worker1 # [ 0.152010] fbcon: Taking over console797worker1 # [ 0.152027] ACPI: Interpreter disabled.798lb # [ 0.215181] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)799worker1 # [ 0.153941] iommu: Default domain type: Translated800lb # [ 0.215218] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)801worker1 # [ 0.153952] iommu: DMA domain TLB invalidation policy: strict mode802lb # [ 0.215254] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)803worker2 # [ 0.173880] VFS: Disk quotas dquot_6.6.0804worker1 # [ 0.155654] SCSI subsystem initialized805lb # [ 0.215351] NET: Registered PF_UNIX/PF_LOCAL protocol family806worker2 # [ 0.173926] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807lb # [ 0.215374] NET: Registered PF_XDP protocol family808lb # [ 0.215392] PCI: CLS 0 bytes, default 64809lb # [ 0.215700] Trying to unpack rootfs image as initramfs...810worker2 # [ 0.177607] netfs: FS-Cache loaded811worker2 # [ 0.177738] pnp: PnP ACPI: disabled812worker1 # [ 0.160944] usbcore: registered new interface driver usbfs813worker1 # [ 0.160982] usbcore: registered new interface driver hub814worker1 # [ 0.161000] usbcore: registered new device driver usb815worker1 # [ 0.161308] pps_core: LinuxPPS API ver. 1 registered816worker2 # [ 0.181676] NET: Registered PF_INET protocol family817worker2 # [ 0.181951] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)818lb # [ 0.230499] kvm [1]: HYP mode not available819worker1 # [ 0.161314] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>820worker1 # [ 0.161335] PTP clock support registered821worker1 # [ 0.161384] EDAC MC: Ver: 3.0.0822worker1 # [ 0.166095] scmi_core: SCMI protocol bus registered823worker1 # [ 0.167044] FPGA manager framework824worker1 # [ 0.168030] vgaarb: loaded825worker1 # [ 0.168675] clocksource: Switched to clocksource arch_sys_counter826worker1 # [ 0.172548] VFS: Disk quotas dquot_6.6.0827worker1 # [ 0.172580] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)828worker1 # [ 0.176286] netfs: FS-Cache loaded829worker1 # [ 0.176416] pnp: PnP ACPI: disabled830worker1 # [ 0.180408] NET: Registered PF_INET protocol family831worker1 # [ 0.180673] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832client # [ 0.212106] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833client # [ 0.212166] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834client # [ 0.212197] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835client # [ 0.212282] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)836client # [ 0.212425] TCP: Hash tables configured (established 16384 bind 16384)837client # [ 0.212520] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)838client # [ 0.212559] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)839client # [ 0.212596] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)840client # [ 0.212686] NET: Registered PF_UNIX/PF_LOCAL protocol family841client # [ 0.212732] NET: Registered PF_XDP protocol family842client # [ 0.212751] PCI: CLS 0 bytes, default 64843client # [ 0.212955] Trying to unpack rootfs image as initramfs...844client # [ 0.231181] kvm [1]: HYP mode not available845worker2 # [ 0.212589] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846worker2 # [ 0.212649] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847worker2 # [ 0.212683] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848worker2 # [ 0.212772] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849worker2 # [ 0.212914] TCP: Hash tables configured (established 16384 bind 16384)850worker2 # [ 0.213006] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851worker2 # [ 0.213043] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852worker2 # [ 0.213079] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)853worker2 # [ 0.213170] NET: Registered PF_UNIX/PF_LOCAL protocol family854worker2 # [ 0.213195] NET: Registered PF_XDP protocol family855worker2 # [ 0.213214] PCI: CLS 0 bytes, default 64856worker2 # [ 0.213469] Trying to unpack rootfs image as initramfs...857worker1 # [ 0.210658] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)858worker1 # [ 0.210709] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)859worker1 # [ 0.210738] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)860worker2 # [ 0.231815] kvm [1]: HYP mode not available861worker1 # [ 0.210825] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862worker1 # [ 0.210967] TCP: Hash tables configured (established 16384 bind 16384)863worker1 # [ 0.211053] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864worker1 # [ 0.211089] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865worker1 # [ 0.211126] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866worker1 # [ 0.211250] NET: Registered PF_UNIX/PF_LOCAL protocol family867worker1 # [ 0.211277] NET: Registered PF_XDP protocol family868worker1 # [ 0.211298] PCI: CLS 0 bytes, default 64869worker1 # [ 0.211571] Trying to unpack rootfs image as initramfs...870worker1 # [ 0.226436] kvm [1]: HYP mode not available871lb # [ 0.342611] Initialise system trusted keyrings872lb # [ 0.343370] workingset: timestamp_bits=42 max_order=19 bucket_order=0873lb # [ 0.344680] squashfs: version 4.0 (2009/01/31) Phillip Lougher874lb # [ 0.345576] 9p: Installing v9fs 9p2000 file system support875lb # [ 0.365511] Key type asymmetric registered876lb # [ 0.365540] Asymmetric key parser 'x509' registered877lb # [ 0.365622] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878lb # [ 0.372788] io scheduler mq-deadline registered879lb # [ 0.372812] io scheduler kyber registered880client # [ 0.342673] Initialise system trusted keyrings881client # [ 0.343450] workingset: timestamp_bits=42 max_order=19 bucket_order=0882client # [ 0.344747] squashfs: version 4.0 (2009/01/31) Phillip Lougher883lb # [ 0.380887] pl061_gpio 9030000.pl061: PL061 GPIO chip registered884worker2 # [ 0.332510] Initialise system trusted keyrings885client # [ 0.345593] 9p: Installing v9fs 9p2000 file system support886worker2 # [ 0.338065] workingset: timestamp_bits=42 max_order=19 bucket_order=0887worker2 # [ 0.339500] squashfs: version 4.0 (2009/01/31) Phillip Lougher888worker2 # [ 0.340267] 9p: Installing v9fs 9p2000 file system support889lb # [ 0.388774] ledtrig-cpu: registered to indicate activity on CPUs890lb # [ 0.389267] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:891lb # [ 0.389286] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000892client # [ 0.365523] Key type asymmetric registered893client # [ 0.365550] Asymmetric key parser 'x509' registered894lb # [ 0.389298] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000895client # [ 0.365626] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)896lb # [ 0.389307] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000897lb # [ 0.389330] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits898lb # [ 0.389359] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]899lb # [ 0.389488] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00900lb # [ 0.389498] pci_bus 0000:00: root bus resource [bus 00-ff]901lb # [ 0.389505] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]902worker1 # [ 0.333215] Initialise system trusted keyrings903lb # [ 0.389510] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]904client # [ 0.372916] io scheduler mq-deadline registered905worker1 # [ 0.333959] workingset: timestamp_bits=42 max_order=19 bucket_order=0906client # [ 0.372939] io scheduler kyber registered907lb # [ 0.389515] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]908worker1 # [ 0.335263] squashfs: version 4.0 (2009/01/31) Phillip Lougher909lb # [ 0.389625] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint910worker1 # [ 0.336051] 9p: Installing v9fs 9p2000 file system support911lb # [ 0.390072] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint912lb # [ 0.390257] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]913lb # [ 0.390274] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]914lb # [ 0.390303] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]915lb # [ 0.390320] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]916lb # [ 0.390762] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint917lb # [ 0.390944] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]918lb # [ 0.390959] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]919worker2 # [ 0.360850] Key type asymmetric registered920worker2 # [ 0.360877] Asymmetric key parser 'x509' registered921lb # [ 0.390989] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]922client # [ 0.381117] pl061_gpio 9030000.pl061: PL061 GPIO chip registered923worker2 # [ 0.360947] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)924lb # [ 0.391442] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint925lb # [ 0.391623] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]926lb # [ 0.391639] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]927client # [ 0.382468] ledtrig-cpu: registered to indicate activity on CPUs928lb # [ 0.391680] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]929client # [ 0.382847] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:930lb # [ 0.392147] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint931lb # [ 0.392332] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]932client # [ 0.382864] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000933lb # [ 0.392347] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]934client # [ 0.382876] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000935lb # [ 0.392380] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]936lb # [ 0.392858] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint937client # [ 0.382885] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000938lb # [ 0.393039] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]939client # [ 0.382905] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits940worker2 # [ 0.370034] io scheduler mq-deadline registered941lb # [ 0.393055] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]942worker2 # [ 0.370058] io scheduler kyber registered943client # [ 0.382929] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]944lb # [ 0.393084] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]945client # [ 0.383002] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00946lb # [ 0.393532] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint947client # [ 0.383011] pci_bus 0000:00: root bus resource [bus 00-ff]948lb # [ 0.393713] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]949client # [ 0.383018] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]950lb # [ 0.393729] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]951client # [ 0.383023] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]952lb # [ 0.393766] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]953client # [ 0.383028] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]954worker1 # [ 0.356667] Key type asymmetric registered955lb # [ 0.394209] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker2 # [ 0.375113] pl061_gpio 9030000.pl061: PL061 GPIO chip registered957lb # [ 0.394391] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]958client # [ 0.383086] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint959lb # [ 0.394407] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]960client # [ 0.383520] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint961lb # [ 0.394436] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962client # [ 0.383707] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]963worker2 # [ 0.376389] ledtrig-cpu: registered to indicate activity on CPUs964lb # [ 0.394452] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]965client # [ 0.383724] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]966worker2 # [ 0.376781] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:967lb # [ 0.394935] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint968client # [ 0.383753] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]969lb # [ 0.395120] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]970client # [ 0.383769] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]971worker2 # [ 0.376798] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000972lb # [ 0.395150] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973client # [ 0.384220] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint974worker2 # [ 0.376810] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000975client # [ 0.384403] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]976lb # [ 0.395604] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint977worker2 # [ 0.376819] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000978client # [ 0.384419] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]979lb # [ 0.395804] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]980lb # [ 0.395833] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981client # [ 0.384453] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]982worker2 # [ 0.376850] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits983worker1 # [ 0.364726] Asymmetric key parser 'x509' registered984lb # [ 0.396224] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint985worker2 # [ 0.376872] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]986worker1 # [ 0.364815] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)987lb # [ 0.396405] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]988worker2 # [ 0.376945] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00989worker1 # [ 0.366404] io scheduler mq-deadline registered990worker1 # [ 0.366415] io scheduler kyber registered991lb # [ 0.396648] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint992worker2 # [ 0.376954] pci_bus 0000:00: root bus resource [bus 00-ff]993worker2 # [ 0.376960] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]994worker2 # [ 0.376966] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]995worker2 # [ 0.376971] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]996worker2 # [ 0.377058] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint997worker2 # [ 0.377489] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint998worker2 # [ 0.377674] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]999worker2 # [ 0.377690] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker2 # [ 0.377719] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001worker2 # [ 0.377735] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1002lb # [ 0.444928] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1003lb # [ 0.444956] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker1 # [ 0.376819] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1005lb # [ 0.444986] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006client # [ 0.408957] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1007client # [ 0.409156] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1008lb # [ 0.445503] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1009client # [ 0.409173] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1010lb # [ 0.445687] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1011client # [ 0.409202] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1012lb # [ 0.445703] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1013worker1 # [ 0.378204] ledtrig-cpu: registered to indicate activity on CPUs1014lb # [ 0.445733] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015client # [ 0.409692] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1016worker1 # [ 0.378597] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1017client # [ 0.409877] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1018lb # [ 0.446315] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1019worker1 # [ 0.378615] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001020client # [ 0.409893] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1021lb # [ 0.446326] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1022worker1 # [ 0.378626] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001023worker2 # [ 0.398287] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1024client # [ 0.409926] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025lb # [ 0.446332] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1026worker2 # [ 0.398488] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1027worker1 # [ 0.378634] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001028client # [ 0.410401] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1029lb # [ 0.446377] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1030worker2 # [ 0.398505] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1031client # [ 0.410582] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1032worker1 # [ 0.378655] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1033lb # [ 0.446428] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1034worker2 # [ 0.398535] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035client # [ 0.410598] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1036worker1 # [ 0.378678] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1037worker2 # [ 0.398993] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1038client # [ 0.410627] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1039lb # [ 0.446477] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1040worker2 # [ 0.399175] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1041worker1 # [ 0.378752] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001042lb # [ 0.446526] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1043client # [ 0.411075] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1044worker2 # [ 0.399190] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker1 # [ 0.378761] pci_bus 0000:00: root bus resource [bus 00-ff]1046client # [ 0.411256] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1047lb # [ 0.446573] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1048worker1 # [ 0.378767] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1049worker2 # [ 0.399219] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050client # [ 0.411272] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1051lb # [ 0.446620] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1052worker1 # [ 0.378771] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1053worker2 # [ 0.399668] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1054client # [ 0.411301] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055worker2 # [ 0.399848] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1056worker1 # [ 0.378776] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1057lb # [ 0.446668] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1058client # [ 0.411801] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1059worker2 # [ 0.399864] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1060lb # [ 0.446715] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1061client # [ 0.411987] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1062worker1 # [ 0.378834] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1063worker2 # [ 0.399895] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064client # [ 0.412002] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1065lb # [ 0.446762] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1066worker1 # [ 0.379271] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1067worker2 # [ 0.400351] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068client # [ 0.412031] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069worker1 # [ 0.379475] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1070lb # [ 0.446874] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1071worker2 # [ 0.400534] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1072client # [ 0.412047] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1073worker1 # [ 0.379491] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1074lb # [ 0.446921] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1075worker2 # [ 0.400550] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1076client # [ 0.412500] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1077worker1 # [ 0.379521] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078lb # [ 0.446943] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1079worker2 # [ 0.400579] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1080client # [ 0.412689] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1081worker1 # [ 0.379537] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1082lb # [ 0.446964] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1083worker2 # [ 0.401027] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1084client # [ 0.412719] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085worker1 # [ 0.380017] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086lb # [ 0.446986] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1087worker2 # [ 0.401209] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1088worker1 # [ 0.380203] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1089client # [ 0.413190] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1090lb # [ 0.447013] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1091worker2 # [ 0.401225] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1092worker1 # [ 0.380219] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1093client # [ 0.413376] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1094lb # [ 0.447035] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1095worker2 # [ 0.401254] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096worker1 # [ 0.380249] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097client # [ 0.413405] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098lb # [ 0.447058] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1099worker2 # [ 0.401707] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100lb # [ 0.447080] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1101client # [ 0.413804] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1102worker2 # [ 0.401900] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1103client # [ 0.413998] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1104lb # [ 0.447102] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1105worker2 # [ 0.401917] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker1 # [ 0.399183] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1107lb # [ 0.447124] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1108worker2 # [ 0.401946] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109client # [ 0.414265] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1110worker1 # [ 0.399397] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1111client # [ 0.414549] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1112worker2 # [ 0.401963] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1113lb # [ 0.447147] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1114worker1 # [ 0.399415] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1115client # [ 0.414566] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1116lb # [ 0.447175] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1117worker1 # [ 0.399446] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118client # [ 0.414595] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119lb # [ 0.447197] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1120worker1 # [ 0.400010] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1121lb # [ 0.447218] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1122client # [ 0.415061] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123worker1 # [ 0.400205] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1124lb # [ 0.447241] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1125client # [ 0.415243] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1126worker1 # [ 0.400226] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1127client # [ 0.415258] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1128lb # [ 0.447263] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1129worker1 # [ 0.400262] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130lb # [ 0.447286] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1131client # [ 0.415287] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132lb # [ 0.447308] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1133client # [ 0.415890] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1134lb # [ 0.447331] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1135client # [ 0.415901] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1136lb # [ 0.447353] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1137client # [ 0.415907] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1138lb # [ 0.447377] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1139lb # [ 0.447405] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1140client # [ 0.415954] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1141lb # [ 0.447414] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1142client # [ 0.416001] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1143lb # [ 0.447419] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1144worker2 # [ 0.426502] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1145client # [ 0.416046] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1146lb # [ 0.448602] pci 0000:00:0a.0: enabling device (0000 -> 0002)1147worker2 # [ 0.426698] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1148client # [ 0.416092] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1149worker2 # [ 0.426727] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150client # [ 0.416137] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1151worker2 # [ 0.427179] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1152client # [ 0.416183] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1153worker2 # [ 0.427365] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker2 # [ 0.427395] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155client # [ 0.416230] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1156worker2 # [ 0.427790] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1157client # [ 0.416277] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1158worker2 # [ 0.427969] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1159client # [ 0.416323] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1160worker2 # [ 0.428231] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1161client # [ 0.416391] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1162worker2 # [ 0.428516] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1163client # [ 0.416437] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1164worker2 # [ 0.428533] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1165client # [ 0.416458] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1166worker2 # [ 0.428562] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167worker1 # [ 0.412828] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168client # [ 0.416479] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1169worker1 # [ 0.413032] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1170worker2 # [ 0.429016] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171lb # [ 0.492869] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 43233 usecs1172client # [ 0.416502] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1173worker1 # [ 0.413050] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1174worker2 # [ 0.429196] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1175client # [ 0.416525] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1176worker1 # [ 0.413081] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1177worker2 # [ 0.429212] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1178client # [ 0.416546] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1179worker2 # [ 0.429240] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180worker1 # [ 0.413582] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181client # [ 0.416568] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1182worker1 # [ 0.413773] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1183worker2 # [ 0.429818] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1184client # [ 0.416590] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1185worker1 # [ 0.413791] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1186worker2 # [ 0.429829] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1187client # [ 0.416611] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1188worker1 # [ 0.413823] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189worker2 # [ 0.429835] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1190client # [ 0.416633] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1191worker1 # [ 0.414300] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1192worker2 # [ 0.429878] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1193client # [ 0.416656] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1194worker1 # [ 0.414492] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1195worker2 # [ 0.429939] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1196worker1 # [ 0.414509] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1197client # [ 0.416678] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1198client # [ 0.416699] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1199worker1 # [ 0.414541] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200client # [ 0.416720] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1201worker1 # [ 0.414559] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1202client # [ 0.416741] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1203worker1 # [ 0.415041] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1204client # [ 0.416762] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1205worker1 # [ 0.415237] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1206worker2 # [ 0.447769] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1207client # [ 0.416783] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1208worker1 # [ 0.415269] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209client # [ 0.416804] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1210worker2 # [ 0.447820] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1211client # [ 0.416825] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1212worker1 # [ 0.415757] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1213worker2 # [ 0.447866] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1214worker1 # [ 0.415952] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1215client # [ 0.416847] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1216worker2 # [ 0.447913] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1217worker1 # [ 0.415984] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1218worker2 # [ 0.447960] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1219worker1 # [ 0.416410] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1220worker2 # [ 0.448006] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1221worker1 # [ 0.416601] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1222worker2 # [ 0.448052] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1223worker1 # [ 0.416885] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1224worker1 # [ 0.417169] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1225worker2 # [ 0.448125] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1226worker1 # [ 0.417190] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1227worker2 # [ 0.448170] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1228worker1 # [ 0.417222] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1229worker2 # [ 0.448192] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1230worker1 # [ 0.417714] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1231worker2 # [ 0.448213] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1232worker1 # [ 0.417903] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1233worker2 # [ 0.448234] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1234worker1 # [ 0.417923] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1235worker2 # [ 0.448258] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1236worker1 # [ 0.417955] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1237worker2 # [ 0.448280] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1238worker1 # [ 0.418556] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1239worker2 # [ 0.448301] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1240lb # [ 0.513003] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1241client # [ 0.480906] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1242worker1 # [ 0.418571] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1243worker2 # [ 0.448322] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1244lb # [ 0.515125] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1245client # [ 0.480955] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1246worker1 # [ 0.418577] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1247worker2 # [ 0.448344] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1248client # [ 0.480965] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1249worker2 # [ 0.448366] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1250worker1 # [ 0.418624] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1251client # [ 0.480970] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1252worker2 # [ 0.448388] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1253client # [ 0.482161] pci 0000:00:0a.0: enabling device (0000 -> 0002)1254worker1 # [ 0.418676] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1255worker2 # [ 0.448410] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1256worker1 # [ 0.418726] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1257worker2 # [ 0.448431] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1258worker1 # [ 0.418777] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1259worker2 # [ 0.448453] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1260worker2 # [ 0.448476] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1261worker1 # [ 0.418826] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1262worker2 # [ 0.448497] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1263worker1 # [ 0.418875] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1264worker2 # [ 0.448518] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1265worker1 # [ 0.418925] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1266worker2 # [ 0.448539] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1267worker2 # [ 0.448560] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1268worker1 # [ 0.418974] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1269worker2 # [ 0.448582] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1270worker1 # [ 0.419024] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1271worker2 # [ 0.448603] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1272worker1 # [ 0.419089] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1273worker2 # [ 0.448630] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1274lb # [ 0.524998] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1275worker2 # [ 0.448639] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1276worker1 # [ 0.419138] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1277lb # [ 0.527750] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1278worker2 # [ 0.448644] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1279worker1 # [ 0.419163] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1280worker2 # [ 0.449820] pci 0000:00:0a.0: enabling device (0000 -> 0002)1281worker1 # [ 0.419187] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1282worker1 # [ 0.419212] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1283worker1 # [ 0.419239] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1284worker1 # [ 0.419264] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1285worker1 # [ 0.419289] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1286worker1 # [ 0.419313] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1287worker1 # [ 0.419357] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1288worker1 # [ 0.419384] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1289worker1 # [ 0.419409] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290worker2 # [ 0.482075] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 31504 usecs1291worker1 # [ 0.419435] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1292worker1 # [ 0.419459] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1293worker1 # [ 0.419483] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1294worker1 # [ 0.419507] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1295lb # [ 0.538603] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1296worker1 # [ 0.419531] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1297worker1 # [ 0.419556] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1298worker1 # [ 0.419580] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1299worker1 # [ 0.419605] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1300worker1 # [ 0.419630] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1301worker1 # [ 0.419655] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1302lb # [ 0.541574] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1303worker1 # [ 0.419685] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1304lb # [ 0.543705] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1305worker1 # [ 0.419696] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1306client # [ 0.511114] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1307worker1 # [ 0.419701] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1308worker1 # [ 0.481074] pci 0000:00:0a.0: enabling device (0000 -> 0002)1309client # [ 0.514154] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1310client # [ 0.516346] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1311lb # [ 0.554309] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1312client # [ 0.518537] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1313lb # [ 0.556666] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1314client # [ 0.520642] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1315worker2 # [ 0.502775] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1316worker2 # [ 0.504934] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1317lb # [ 0.560337] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1318worker2 # [ 0.515962] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1319lb # [ 0.568799] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1320client # [ 0.538340] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1321client # [ 0.540666] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1322worker1 # [ 0.502438] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1323client # [ 0.544491] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1324worker2 # [ 0.522146] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1325worker2 # [ 0.524215] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1326worker1 # [ 0.512821] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1327lb # [ 0.590001] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1328client # [ 0.553136] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1329client # [ 0.555089] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1330worker2 # [ 0.534126] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1331worker2 # [ 0.536425] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1332lb # [ 0.592774] msm_serial: driver initialized1333lb # [ 0.592923] SuperH (H)SCI(F) driver initialized1334worker1 # [ 0.520850] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335lb # [ 0.592978] STM32 USART driver initialized1336worker1 # [ 0.522978] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337worker2 # [ 0.546625] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1338worker1 # [ 0.532894] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1339worker2 # [ 0.549515] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340worker1 # [ 0.535567] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1341worker2 # [ 0.551347] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1342client # [ 0.573102] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1343worker2 # [ 0.553729] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1344client # [ 0.578883] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1345client # [ 0.581349] msm_serial: driver initialized1346client # [ 0.581493] SuperH (H)SCI(F) driver initialized1347client # [ 0.581544] STM32 USART driver initialized1348worker1 # [ 0.546942] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1349lb # [ 0.622929] loop: module loaded1350lb # [ 0.623123] virtio_blk virtio5: 1/0/0 default/read/poll queues1351worker1 # [ 0.550044] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1352lb # [ 0.623922] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1353worker1 # [ 0.552329] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1354worker2 # [ 0.574181] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1355worker2 # [ 0.576801] msm_serial: driver initialized1356worker2 # [ 0.576962] SuperH (H)SCI(F) driver initialized1357lb # [ 0.629486] megasas: 07.734.00.00-rc11358worker2 # [ 0.577015] STM32 USART driver initialized1359lb # [ 0.630199] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1360worker1 # [ 0.562321] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1361worker1 # [ 0.565779] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362lb # [ 0.641605] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001363lb # [ 0.641642] Intel/Sharp Extended Query Table at 0x00311364lb # [ 0.643246] Using buffer write method1365lb # [ 0.643324] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1366lb # [ 0.648553] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001367client # [ 0.615921] loop: module loaded1368client # [ 0.616171] virtio_blk virtio5: 1/0/0 default/read/poll queues1369lb # [ 0.648582] Intel/Sharp Extended Query Table at 0x00311370worker1 # [ 0.579059] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1371client # [ 0.618132] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1372worker1 # [ 0.581659] msm_serial: driver initialized1373worker1 # [ 0.581799] SuperH (H)SCI(F) driver initialized1374lb # [ 0.657697] Using buffer write method1375worker1 # [ 0.581850] STM32 USART driver initialized1376lb # [ 0.657733] Concatenating MTD devices:1377lb # [ 0.657737] (0): "0.flash"1378lb # [ 0.657741] (1): "0.flash"1379lb # [ 0.657745] into device "0.flash"1380worker2 # [ 0.610790] loop: module loaded1381worker2 # [ 0.610978] virtio_blk virtio5: 1/0/0 default/read/poll queues1382worker2 # [ 0.611752] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1383client # [ 0.629581] megasas: 07.734.00.00-rc11384client # [ 0.630473] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1385client # [ 0.632791] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001386client # [ 0.632822] Intel/Sharp Extended Query Table at 0x00311387client # [ 0.634538] Using buffer write method1388client # [ 0.634633] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1389client # [ 0.636341] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390client # [ 0.636374] Intel/Sharp Extended Query Table at 0x00311391worker2 # [ 0.622679] megasas: 07.734.00.00-rc11392worker2 # [ 0.623447] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1393worker2 # [ 0.625733] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001394worker2 # [ 0.625759] Intel/Sharp Extended Query Table at 0x00311395worker2 # [ 0.627350] Using buffer write method1396worker2 # [ 0.627443] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1397client # [ 0.654957] Using buffer write method1398client # [ 0.655000] Concatenating MTD devices:1399client # [ 0.655004] (0): "0.flash"1400client # [ 0.655009] (1): "0.flash"1401worker2 # [ 0.629175] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402client # [ 0.655012] into device "0.flash"1403worker2 # [ 0.629197] Intel/Sharp Extended Query Table at 0x00311404worker1 # [ 0.617320] loop: module loaded1405worker1 # [ 0.617528] virtio_blk virtio5: 1/0/0 default/read/poll queues1406worker1 # [ 0.618310] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1407worker2 # [ 0.648077] Using buffer write method1408worker2 # [ 0.648116] Concatenating MTD devices:1409worker2 # [ 0.648120] (0): "0.flash"1410worker2 # [ 0.648124] (1): "0.flash"1411worker2 # [ 0.648128] into device "0.flash"1412worker1 # [ 0.625371] megasas: 07.734.00.00-rc11413worker1 # [ 0.626135] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1414worker1 # [ 0.628209] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker1 # [ 0.628245] Intel/Sharp Extended Query Table at 0x00311416worker1 # [ 0.629978] Using buffer write method1417worker1 # [ 0.630060] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1418worker1 # [ 0.631965] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001419worker1 # [ 0.632006] Intel/Sharp Extended Query Table at 0x00311420worker1 # [ 0.652294] Using buffer write method1421worker1 # [ 0.652342] Concatenating MTD devices:1422worker1 # [ 0.652346] (0): "0.flash"1423worker1 # [ 0.652350] (1): "0.flash"1424worker1 # [ 0.652354] into device "0.flash"1425lb # [ 0.901922] Freeing initrd memory: 26140K1426lb # [ 0.907997] tun: Universal TUN/TAP device driver, 1.61427lb # [ 0.911892] thunder_xcv, ver 1.01428lb # [ 0.911940] thunder_bgx, ver 1.01429lb # [ 0.911964] nicpf, ver 1.01430lb # [ 0.912509] e1000: Intel(R) PRO/1000 Network Driver1431lb # [ 0.912517] e1000: Copyright (c) 1999-2006 Intel Corporation.1432lb # [ 0.912541] e1000e: Intel(R) PRO/1000 Network Driver1433lb # [ 0.912550] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1434lb # [ 0.912579] igb: Intel(R) Gigabit Ethernet Network Driver1435lb # [ 0.912585] igb: Copyright (c) 2007-2014 Intel Corporation.1436lb # [ 0.912613] igbvf: Intel(R) Gigabit Virtual Function Network Driver1437lb # [ 0.912619] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1438lb # [ 0.919442] sky2: driver version 1.301439lb # [ 0.921485] ehci-pci 0000:00:0a.0: EHCI Host Controller1440lb # [ 0.921541] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11441lb # [ 0.921833] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001442lb # [ 0.925114] usbcore: registered new interface driver usb-storage1443lb # [ 0.925206] usbcore: registered new interface driver usbserial_generic1444lb # [ 0.925221] usbserial: USB Serial support registered for generic1445lb # [ 0.925809] hv_vmbus: registering driver hyperv_keyboard1446client # [ 0.896393] Freeing initrd memory: 26160K1447lb # [ 0.929620] rtc-pl031 9010000.pl031: registered as rtc01448lb # [ 0.929651] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:13:13 UTC (1789981993)1449lb # [ 0.929953] i2c_dev: i2c /dev entries driver1450lb # [ 0.932984] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001451lb # [ 0.933290] hub 1-0:1.0: USB hub found1452lb # [ 0.933314] hub 1-0:1.0: 6 ports detected1453client # [ 0.902541] tun: Universal TUN/TAP device driver, 1.61454lb # [ 0.937269] sdhci: Secure Digital Host Controller Interface driver1455lb # [ 0.937279] sdhci: Copyright(c) Pierre Ossman1456lb # [ 0.937542] Synopsys Designware Multimedia Card Interface Driver1457lb # [ 0.937910] sdhci-pltfm: SDHCI platform and OF driver helper1458client # [ 0.906389] thunder_xcv, ver 1.01459lb # [ 0.939617] hid: raw HID events driver (C) Jiri Kosina1460client # [ 0.906438] thunder_bgx, ver 1.01461client # [ 0.906461] nicpf, ver 1.01462lb # [ 0.939871] usbcore: registered new interface driver usbhid1463lb # [ 0.939878] usbhid: USB HID core driver1464client # [ 0.907030] e1000: Intel(R) PRO/1000 Network Driver1465client # [ 0.907038] e1000: Copyright (c) 1999-2006 Intel Corporation.1466worker2 # [ 0.890154] Freeing initrd memory: 26148K1467client # [ 0.907062] e1000e: Intel(R) PRO/1000 Network Driver1468client # [ 0.907071] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1469client # [ 0.907101] igb: Intel(R) Gigabit Ethernet Network Driver1470lb # [ 0.945405] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1471client # [ 0.907107] igb: Copyright (c) 2007-2014 Intel Corporation.1472lb # [ 0.946935] drop_monitor: Initializing network drop monitor service1473client # [ 0.907131] igbvf: Intel(R) Gigabit Virtual Function Network Driver1474lb # [ 0.947068] NET: Registered PF_INET6 protocol family1475client # [ 0.907136] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1476client # [ 0.907273] sky2: driver version 1.301477worker2 # [ 0.896216] tun: Universal TUN/TAP device driver, 1.61478lb # [ 0.950154] Segment Routing with IPv61479lb # [ 0.950187] In-situ OAM (IOAM) with IPv61480client # [ 0.909676] ehci-pci 0000:00:0a.0: EHCI Host Controller1481lb # [ 0.950225] NET: Registered PF_PACKET protocol family1482client # [ 0.909705] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11483client # [ 0.909897] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001484lb # [ 0.951901] 9pnet: Installing 9P2000 support1485client # [ 0.919440] usbcore: registered new interface driver usb-storage1486worker2 # [ 0.899923] thunder_xcv, ver 1.01487worker2 # [ 0.899971] thunder_bgx, ver 1.01488client # [ 0.919499] usbcore: registered new interface driver usbserial_generic1489worker2 # [ 0.899995] nicpf, ver 1.01490client # [ 0.919513] usbserial: USB Serial support registered for generic1491worker2 # [ 0.900546] e1000: Intel(R) PRO/1000 Network Driver1492lb # [ 0.954733] Key type dns_resolver registered1493client # [ 0.920185] hv_vmbus: registering driver hyperv_keyboard1494worker2 # [ 0.900554] e1000: Copyright (c) 1999-2006 Intel Corporation.1495worker2 # [ 0.900578] e1000e: Intel(R) PRO/1000 Network Driver1496client # [ 0.920912] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001497client # [ 0.921241] hub 1-0:1.0: USB hub found1498worker2 # [ 0.900588] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1499client # [ 0.921261] hub 1-0:1.0: 6 ports detected1500worker2 # [ 0.900618] igb: Intel(R) Gigabit Ethernet Network Driver1501worker2 # [ 0.900624] igb: Copyright (c) 2007-2014 Intel Corporation.1502worker2 # [ 0.900648] igbvf: Intel(R) Gigabit Virtual Function Network Driver1503client # [ 0.926552] rtc-pl031 9010000.pl031: registered as rtc01504worker2 # [ 0.900654] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1505worker2 # [ 0.900791] sky2: driver version 1.301506client # [ 0.926581] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:13:13 UTC (1789981993)1507client # [ 0.926938] i2c_dev: i2c /dev entries driver1508worker2 # [ 0.909811] ehci-pci 0000:00:0a.0: EHCI Host Controller1509worker2 # [ 0.909843] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11510lb # [ 0.961231] registered taskstats version 11511worker2 # [ 0.910075] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001512lb # [ 0.961399] Loading compiled-in X.509 certificates1513client # [ 0.932001] sdhci: Secure Digital Host Controller Interface driver1514client # [ 0.932013] sdhci: Copyright(c) Pierre Ossman1515client # [ 0.932277] Synopsys Designware Multimedia Card Interface Driver1516worker2 # [ 0.912830] usbcore: registered new interface driver usb-storage1517client # [ 0.932651] sdhci-pltfm: SDHCI platform and OF driver helper1518worker2 # [ 0.912947] usbcore: registered new interface driver usbserial_generic1519worker2 # [ 0.912961] usbserial: USB Serial support registered for generic1520worker2 # [ 0.913546] hv_vmbus: registering driver hyperv_keyboard1521worker1 # [ 0.896177] Freeing initrd memory: 26148K1522lb # [ 0.970183] Demotion targets for Node 0: null1523lb # [ 0.970302] Key type .fscrypt registered1524lb # [ 0.970309] Key type fscrypt-provisioning registered1525worker2 # [ 0.917392] rtc-pl031 9010000.pl031: registered as rtc01526client # [ 0.937157] hid: raw HID events driver (C) Jiri Kosina1527lb # [ 0.970403] ima: No TPM chip found, activating TPM-bypass!1528client # [ 0.937414] usbcore: registered new interface driver usbhid1529worker2 # [ 0.917420] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:13:13 UTC (1789981993)1530lb # [ 0.970423] ima: Allocated hash algorithm: sha11531client # [ 0.937420] usbhid: USB HID core driver1532lb # [ 0.970445] ima: No architecture policies found1533worker2 # [ 0.917717] i2c_dev: i2c /dev entries driver1534worker1 # [ 0.902321] tun: Universal TUN/TAP device driver, 1.61535lb # [ 0.974472] input: gpio-keys as /devices/platform/gpio-keys/input/input01536client # [ 0.940292] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1537client # [ 0.942991] drop_monitor: Initializing network drop monitor service1538client # [ 0.943154] NET: Registered PF_INET6 protocol family1539client # [ 0.945361] Segment Routing with IPv61540client # [ 0.945381] In-situ OAM (IOAM) with IPv61541client # [ 0.945428] NET: Registered PF_PACKET protocol family1542worker2 # [ 0.922792] sdhci: Secure Digital Host Controller Interface driver1543worker2 # [ 0.922803] sdhci: Copyright(c) Pierre Ossman1544worker2 # [ 0.923071] Synopsys Designware Multimedia Card Interface Driver1545worker1 # [ 0.906332] thunder_xcv, ver 1.01546worker1 # [ 0.906377] thunder_bgx, ver 1.01547client # [ 0.947109] 9pnet: Installing 9P2000 support1548worker2 # [ 0.923431] sdhci-pltfm: SDHCI platform and OF driver helper1549worker1 # [ 0.906399] nicpf, ver 1.01550worker2 # [ 0.925147] hid: raw HID events driver (C) Jiri Kosina1551worker1 # [ 0.906942] e1000: Intel(R) PRO/1000 Network Driver1552worker2 # [ 0.925389] usbcore: registered new interface driver usbhid1553worker1 # [ 0.906950] e1000: Copyright (c) 1999-2006 Intel Corporation.1554worker2 # [ 0.925396] usbhid: USB HID core driver1555worker1 # [ 0.906975] e1000e: Intel(R) PRO/1000 Network Driver1556worker2 # [ 0.926030] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001557worker1 # [ 0.906983] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1558worker2 # [ 0.926337] hub 1-0:1.0: USB hub found1559worker1 # [ 0.907013] igb: Intel(R) Gigabit Ethernet Network Driver1560worker2 # [ 0.926357] hub 1-0:1.0: 6 ports detected1561client # [ 0.950019] Key type dns_resolver registered1562worker1 # [ 0.907019] igb: Copyright (c) 2007-2014 Intel Corporation.1563worker1 # [ 0.907042] igbvf: Intel(R) Gigabit Virtual Function Network Driver1564worker1 # [ 0.907048] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1565worker1 # [ 0.907179] sky2: driver version 1.301566worker1 # [ 0.909491] ehci-pci 0000:00:0a.0: EHCI Host Controller1567worker2 # [ 0.933603] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1568worker1 # [ 0.909525] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11569client # [ 0.956524] registered taskstats version 11570worker2 # [ 0.935131] drop_monitor: Initializing network drop monitor service1571worker1 # [ 0.909737] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001572client # [ 0.956670] Loading compiled-in X.509 certificates1573worker2 # [ 0.935329] NET: Registered PF_INET6 protocol family1574worker2 # [ 0.938327] Segment Routing with IPv61575worker2 # [ 0.938359] In-situ OAM (IOAM) with IPv61576lb # [ 0.992682] clk: Disabling unused clocks1577worker2 # [ 0.938400] NET: Registered PF_PACKET protocol family1578worker1 # [ 0.919202] usbcore: registered new interface driver usb-storage1579lb # [ 0.992711] PM: genpd: Disabling unused power domains1580worker1 # [ 0.919371] usbcore: registered new interface driver usbserial_generic1581worker1 # [ 0.919393] usbserial: USB Serial support registered for generic1582worker1 # [ 0.920002] hv_vmbus: registering driver hyperv_keyboard1583worker2 # [ 0.940049] 9pnet: Installing 9P2000 support1584worker1 # [ 0.923645] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001585worker1 # [ 0.923988] hub 1-0:1.0: USB hub found1586lb # [ 0.997011] Freeing unused kernel memory: 4736K1587worker1 # [ 0.924018] hub 1-0:1.0: 6 ports detected1588lb # [ 0.997213] Run /init as init process1589client # [ 0.965526] Demotion targets for Node 0: null1590worker2 # [ 0.942782] Key type dns_resolver registered1591worker1 # [ 0.926354] rtc-pl031 9010000.pl031: registered as rtc01592client # [ 0.965645] Key type .fscrypt registered1593client # [ 0.965652] Key type fscrypt-provisioning registered1594worker1 # [ 0.926387] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:13:13 UTC (1789981993)1595client # [ 0.965746] ima: No TPM chip found, activating TPM-bypass!1596worker1 # [ 0.926720] i2c_dev: i2c /dev entries driver1597client # [ 0.965765] ima: Allocated hash algorithm: sha11598client # [ 0.965786] ima: No architecture policies found1599worker2 # [ 0.949064] registered taskstats version 11600client # [ 0.969948] input: gpio-keys as /devices/platform/gpio-keys/input/input01601worker2 # [ 0.949226] Loading compiled-in X.509 certificates1602worker1 # [ 0.931654] sdhci: Secure Digital Host Controller Interface driver1603worker1 # [ 0.931664] sdhci: Copyright(c) Pierre Ossman1604worker1 # [ 0.931926] Synopsys Designware Multimedia Card Interface Driver1605worker1 # [ 0.932289] sdhci-pltfm: SDHCI platform and OF driver helper1606worker1 # [ 0.936674] hid: raw HID events driver (C) Jiri Kosina1607worker1 # [ 0.937581] usbcore: registered new interface driver usbhid1608worker1 # [ 0.937592] usbhid: USB HID core driver1609worker2 # [ 0.958177] Demotion targets for Node 0: null1610worker2 # [ 0.958293] Key type .fscrypt registered1611worker2 # [ 0.958300] Key type fscrypt-provisioning registered1612worker1 # [ 0.939862] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1613worker2 # [ 0.958396] ima: No TPM chip found, activating TPM-bypass!1614worker2 # [ 0.958415] ima: Allocated hash algorithm: sha11615worker2 # [ 0.958441] ima: No architecture policies found1616worker1 # [ 0.942496] drop_monitor: Initializing network drop monitor service1617lb # [ 1.015026] systemd[1]: Successfully made /usr/ read-only.1618worker1 # [ 0.942637] NET: Registered PF_INET6 protocol family1619worker2 # [ 0.962488] input: gpio-keys as /devices/platform/gpio-keys/input/input01620worker1 # [ 0.944520] Segment Routing with IPv61621worker1 # [ 0.944542] In-situ OAM (IOAM) with IPv61622worker1 # [ 0.944572] NET: Registered PF_PACKET protocol family1623worker1 # [ 0.946436] 9pnet: Installing 9P2000 support1624client # [ 0.988164] clk: Disabling unused clocks1625client # [ 0.988196] PM: genpd: Disabling unused power domains1626worker1 # [ 0.949238] Key type dns_resolver registered1627client # [ 0.992434] Freeing unused kernel memory: 4736K1628client # [ 0.992622] Run /init as init process1629worker1 # [ 0.955877] registered taskstats version 11630worker1 # [ 0.956051] Loading compiled-in X.509 certificates1631worker2 # [ 0.980273] clk: Disabling unused clocks1632worker2 # [ 0.980301] PM: genpd: Disabling unused power domains1633worker1 # [ 0.964877] Demotion targets for Node 0: null1634worker1 # [ 0.965001] Key type .fscrypt registered1635worker1 # [ 0.965008] Key type fscrypt-provisioning registered1636worker2 # [ 0.984556] Freeing unused kernel memory: 4736K1637worker2 # [ 0.984752] Run /init as init process1638worker1 # [ 0.965114] ima: No TPM chip found, activating TPM-bypass!1639worker1 # [ 0.965134] ima: Allocated hash algorithm: sha11640worker1 # [ 0.965157] ima: No architecture policies found1641client # [ 1.007187] systemd[1]: Successfully made /usr/ read-only.1642worker1 # [ 0.969457] input: gpio-keys as /devices/platform/gpio-keys/input/input01643worker2 # [ 1.000285] systemd[1]: Successfully made /usr/ read-only.1644worker1 # [ 0.988058] clk: Disabling unused clocks1645worker1 # [ 0.988095] PM: genpd: Disabling unused power domains1646worker1 # [ 0.992354] Freeing unused kernel memory: 4736K1647worker1 # [ 0.992570] Run /init as init process1648worker1 # [ 1.006991] systemd[1]: Successfully made /usr/ read-only.1649lb # [ 1.180822] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.168935] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker2 # [ 1.182017] usb 1-1: new high-speed USB device number 2 using ehci-pci1652worker1 # [ 1.168735] usb 1-1: new high-speed USB device number 2 using ehci-pci1653lb # [ 1.334803] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654lb # [ 1.349806] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1655lb # [ 1.356908] systemd[1]: Detected virtualization qemu.1656lb # [ 1.358044] systemd[1]: Detected architecture arm64.1657lb # [ 1.359126] systemd[1]: Running in initrd.1658lb # [ 1.360637] systemd[1]: Initializing machine ID from random generator.1659client # [ 1.323212] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660lb # [ 1.362342] systemd[1]: Hostname set to <lb>.1661client # [ 1.342113] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662client # [ 1.354297] systemd[1]: Detected virtualization qemu.1663worker2 # [ 1.331520] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11664client # [ 1.356402] systemd[1]: Detected architecture arm64.1665client # [ 1.358437] systemd[1]: Running in initrd.1666client # [ 1.361137] systemd[1]: Initializing machine ID from random generator.1667worker1 # [ 1.321170] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11668client # [ 1.364213] systemd[1]: Hostname set to <client>.1669worker2 # [ 1.337432] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670worker2 # [ 1.350041] systemd[1]: Detected virtualization qemu.1671worker2 # [ 1.352106] systemd[1]: Detected architecture arm64.1672worker2 # [ 1.352146] systemd[1]: Running in initrd.1673worker2 # [ 1.353135] systemd[1]: Initializing machine ID from random generator.1674worker2 # [ 1.353425] systemd[1]: Hostname set to <worker2>.1675lb # [ 1.416947] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01676worker1 # [ 1.341919] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677worker1 # [ 1.354005] systemd[1]: Detected virtualization qemu.1678worker1 # [ 1.356008] systemd[1]: Detected architecture arm64.1679worker1 # [ 1.357977] systemd[1]: Running in initrd.1680worker1 # [ 1.360569] systemd[1]: Initializing machine ID from random generator.1681worker1 # [ 1.363565] systemd[1]: Hostname set to <worker1>.1682client # [ 1.409172] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker2 # [ 1.414264] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684worker1 # [ 1.409003] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01685lb # [ 1.536789] usb 1-2: new high-speed USB device number 3 using ehci-pci1686client # [ 1.532937] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.534022] usb 1-2: new high-speed USB device number 3 using ehci-pci1688worker1 # [ 1.532733] usb 1-2: new high-speed USB device number 3 using ehci-pci1689lb # [ 1.628986] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690lb # [ 1.695658] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691lb # [ 1.698336] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01692lb # [ 1.715953] systemd[1]: Queued start job for default target Initrd Default Target.1693worker2 # [ 1.662432] systemd[1]: bpf-restrict-fs: LSM BPF program attached1694client # [ 1.683285] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695lb # [ 1.727438] systemd[1]: Created slice Slice /system/modprobe.1696lb # [ 1.728840] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1697lb # [ 1.730212] systemd[1]: Expecting device /dev/disk/by-label/nixos...1698lb # [ 1.731386] systemd[1]: Reached target Path Units.1699lb # [ 1.732238] systemd[1]: Reached target Slice Units.1700lb # [ 1.733165] systemd[1]: Reached target Swaps.1701lb # [ 1.733939] systemd[1]: Reached target Timer Units.1702lb # [ 1.735026] systemd[1]: Listening on D-Bus System Message Bus Socket.1703lb # [ 1.736336] systemd[1]: Listening on Journal Socket (/dev/log).1704client # [ 1.698277] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21705lb # [ 1.737603] systemd[1]: Listening on Journal Sockets.1706lb # [ 1.737751] systemd[1]: Listening on udev Control Socket.1707lb # [ 1.737872] systemd[1]: Listening on udev Kernel Socket.1708client # [ 1.698614] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01709lb # [ 1.737897] systemd[1]: Reached target Socket Units.1710lb # [ 1.743168] systemd[1]: Starting Create List of Static Device Nodes...1711worker2 # [ 1.693402] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21712worker1 # [ 1.678882] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713lb # [ 1.753011] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1714lb # [ 1.753110] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1715worker2 # [ 1.702459] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01716lb # [ 1.761043] systemd[1]: Mounting Kernel Configuration File System...1717worker1 # [ 1.696751] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker1 # [ 1.701918] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01719lb # [ 1.792929] systemd[1]: Starting Journal Service...1720lb # [ 1.801025] systemd[1]: Starting Load Kernel Modules...1721lb # [ 1.801948] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1722lb # [ 1.813984] systemd[1]: Starting Coldplug All udev Devices...1723client # [ 1.794678] systemd[1]: Queued start job for default target Initrd Default Target.1724worker2 # [ 1.777084] systemd[1]: Queued start job for default target Initrd Default Target.1725client # [ 1.803269] systemd[1]: Created slice Slice /system/modprobe.1726worker2 # [ 1.784198] systemd[1]: Created slice Slice /system/modprobe.1727client # [ 1.804454] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1728worker2 # [ 1.785332] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1729client # [ 1.805830] systemd[1]: Expecting device /dev/disk/by-label/nixos...1730lb # [ 1.836929] systemd[1]: Finished Create List of Static Device Nodes.1731client # [ 1.806864] systemd[1]: Reached target Path Units.1732worker2 # [ 1.786622] systemd[1]: Expecting device /dev/disk/by-label/nixos...1733lb # [ 1.838133] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1734client # [ 1.807644] systemd[1]: Reached target Slice Units.1735worker2 # [ 1.787581] systemd[1]: Reached target Path Units.1736lb # [ 1.838446] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1737client # [ 1.808440] systemd[1]: Reached target Swaps.1738worker2 # [ 1.788318] systemd[1]: Reached target Slice Units.1739lb # [ 1.838681] systemd[1]: Mounted Kernel Configuration File System.1740worker2 # [ 1.789058] systemd[1]: Reached target Swaps.1741client # [ 1.809196] systemd[1]: Reached target Timer Units.1742worker2 # [ 1.789738] systemd[1]: Reached target Timer Units.1743client # [ 1.810195] systemd[1]: Listening on D-Bus System Message Bus Socket.1744worker2 # [ 1.790735] systemd[1]: Listening on D-Bus System Message Bus Socket.1745client # [ 1.811374] systemd[1]: Listening on Journal Socket (/dev/log).1746worker2 # [ 1.791852] systemd[1]: Listening on Journal Socket (/dev/log).1747client # [ 1.812482] systemd[1]: Listening on Journal Sockets.1748worker2 # [ 1.792886] systemd[1]: Listening on Journal Sockets.1749worker2 # [ 1.793864] systemd[1]: Listening on udev Control Socket.1750client # [ 1.813576] systemd[1]: Listening on udev Control Socket.1751client # [ 1.813691] systemd[1]: Listening on udev Kernel Socket.1752worker2 # [ 1.794896] systemd[1]: Listening on udev Kernel Socket.1753client # [ 1.813714] systemd[1]: Reached target Socket Units.1754worker2 # [ 1.794926] systemd[1]: Reached target Socket Units.1755client # [ 1.817914] systemd[1]: Starting Create List of Static Device Nodes...1756worker2 # [ 1.798038] systemd[1]: Starting Create List of Static Device Nodes...1757lb # [ 1.849293] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1758worker1 # [ 1.785465] systemd[1]: Queued start job for default target Initrd Default Target.1759worker2 # [ 1.806156] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1760worker2 # [ 1.807297] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1761client # [ 1.828385] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1762lb # [ 1.862262] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1763client # [ 1.829708] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1764worker1 # [ 1.794993] systemd[1]: Created slice Slice /system/modprobe.1765worker2 # [ 1.814304] systemd[1]: Mounting Kernel Configuration File System...1766worker1 # [ 1.796152] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1767worker1 # [ 1.797456] systemd[1]: Expecting device /dev/disk/by-label/nixos...1768worker1 # [ 1.798436] systemd[1]: Reached target Path Units.1769worker1 # [ 1.799195] systemd[1]: Reached target Slice Units.1770worker1 # [ 1.799969] systemd[1]: Reached target Swaps.1771worker1 # [ 1.800771] systemd[1]: Reached target Timer Units.1772worker1 # [ 1.801709] systemd[1]: Listening on D-Bus System Message Bus Socket.1773worker1 # [ 1.802850] systemd[1]: Listening on Journal Socket (/dev/log).1774lb # [ 1.875501] systemd-journald[73]: Collecting audit messages is disabled.1775worker1 # [ 1.803937] systemd[1]: Listening on Journal Sockets.1776worker1 # [ 1.804929] systemd[1]: Listening on udev Control Socket.1777client # [ 1.842579] systemd[1]: Mounting Kernel Configuration File System...1778worker1 # [ 1.805045] systemd[1]: Listening on udev Kernel Socket.1779worker1 # [ 1.805066] systemd[1]: Reached target Socket Units.1780lb # [ 1.880884] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1781worker1 # [ 1.809065] systemd[1]: Starting Create List of Static Device Nodes...1782worker1 # [ 1.816889] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1783worker1 # [ 1.817952] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784lb # [ 1.892648] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01785worker1 # [ 1.824197] systemd[1]: Mounting Kernel Configuration File System...1786lb # [ 1.896759] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1787worker2 # [ 1.844620] systemd[1]: Starting Journal Service...1788client # [ 1.865228] systemd[1]: Starting Journal Service...1789lb # [ 1.901134] systemd[1]: Starting Create Static Device Nodes in /dev...1790client # [ 1.867716] systemd[1]: Starting Load Kernel Modules...1791client # [ 1.868666] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1792worker2 # [ 1.851263] systemd[1]: Starting Load Kernel Modules...1793worker2 # [ 1.852236] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1794lb # [ 1.908945] [drm] features: -virgl +edid -resource_blob -host_visible1795lb # [ 1.908965] [drm] features: -context_init1796lb # [ 1.909716] [drm] number of scanouts: 11797lb # [ 1.909734] [drm] number of cap sets: 01798worker2 # [ 1.862540] systemd[1]: Starting Coldplug All udev Devices...1799worker1 # [ 1.843839] systemd[1]: Starting Journal Service...1800lb # [ 1.917073] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1801client # [ 1.889297] systemd[1]: Starting Coldplug All udev Devices...1802lb # [ 1.917092] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01803worker2 # [ 1.875745] systemd[1]: Finished Create List of Static Device Nodes.1804worker2 # [ 1.876650] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1805worker2 # [ 1.876971] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1806worker2 # [ 1.877227] systemd[1]: Mounted Kernel Configuration File System.1807worker1 # [ 1.861137] systemd[1]: Starting Load Kernel Modules...1808worker1 # [ 1.861268] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1809client # [ 1.901014] systemd[1]: Finished Create List of Static Device Nodes.1810client # [ 1.901999] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1811client # [ 1.902352] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1812worker1 # [ 1.872854] systemd[1]: Starting Coldplug All udev Devices...1813lb # [ 1.937122] Console: switching to colour frame buffer device 160x501814worker2 # [ 1.895128] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815worker1 # [ 1.887462] systemd-journald[73]: Collecting audit messages is disabled.1816client # [ 1.926702] systemd-journald[73]: Collecting audit messages is disabled.1817lb # [ 1.961521] systemd[1]: Finished Create Static Device Nodes in /dev.1818lb # [ 1.961731] systemd[1]: Reached target Preparation for Local File Systems.1819lb # [ 1.961757] systemd[1]: Reached target Local File Systems.1820client # [ 1.932994] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821worker1 # [ 1.892848] systemd[1]: Finished Create List of Static Device Nodes.1822worker1 # [ 1.893779] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1823worker1 # [ 1.894092] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1824lb # [ 1.973288] systemd[1]: Starting Rule-based Manager for Device Events and Files...1825lb # [ 1.978414] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1826client # [ 1.949079] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827client # [ 1.949768] systemd[1]: Mounted Kernel Configuration File System.1828client # [ 1.951712] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829worker2 # [ 1.929968] systemd-journald[73]: Collecting audit messages is disabled.1830worker1 # [ 1.916968] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1831worker2 # [ 1.935671] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1832lb # [ 1.991291] systemd[1]: Finished Load Kernel Modules.1833client # [ 1.957872] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01834client # [ 1.958133] [drm] features: -virgl +edid -resource_blob -host_visible1835client # [ 1.958143] [drm] features: -context_init1836client # [ 1.958865] [drm] number of scanouts: 11837client # [ 1.958884] [drm] number of cap sets: 01838lb # [ 2.001236] systemd[1]: Starting Apply Kernel Variables...1839worker2 # [ 1.950780] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840worker2 # [ 1.953575] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1841worker2 # [ 1.956300] systemd[1]: Starting Create Static Device Nodes in /dev...1842worker1 # [ 1.937742] systemd[1]: Mounted Kernel Configuration File System.1843worker1 # [ 1.939261] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844worker1 # [ 1.941741] systemd[1]: Starting Create Static Device Nodes in /dev...1845client # [ 1.977261] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1846client # [ 1.977286] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01847worker2 # [ 1.962136] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01848worker2 # [ 1.962414] [drm] features: -virgl +edid -resource_blob -host_visible1849worker2 # [ 1.962424] [drm] features: -context_init1850worker2 # [ 1.963138] [drm] number of scanouts: 11851worker2 # [ 1.963155] [drm] number of cap sets: 01852client # [ 1.985593] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853client # [ 1.988330] systemd[1]: Starting Create Static Device Nodes in /dev...1854worker1 # [ 1.965537] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1855worker2 # [ 1.990359] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1856worker2 # [ 1.990387] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01857worker1 # [ 1.980742] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1858client # [ 2.013286] Console: switching to colour frame buffer device 160x501859lb # [ 2.065143] systemd[1]: Finished Apply Kernel Variables.1860worker1 # [ 1.992657] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01861worker2 # [ 2.012370] Console: switching to colour frame buffer device 160x501862worker1 # [ 2.001394] systemd[1]: Finished Create Static Device Nodes in /dev.1863worker1 # [ 2.002486] systemd[1]: Reached target Preparation for Local File Systems.1864worker1 # [ 2.003385] systemd[1]: Reached target Local File Systems.1865lb # [ 2.073882] systemd[1]: Started Journal Service.1866client # [ 2.041520] systemd[1]: Finished Create Static Device Nodes in /dev.1867lb # [ 2.060457] systemd-modules-load[74]: Inserted module 'dm_mod'1868client # [ 2.041907] systemd[1]: Reached target Preparation for Local File Systems.1869worker2 # [ 2.022687] systemd[1]: Finished Create Static Device Nodes in /dev.1870client # [ 2.041932] systemd[1]: Reached target Local File Systems.1871lb # [ 2.061705] systemd-modules-load[74]: Module 'virtio_balloon' is built in1872worker2 # [ 2.022902] systemd[1]: Reached target Preparation for Local File Systems.1873worker2 # [ 2.022926] systemd[1]: Reached target Local File Systems.1874client # [ 2.045965] systemd[1]: Starting Rule-based Manager for Device Events and Files...1875worker1 # [ 2.009044] [drm] features: -virgl +edid -resource_blob -host_visible1876worker1 # [ 2.009064] [drm] features: -context_init1877worker1 # [ 2.009820] [drm] number of scanouts: 11878worker1 # [ 2.009840] [drm] number of cap sets: 01879client # [ 2.049575] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1880worker2 # [ 2.030354] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881lb # [ 2.072338] systemd-modules-load[74]: Module 'virtio_console' is built in1882worker2 # [ 2.035888] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1883lb # [ 2.073510] systemd-modules-load[74]: Inserted module 'virtio_gpu'1884lb # [ 2.075488] systemd-modules-load[74]: Module 'virtio_rng' is built in1885worker1 # [ 2.017161] systemd[1]: Starting Rule-based Manager for Device Events and Files...1886lb # [ 2.080689] systemd-udevd[80]: Using default interface naming scheme 'v261'.1887lb # [ 2.083033] systemd[1]: Starting Create System Files and Directories...1888worker1 # [ 2.022003] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1889worker1 # [ 2.022035] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01890client # [ 2.073066] systemd[1]: Finished Load Kernel Modules.1891client # [ 2.076793] systemd[1]: Starting Apply Kernel Variables...1892worker2 # [ 2.058281] systemd[1]: Finished Load Kernel Modules.1893worker2 # [ 2.066376] systemd[1]: Starting Apply Kernel Variables...1894worker1 # [ 2.045440] Console: switching to colour frame buffer device 160x501895lb # [ 2.123363] systemd[1]: Finished Create System Files and Directories.1896lb # [ 2.126620] systemd[1]: Started Rule-based Manager for Device Events and Files.1897worker1 # [ 2.076902] systemd[1]: Started Journal Service.1898client # [ 2.117071] systemd[1]: Started Journal Service.1899worker1 # [ 2.079431] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1900worker1 # [ 2.066183] systemd-modules-load[74]: Inserted module 'dm_mod'1901worker1 # [ 2.072458] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902worker1 # [ 2.073940] systemd-modules-load[74]: Module 'virtio_console' is built in1903worker1 # [ 2.075060] systemd[1]: Starting Create System Files and Directories...1904worker2 # [ 2.114035] systemd[1]: Started Journal Service.1905client # [ 2.124097] systemd-modules-load[74]: Inserted module 'dm_mod'1906client # [ 2.125303] systemd-modules-load[74]: Module 'virtio_balloon' is built in1907client # [ 2.126438] systemd-modules-load[74]: Module 'virtio_console' is built in1908client # [ 2.127501] systemd-modules-load[74]: Inserted module 'virtio_gpu'1909worker1 # [ 2.088445] systemd-modules-load[74]: Inserted module 'virtio_gpu'1910worker1 # [ 2.091074] systemd-modules-load[74]: Module 'virtio_rng' is built in1911worker1 # [ 2.096359] systemd[1]: Finished Load Kernel Modules.1912client # [ 2.138778] systemd-modules-load[74]: Module 'virtio_rng' is built in1913client # [ 2.139897] systemd[1]: Starting Create System Files and Directories...1914worker2 # [ 2.118799] systemd-modules-load[74]: Inserted module 'dm_mod'1915worker2 # [ 2.124210] systemd-modules-load[74]: Module 'virtio_balloon' is built in1916client # [ 2.146699] systemd[1]: Finished Apply Kernel Variables.1917worker2 # [ 2.128290] systemd-modules-load[74]: Module 'virtio_console' is built in1918worker1 # [ 2.109657] systemd-udevd[79]: Using default interface naming scheme 'v261'.1919worker2 # [ 2.132331] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920lb # [ 2.184287] systemd[1]: Starting Virtual Console Setup...1921worker2 # [ 2.134821] systemd-modules-load[74]: Module 'virtio_rng' is built in1922client # [ 2.160588] systemd-udevd[79]: Using default interface naming scheme 'v261'.1923worker1 # [ 2.123790] systemd[1]: Starting Apply Kernel Variables...1924worker2 # [ 2.148318] systemd[1]: Starting Create System Files and Directories...1925worker2 # [ 2.151068] systemd-udevd[79]: Using default interface naming scheme 'v261'.1926client # [ 2.173835] systemd[1]: Finished Create System Files and Directories.1927worker2 # [ 2.155547] systemd[1]: Finished Apply Kernel Variables.1928worker1 # [ 2.143244] systemd[1]: Finished Create System Files and Directories.1929worker2 # [ 2.167245] systemd[1]: Finished Create System Files and Directories.1930client # [ 2.197172] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker1 # [ 2.157578] systemd[1]: Finished Apply Kernel Variables.1932worker2 # [ 2.181492] systemd[1]: Started Rule-based Manager for Device Events and Files.1933lb # [ 2.244569] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1934worker1 # [ 2.173528] systemd[1]: Started Rule-based Manager for Device Events and Files.1935lb # [ 2.248205] systemd[1]: Finished Virtual Console Setup.1936client # [ 2.254129] systemd[1]: Starting Virtual Console Setup...1937worker2 # [ 2.242478] systemd[1]: Starting Virtual Console Setup...1938worker1 # [ 2.230471] systemd[1]: Starting Virtual Console Setup...1939client # [ 2.312537] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940worker2 # [ 2.292491] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1941client # [ 2.316176] systemd[1]: Finished Virtual Console Setup.1942worker2 # [ 2.295960] systemd[1]: Finished Virtual Console Setup.1943worker1 # [ 2.285332] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1944worker1 # [ 2.288127] systemd[1]: Finished Virtual Console Setup.1945lb # [ 2.841310] systemd[1]: Finished Coldplug All udev Devices.1946lb # [ 2.842266] systemd[1]: Reached target System Initialization.1947lb # [ 2.843093] systemd[1]: Reached target Basic System.1948worker2 # [ 2.893728] systemd[1]: Finished Coldplug All udev Devices.1949worker2 # [ 2.894681] systemd[1]: Reached target System Initialization.1950worker2 # [ 2.895488] systemd[1]: Reached target Basic System.1951worker1 # [ 2.894228] systemd[1]: Finished Coldplug All udev Devices.1952client # [ 2.937752] systemd[1]: Finished Coldplug All udev Devices.1953worker1 # [ 2.895418] systemd[1]: Reached target System Initialization.1954worker1 # [ 2.896381] systemd[1]: Reached target Basic System.1955client # [ 2.940161] systemd[1]: Reached target System Initialization.1956client # [ 2.944161] systemd[1]: Reached target Basic System.1957lb # [ 2.995776] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958lb # [ 3.024595] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 3.031612] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1960worker2 # [ 3.031291] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1961lb # [ 3.089234] systemd[1]: Found device /dev/disk/by-label/nixos.1962lb # [ 3.095167] systemd[1]: Reached target Initrd Root Device.1963lb # [ 3.099881] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1964client # [ 3.097659] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965worker2 # [ 3.077055] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966worker1 # [ 3.059633] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1967worker2 # [ 3.083596] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1968worker1 # [ 3.068744] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 3.115465] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 3.076570] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 3.122742] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 3.153284] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1973lb # [ 3.159802] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1974lb # [ 3.171403] systemd[1]: Mounting /sysroot...1975worker2 # [ 3.133620] systemd[1]: Found device /dev/disk/by-label/nixos.1976worker2 # [ 3.136085] systemd[1]: Reached target Initrd Root Device.1977worker2 # [ 3.140104] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1978lb # [ 3.220734] EXT4-fs (vda): mounted filesystem e3ad1f47-6dca-447e-a1a3-ac15f3b96d98 r/w with ordered data mode. Quota mode: none.1979lb # [ 3.210063] systemd[1]: Mounted /sysroot.1980client # [ 3.181161] systemd[1]: Found device /dev/disk/by-label/nixos.1981lb # [ 3.213257] systemd[1]: Reached target Initrd Root File System.1982client # [ 3.184535] systemd[1]: Reached target Initrd Root Device.1983lb # [ 3.217152] systemd[1]: Starting Mountpoints Configured in the Real Root...1984client # [ 3.188460] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985worker1 # [ 3.164773] systemd[1]: Found device /dev/disk/by-label/nixos.1986worker1 # [ 3.167540] systemd[1]: Reached target Initrd Root Device.1987lb # [ 3.242543] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1988worker1 # [ 3.171963] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.194631] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1990lb # [ 3.248451] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1991worker2 # [ 3.200375] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992lb # [ 3.253935] systemd[1]: Reloading...1993worker2 # [ 3.208704] systemd[1]: Mounting /sysroot...1994client # [ 3.242039] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1995client # [ 3.252144] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996client # [ 3.254424] systemd[1]: Mounting /sysroot...1997worker1 # [ 3.217930] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1998worker2 # [ 3.253886] EXT4-fs (vda): mounted filesystem 2fabf295-1902-4332-9c33-777d2ced073d r/w with ordered data mode. Quota mode: none.1999worker2 # [ 3.245342] systemd[1]: Mounted /sysroot.2000worker1 # [ 3.225620] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.
loading 1,631 hidden lines…
3632worker2 # [ 20.696098] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker1 # [ 20.732889] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513634worker1 # [ 20.738299] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker2 # [ 20.897937] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513636worker2 # [ 20.903490] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker1 # [ 20.939804] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 20.945471] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 21.105273] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513640worker2 # [ 21.110681] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker1 # [ 21.149990] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 21.154492] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 21.314468] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513644worker2 # [ 21.318822] nix-grpc-daemon[777]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker1 # [ 21.349831] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 21.355336] nix-grpc-daemon[776]: ts=2026-09-21T09:13:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 21.515201] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513648worker2 # [ 21.521159] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker1 # [ 21.556130] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 21.562014] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker2 # [ 21.722488] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513652worker2 # [ 21.728097] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker1 # [ 21.763357] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 21.769107] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 21.929962] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513656worker2 # [ 21.935412] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker1 # [ 21.970148] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 21.975902] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 22.137236] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513660worker2 # [ 22.142894] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker1 # [ 22.180319] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 22.184481] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 22.347106] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513664worker2 # [ 22.351478] nix-grpc-daemon[777]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker1 # [ 22.380829] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_connected addr=lb:500513666worker1 # [ 22.386415] nix-grpc-daemon[776]: ts=2026-09-21T09:13:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker2 # [ 22.549855] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513668worker2 # [ 22.560327] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker1 # [ 22.587382] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 22.593739] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 22.752260] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513672worker2 # [ 22.757832] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker1 # [ 22.792741] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 22.798159] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 22.959318] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513676worker2 # [ 22.965191] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker1 # [ 22.999455] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 23.005255] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 23.166710] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513680worker2 # [ 23.172920] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker1 # [ 23.208839] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513682worker1 # [ 23.213112] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker2 # [ 23.377233] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513684worker2 # [ 23.381570] nix-grpc-daemon[777]: ts=2026-09-21T09:13:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker1 # [ 23.409873] nix-grpc-daemon[776]: ts=2026-09-21T09:13:35Z level=info event=scheduler_connected addr=lb:500513686worker1 # [ 23.415233] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker2 # [ 23.578283] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513688worker2 # [ 23.584328] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker1 # [ 23.612704] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 23.618590] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 23.786176] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513692worker2 # [ 23.792160] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker1 # [ 23.820397] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 23.826280] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 23.993490] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513696worker2 # [ 23.998882] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3697worker1 # [ 24.027382] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513698worker1 # [ 24.033045] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker2 # [ 24.200115] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513700worker2 # [ 24.205317] nix-grpc-daemon[777]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker1 # [ 24.236181] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 24.240457] nix-grpc-daemon[776]: ts=2026-09-21T09:13:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 24.408779] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513704worker2 # [ 24.412898] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker1 # [ 24.436781] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 24.442393] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 24.610059] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 24.615422] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker1 # [ 24.643443] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 24.649111] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 24.816788] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 24.821985] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker1 # [ 24.850181] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 24.855865] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 25.023213] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 25.028787] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker1 # [ 25.057366] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 25.062467] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 25.229913] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513720worker2 # [ 25.235219] nix-grpc-daemon[777]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker1 # [ 25.261335] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 25.267436] nix-grpc-daemon[776]: ts=2026-09-21T09:13:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 25.437937] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 25.442078] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker1 # [ 25.462201] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 25.467631] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 25.638993] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 25.644262] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 25.668696] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 25.674046] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 25.845652] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 25.851899] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 25.875561] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 25.881573] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 26.052440] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 26.057713] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker1 # [ 26.081983] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 26.087337] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker2 # [ 26.258849] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 26.264095] nix-grpc-daemon[777]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker1 # [ 26.290043] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_connected addr=lb:500513742worker1 # [ 26.294278] nix-grpc-daemon[776]: ts=2026-09-21T09:13:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker2 # [ 26.467010] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 26.471450] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker1 # [ 26.490448] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 26.495707] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 26.668140] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 26.673543] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker1 # [ 26.696739] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 26.701957] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker2 # [ 26.874906] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513752worker2 # [ 26.880411] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker1 # [ 26.902972] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 26.908652] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker2 # [ 27.081726] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513756worker2 # [ 27.087123] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3757worker1 # [ 27.109811] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513758worker1 # [ 27.115354] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker2 # [ 27.288630] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513760worker2 # [ 27.294148] nix-grpc-daemon[777]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker1 # [ 27.317947] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_connected addr=lb:500513762worker1 # [ 27.322207] nix-grpc-daemon[776]: ts=2026-09-21T09:13:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763worker2 # [ 27.497347] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513764worker2 # [ 27.502094] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3765worker1 # [ 27.518562] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513766worker1 # [ 27.523751] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3767worker2 # [ 27.703373] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513768worker2 # [ 27.710102] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3769worker1 # [ 27.724665] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513770worker1 # [ 27.729771] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3771worker2 # [ 27.911339] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513772worker2 # [ 27.916809] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3773worker1 # [ 27.930811] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513774worker1 # [ 27.935991] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3775worker2 # [ 28.117974] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513776worker2 # [ 28.123223] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3777worker1 # [ 28.136650] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513778worker1 # [ 28.141629] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3779worker2 # [ 28.324354] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513780worker2 # [ 28.329414] nix-grpc-daemon[777]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3781worker1 # [ 28.345014] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_connected addr=lb:500513782worker1 # [ 28.349374] nix-grpc-daemon[776]: ts=2026-09-21T09:13:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3783worker2 # [ 28.532570] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513784worker2 # [ 28.536682] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3785worker1 # [ 28.545060] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513786worker1 # [ 28.550291] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3787worker2 # [ 28.733452] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513788worker2 # [ 28.738725] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3789worker1 # [ 28.751602] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 28.757960] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791worker2 # [ 28.939879] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513792worker2 # [ 28.945175] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3793worker1 # [ 28.958390] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513794worker1 # [ 28.964411] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795worker2 # [ 29.146130] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513796worker2 # [ 29.151266] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3797worker1 # [ 29.165737] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513798worker1 # [ 29.176947] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3799worker2 # [ 29.352621] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513800worker2 # [ 29.357727] nix-grpc-daemon[777]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3801worker1 # [ 29.375667] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_connected addr=lb:500513802worker1 # [ 29.379998] nix-grpc-daemon[776]: ts=2026-09-21T09:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803worker2 # [ 29.560620] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513804worker2 # [ 29.564776] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3805worker1 # [ 29.576517] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513806worker1 # [ 29.581900] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807worker2 # [ 29.761414] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513808worker2 # [ 29.766611] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3809worker1 # [ 29.782842] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513810worker1 # [ 29.788484] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3811worker2 # [ 29.967925] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513812worker2 # [ 29.973097] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3813worker1 # [ 29.989678] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513814worker1 # [ 29.995222] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3815worker2 # [ 30.174267] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513816worker2 # [ 30.179595] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3817worker1 # [ 30.196096] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513818worker1 # [ 30.201408] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3819worker2 # [ 30.380649] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513820worker2 # [ 30.385828] nix-grpc-daemon[777]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3821worker1 # [ 30.403827] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_connected addr=lb:500513822worker1 # [ 30.408288] nix-grpc-daemon[776]: ts=2026-09-21T09:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3823worker2 # [ 30.588555] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513824worker2 # [ 30.593750] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3825worker1 # [ 30.604317] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513826worker1 # [ 30.609414] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3827worker2 # [ 30.794985] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513828worker2 # [ 30.800429] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker1 # [ 30.810360] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513830worker1 # [ 30.815640] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3831worker2 # [ 31.001570] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513832worker2 # [ 31.006771] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833worker1 # [ 31.016455] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513834worker1 # [ 31.021807] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3835worker2 # [ 31.207911] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513836worker2 # [ 31.213104] nix-grpc-daemon[777]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3837worker1 # [ 31.222922] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_connected addr=lb:500513838worker1 # [ 31.229282] nix-grpc-daemon[776]: ts=2026-09-21T09:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3839worker2 # [ 31.414201] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513840worker2 # [ 31.419713] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3841worker1 # [ 31.431921] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513842worker1 # [ 31.436235] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3843worker2 # [ 31.622948] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513844worker2 # [ 31.627287] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3845worker1 # [ 31.632719] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513846worker1 # [ 31.638078] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3847worker2 # [ 31.824115] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513848worker2 # [ 31.829269] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3849worker1 # [ 31.838925] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513850worker1 # [ 31.844340] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3851worker2 # [ 32.030535] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513852worker2 # [ 32.036176] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3853worker1 # [ 32.045283] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513854worker1 # [ 32.050570] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3855worker2 # [ 32.237388] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513856worker2 # [ 32.242670] nix-grpc-daemon[777]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3857worker1 # [ 32.251358] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_connected addr=lb:500513858worker1 # [ 32.256614] nix-grpc-daemon[776]: ts=2026-09-21T09:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3859worker2 # [ 32.445284] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513860worker2 # [ 32.450219] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3861worker1 # [ 32.458323] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513862lb # [ 32.533045] rustfs-bucket-start[1039]: mb s3://farm3863worker1 # [ 32.462850] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3864lb # [ 32.544281] systemd[1]: Finished rustfs-bucket.service.3865lb # [ 32.556244] systemd[1]: Starting niks3 server...3866worker2 # [ 32.653191] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513867worker2 # [ 32.657394] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3868worker1 # [ 32.663646] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513869worker1 # [ 32.669084] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3870lb # [ 32.827779] postgres[1055]: [1055] ERROR: relation "goose_db_version" does not exist at character 363871lb # [ 32.832237] postgres[1055]: [1055] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3872lb # [ 32.860860] niks3-server[1049]: 2026/09/21 09:13:45 OK 20241026095416_initial_model.sql (19.06ms)3873lb # [ 32.864810] niks3-server[1049]: 2026/09/21 09:13:45 OK 20251210153512_drop_unused_gin_index.sql (2.26ms)3874lb # [ 32.873642] niks3-server[1049]: 2026/09/21 09:13:45 OK 20251218171726_add_pins.sql (8.87ms)3875lb # [ 32.880081] niks3-server[1049]: 2026/09/21 09:13:45 OK 20260628120000_add_object_size_and_stats.sql (6.37ms)3876lb # [ 32.889135] niks3-server[1049]: 2026/09/21 09:13:45 OK 20260905000000_add_claims.sql (9.04ms)3877lb # [ 32.895037] niks3-server[1049]: 2026/09/21 09:13:45 OK 20260920000000_drop_claims.sql (5.77ms)3878lb # [ 32.898024] niks3-server[1049]: 2026/09/21 09:13:45 goose: successfully migrated database to version: 202609200000003879lb # [ 32.903079] niks3-server[1049]: 2026/09/21 09:13:45 OK 1_commit_pending_closure.sql (7.99ms)3880lb # [ 32.906935] niks3-server[1049]: 2026/09/21 09:13:45 OK 2_object_stats_trigger.sql (3.69ms)3881worker2 # [ 32.853825] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513882lb # [ 32.908570] niks3-server[1049]: 2026/09/21 09:13:45 goose: up to current file version: 23883worker2 # [ 32.859056] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3884lb # [ 32.913655] niks3-server[1049]: 2026/09/21 09:13:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3885worker1 # [ 32.870054] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513886worker1 # [ 32.875188] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3887lb # [ 32.971563] niks3-server[1049]: 2026/09/21 09:13:45 INFO Read proxy enabled — serving cache objects from S33888lb # [ 32.973942] niks3-server[1049]: 2026/09/21 09:13:45 INFO Using socket-activated listener address=0.0.0.0:57513889lb # [ 32.977876] niks3-server[1049]: 2026/09/21 09:13:45 INFO systemd watchdog enabled interval=15s3890lb # [ 32.982791] systemd[1]: Started niks3 server.3891lb # [ 32.983504] niks3-server[1049]: 2026/09/21 09:13:45 INFO Starting HTTP server address=0.0.0.0:57513892lb # [ 32.990106] systemd[1]: Reached target Multi-User System.3893lb # [ 32.990928] niks3-server[1049]: 2026/09/21 09:13:45 INFO lead: acquired remote=192.168.1.4:529843894lb # [ 32.995568] systemd[1]: Startup finished in 985ms (kernel) + 5.323s (initrd) + 26.672s (userspace) = 32.981s.3895worker2 # [ 32.945502] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_take_over3896worker2 # [ 33.060277] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513897worker2 # [ 33.065409] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3898worker1 # [ 33.076134] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513899worker1 # [ 33.081361] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3900worker2 # [ 33.266600] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513901worker2 # [ 33.272479] nix-grpc-daemon[777]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3902worker1 # [ 33.282214] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_connected addr=lb:500513903worker1 # [ 33.287324] nix-grpc-daemon[776]: ts=2026-09-21T09:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3904worker2 # [ 33.473366] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=scheduler_connected addr=lb:500513905worker2 # [ 33.498598] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=oidc_jwks provider=mock keys=13906worker2 # [ 33.503254] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:354343907worker2 # [ 33.508689] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03908worker1 # [ 33.489350] nix-grpc-daemon[776]: ts=2026-09-21T09:13:46Z level=info event=scheduler_connected addr=lb:500513909worker2 # [ 33.515529] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:354343910worker2 # [ 33.520438] nix-grpc-daemon[777]: ts=2026-09-21T09:13:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03911lb: (finished: waiting for unit niks3.service, in 19.29 seconds)3912lb: waiting for TCP port 5751 on localhost3913lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3914lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3915lb: waiting for unit envoy.service3916lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3917lb: waiting for TCP port 50051 on localhost3918lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3919lb # [ 34.275218] envoy[911]: conn peer=[::1]:43132 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33920lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3921worker2 # [ 34.416700] systemd[1]: Starting scheduler-token.service...3922worker2 # [ 34.513409] systemd[1]: scheduler-token.service: Deactivated successfully.3923worker2 # [ 34.515813] systemd[1]: Finished scheduler-token.service.3924worker2 # [ 34.518366] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3925lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3926lb: must succeed: curl -sf localhost:9901/clusters3927lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3928lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3929subtest: one node schedules and both builders hold a WorkerSession on it3930lb: must succeed: curl -sf localhost:9901/clusters3931lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3932worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3933worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3934lb: must succeed: curl -sf localhost:9901/clusters3935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3936lb: must succeed: curl -sf localhost:9901/clusters3937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3938lb: must succeed: curl -sf localhost:9901/clusters3939lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3940lb: must succeed: curl -sf localhost:9901/clusters3941lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3942worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3943worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3944worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3945worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3946lb: must succeed: curl -sf localhost:9901/clusters3947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3948lb: must succeed: curl -sf localhost:9901/clusters3949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3950worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3951worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3952worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3953worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3954worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3955worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3956worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3957worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)3958(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.87 seconds)3959subtest: balancer auth: client cert, bearer token, nothing, unknown CN3960worker1 # [ 35.918133] nix-grpc-daemon[776]: ts=2026-09-21T09:13:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:361963961worker1 # [ 35.927993] systemd[1]: Started Nix Daemon.3962worker1 # [ 36.196527] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3963worker1 # [ 36.234005] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)3964worker1 # [ 36.323396] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5)3965worker1 # [ 36.429162] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 3/5)3966worker1 # [ 36.507664] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 237 ms (attempt 4/5)3967worker1 # [ 36.755697] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3968worker1 # [ 36.778624] nix-grpc-daemon[776]: ts=2026-09-21T09:13:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:361963969worker1 # [ 36.793842] nix-grpc-daemon[776]: ts=2026-09-21T09:13:49Z level=debug event=path_info_miss path=p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test error="error: path '/nix/store/p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test' is required, but there is no substituter that can build it"3970worker2 # [ 36.848969] nix-grpc-daemon[777]: ts=2026-09-21T09:13:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:354363971worker2 # [ 36.862143] systemd[1]: Started Nix Daemon.3972worker2 # [ 37.159256] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3973worker2 # [ 37.187335] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)3974worker2 # [ 37.228405] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 2/5)3975worker2 # [ 37.257955] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 3/5)3976worker2 # [ 37.409986] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 366 ms (attempt 4/5)3977worker2 # [ 37.786059] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3978worker2 # [ 37.802016] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=path_info_miss path=p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test error="error: path '/nix/store/p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test' is required, but there is no substituter that can build it"3979client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3980client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)3981client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3982client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3983worker2 # [ 38.257285] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:354363984worker2 # [ 38.277581] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:354363985worker2 # [ 38.289516] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=path_info_miss path=p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test error="error: path '/nix/store/p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test' is required, but there is no substituter that can build it"3986worker2 # [ 38.303814] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:354363987worker2 # [ 38.311887] nix-grpc-daemon[777]: ts=2026-09-21T09:13:50Z level=debug event=path_info_miss path=p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test error="error: path '/nix/store/p6p9342ifnppad60sm4s8zcrc0dk790i-nixos-system-client-test' is required, but there is no substituter that can build it"3988client: must succeed: rm -r /var/lib/nix-grpc-store3989client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3990worker2 # [ 38.620157] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=info event=denied method=QueryMissing cn=- role=none3991worker1 # [ 38.863754] nix-grpc-daemon[776]: ts=2026-09-21T09:13:51Z level=info event=denied method=QueryMissing cn=stranger role=none3992(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.41 seconds)3993subtest: DAG build is scheduled across workers and lands in the cache3994client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23995worker2 # [ 39.157187] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354363996client # don't know how to build these paths:3997client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3998worker1 # [ 39.152816] nix-grpc-daemon[776]: ts=2026-09-21T09:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361963999worker2 # [ 39.207974] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04000worker2 # [ 39.216871] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=14001worker2 # [ 39.267257] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364002worker2 # [ 39.280572] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364003client # copying 1 paths to 'lb:50051'...4004worker2 # [ 39.288109] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364005client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4006worker2 # [ 39.308665] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4007worker2 # [ 39.326340] nix-grpc-daemon[777]: ts=2026-09-21T09:13:51Z level=info event=niks3_push_started pid=8744008worker2 # [ 39.607360] nix-grpc-daemon[874]: time=2026-09-21T09:13:52.202Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4009worker2 # [ 39.804146] nix-daemon[862]: accepted connection from pid 887, user nix-grpc-daemon (trusted)4010worker2 # [ 39.823205] nix-daemon[862]: reaped child process 890, status = succeeded4011lb # [ 39.911999] envoy[911]: conn peer=[2001:db8:1::1]:59226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=19134012lb # [ 39.912567] envoy[911]: conn peer=[2001:db8:1::1]:59236 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=694013lb # [ 39.912686] envoy[911]: conn peer=[2001:db8:1::1]:59240 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=124014lb # [ 39.912787] envoy[911]: conn peer=[2001:db8:1::1]:59252 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=214015worker2 # [ 40.006952] nix-daemon[862]: accepted connection from pid 892, user nix-grpc-daemon (trusted)4016worker2 # [ 40.026385] nix-daemon[862]: reaped child process 895, status = succeeded4017lb # [ 40.090137] niks3-server[1049]: 2026/09/21 09:13:52 INFO Received uploads request method=POST path=/api/pending_closures4018worker2 # [ 40.063019] nix-grpc-daemon[874]: time=2026-09-21T09:13:52.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4019worker2 # [ 40.066196] nix-grpc-daemon[874]: time=2026-09-21T09:13:52.661Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4020lb # [ 40.156821] niks3-server[1049]: 2026/09/21 09:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4021lb # [ 40.177545] niks3-server[1049]: 2026/09/21 09:13:52 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4022lb # [ 40.201930] niks3-server[1049]: 2026/09/21 09:13:52 INFO Signed narinfos id=1 count=14023worker2 # [ 40.154139] nix-grpc-daemon[874]: time=2026-09-21T09:13:52.749Z level=INFO msg="Uploading 1 narinfos"4024lb # [ 40.217483] niks3-server[1049]: 2026/09/21 09:13:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4025lb # [ 40.225006] niks3-server[1049]: 2026/09/21 09:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4026lb # [ 40.234125] niks3-server[1049]: 2026/09/21 09:13:52 INFO Completed upload id=14027worker2 # [ 40.183656] nix-grpc-daemon[874]: time=2026-09-21T09:13:52.779Z level=INFO msg="Upload complete. (567ms)"4028lb # [ 40.242733] niks3-server[1049]: 2026/09/21 09:13:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4029worker2 # [ 40.195772] nix-grpc-daemon[777]: ts=2026-09-21T09:13:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=5044030worker2 # [ 40.216279] nix-grpc-daemon[777]: ts=2026-09-21T09:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364031worker2 # [ 40.222198] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4032worker2 # [ 40.232262] nix-daemon[862]: reaped child process 872, status = succeeded4033client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4034worker2 # [ 40.243228] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4035client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4036worker2 # [ 41.063371] nix-daemon[862]: reaped child process 899, status = succeeded4037client # farm-a> LOG-farm-a-t14038worker2 # [ 41.229437] nix-daemon[862]: accepted connection from pid 914, user nix-grpc-daemon (trusted)4039worker2 # [ 41.248968] nix-daemon[862]: reaped child process 917, status = succeeded4040lb # [ 41.311990] niks3-server[1049]: 2026/09/21 09:13:53 INFO Received uploads request method=POST path=/api/pending_closures4041worker2 # [ 41.280270] nix-grpc-daemon[874]: time=2026-09-21T09:13:53.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4042worker2 # [ 41.282246] nix-grpc-daemon[874]: time=2026-09-21T09:13:53.875Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4043lb # [ 41.373400] niks3-server[1049]: 2026/09/21 09:13:53 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4044lb # [ 41.386878] niks3-server[1049]: 2026/09/21 09:13:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4045worker2 # [ 41.341674] nix-grpc-daemon[874]: time=2026-09-21T09:13:53.936Z level=INFO msg="Uploading 1 narinfos"4046lb # [ 41.398695] niks3-server[1049]: 2026/09/21 09:13:53 INFO Signed narinfos id=2 count=14047lb # [ 41.403001] niks3-server[1049]: 2026/09/21 09:13:53 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4048lb # [ 41.407495] niks3-server[1049]: 2026/09/21 09:13:53 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4049lb # [ 41.415755] niks3-server[1049]: 2026/09/21 09:13:53 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4050lb # [ 41.420373] niks3-server[1049]: 2026/09/21 09:13:54 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4051lb # [ 41.424304] niks3-server[1049]: 2026/09/21 09:13:54 INFO Completed upload id=24052worker2 # [ 41.373671] nix-grpc-daemon[874]: time=2026-09-21T09:13:53.970Z level=INFO msg="Upload complete. (320ms)"4053worker2 # [ 41.397103] nix-grpc-daemon[777]: ts=2026-09-21T09:13:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14054worker2 # [ 41.412420] nix-grpc-daemon[777]: ts=2026-09-21T09:13:53Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04055worker2 # [ 41.416786] nix-grpc-daemon[777]: ts=2026-09-21T09:13:53Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=14056worker2 # [ 41.420334] nix-daemon[862]: reaped child process 897, status = succeeded4057worker2 # [ 41.442200] nix-grpc-daemon[777]: ts=2026-09-21T09:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364058worker2 # [ 41.459790] nix-grpc-daemon[777]: ts=2026-09-21T09:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364059worker2 # [ 41.475772] nix-grpc-daemon[874]: time=2026-09-21T09:13:54.072Z level=INFO msg="All 1 paths already cached"4060client # copying 1 paths to 'lb:50051'...4061worker2 # [ 41.486658] nix-grpc-daemon[777]: ts=2026-09-21T09:13:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364062client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4063worker2 # [ 41.509885] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4064worker2 # [ 41.728124] nix-daemon[862]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4065worker2 # [ 41.747610] nix-daemon[862]: reaped child process 926, status = succeeded4066worker2 # [ 41.924712] nix-daemon[862]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4067worker2 # [ 41.942623] nix-daemon[862]: reaped child process 931, status = succeeded4068lb # [ 42.005726] niks3-server[1049]: 2026/09/21 09:13:54 INFO Received uploads request method=POST path=/api/pending_closures4069worker2 # [ 41.965868] nix-grpc-daemon[874]: time=2026-09-21T09:13:54.561Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4070worker2 # [ 41.967405] nix-grpc-daemon[874]: time=2026-09-21T09:13:54.561Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4071worker2 # [ 41.995861] nix-grpc-daemon[874]: time=2026-09-21T09:13:54.592Z level=INFO msg="Uploading 1 narinfos"4072lb # [ 42.038335] niks3-server[1049]: 2026/09/21 09:13:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4073lb # [ 42.056001] niks3-server[1049]: 2026/09/21 09:13:54 INFO Signed narinfos id=3 count=14074lb # [ 42.063562] niks3-server[1049]: 2026/09/21 09:13:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4075lb # [ 42.070488] niks3-server[1049]: 2026/09/21 09:13:54 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4076worker2 # [ 42.031857] nix-grpc-daemon[874]: time=2026-09-21T09:13:54.628Z level=INFO msg="Upload complete. (503ms)"4077lb # [ 42.083241] niks3-server[1049]: 2026/09/21 09:13:54 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4078lb # [ 42.088846] niks3-server[1049]: 2026/09/21 09:13:54 INFO Completed upload id=34079worker2 # [ 42.040306] nix-grpc-daemon[777]: ts=2026-09-21T09:13:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=6244080lb # [ 42.092139] niks3-server[1049]: 2026/09/21 09:13:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4081worker2 # [ 42.051788] nix-grpc-daemon[777]: ts=2026-09-21T09:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364082worker2 # [ 42.059054] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4083worker2 # [ 42.064704] nix-daemon[862]: reaped child process 921, status = succeeded4084client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4085worker2 # [ 42.077924] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4086client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4087client # farm-b> LOG-farm-b-t14088worker2 # [ 43.505328] nix-daemon[862]: reaped child process 935, status = succeeded4089worker2 # [ 43.672797] nix-daemon[862]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4090worker2 # [ 43.692962] nix-daemon[862]: reaped child process 954, status = succeeded4091lb # [ 43.755745] niks3-server[1049]: 2026/09/21 09:13:56 INFO Received uploads request method=POST path=/api/pending_closures4092worker2 # [ 43.715469] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.311Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4093worker2 # [ 43.717158] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.311Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4094lb # [ 43.792290] niks3-server[1049]: 2026/09/21 09:13:56 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4095lb # [ 43.802259] niks3-server[1049]: 2026/09/21 09:13:56 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4096worker2 # [ 43.762096] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.358Z level=INFO msg="Uploading 1 narinfos"4097lb # [ 43.816997] niks3-server[1049]: 2026/09/21 09:13:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4098lb # [ 43.824899] niks3-server[1049]: 2026/09/21 09:13:56 INFO Signed narinfos id=4 count=14099lb # [ 43.827054] niks3-server[1049]: 2026/09/21 09:13:56 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4100lb # [ 43.833021] niks3-server[1049]: 2026/09/21 09:13:56 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4101lb # [ 43.838722] niks3-server[1049]: 2026/09/21 09:13:56 INFO Completed upload id=44102worker2 # [ 43.787729] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.384Z level=INFO msg="Upload complete. (299ms)"4103lb # [ 43.841239] niks3-server[1049]: 2026/09/21 09:13:56 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4104worker2 # [ 43.802138] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14105worker2 # [ 43.815947] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04106worker2 # [ 43.819713] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14107worker2 # [ 43.823476] nix-daemon[862]: reaped child process 933, status = succeeded4108worker2 # [ 43.845886] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364109worker2 # [ 43.860888] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364110worker2 # [ 43.874481] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.470Z level=INFO msg="All 2 paths already cached"4111client # copying 1 paths to 'lb:50051'...4112worker2 # [ 43.885190] nix-grpc-daemon[777]: ts=2026-09-21T09:13:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364113client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4114worker2 # [ 43.908215] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4115worker2 # [ 44.106828] nix-daemon[862]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4116worker2 # [ 44.127034] nix-daemon[862]: reaped child process 963, status = succeeded4117worker2 # [ 44.300131] nix-daemon[862]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4118worker2 # [ 44.319282] nix-daemon[862]: reaped child process 968, status = succeeded4119lb # [ 44.381959] niks3-server[1049]: 2026/09/21 09:13:56 INFO Received uploads request method=POST path=/api/pending_closures4120worker2 # [ 44.340672] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.936Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4121worker2 # [ 44.342188] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.936Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4122lb # [ 44.414099] niks3-server[1049]: 2026/09/21 09:13:57 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4123worker2 # [ 44.372823] nix-grpc-daemon[874]: time=2026-09-21T09:13:56.968Z level=INFO msg="Uploading 1 narinfos"4124lb # [ 44.429557] niks3-server[1049]: 2026/09/21 09:13:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4125lb # [ 44.439986] niks3-server[1049]: 2026/09/21 09:13:57 INFO Signed narinfos id=5 count=14126lb # [ 44.449675] niks3-server[1049]: 2026/09/21 09:13:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4127lb # [ 44.453768] niks3-server[1049]: 2026/09/21 09:13:57 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4128worker2 # [ 44.407163] nix-grpc-daemon[874]: time=2026-09-21T09:13:57.003Z level=INFO msg="Upload complete. (482ms)"4129lb # [ 44.458632] niks3-server[1049]: 2026/09/21 09:13:57 INFO Completed upload id=54130lb # [ 44.461274] niks3-server[1049]: 2026/09/21 09:13:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4131worker2 # [ 44.413784] nix-grpc-daemon[777]: ts=2026-09-21T09:13:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=7524132worker2 # [ 44.427666] nix-grpc-daemon[777]: ts=2026-09-21T09:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364133worker2 # [ 44.433720] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4134worker2 # [ 44.438651] nix-daemon[862]: reaped child process 958, status = succeeded4135client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4136client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4137worker2 # [ 44.452163] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4138client # farm-top> LOG-farm-top-t14139worker2 # [ 45.479066] nix-daemon[862]: reaped child process 972, status = succeeded4140worker2 # [ 45.645823] nix-daemon[862]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4141worker2 # [ 45.664259] nix-daemon[862]: reaped child process 990, status = succeeded4142lb # [ 45.728727] niks3-server[1049]: 2026/09/21 09:13:58 INFO Received uploads request method=POST path=/api/pending_closures4143worker2 # [ 45.691805] nix-grpc-daemon[874]: time=2026-09-21T09:13:58.286Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4144worker2 # [ 45.693505] nix-grpc-daemon[874]: time=2026-09-21T09:13:58.286Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4145lb # [ 45.761880] niks3-server[1049]: 2026/09/21 09:13:58 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4146lb # [ 45.778954] niks3-server[1049]: 2026/09/21 09:13:58 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4147worker2 # [ 45.751494] nix-grpc-daemon[874]: time=2026-09-21T09:13:58.347Z level=INFO msg="Uploading 1 narinfos"4148lb # [ 45.797672] niks3-server[1049]: 2026/09/21 09:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4149lb # [ 45.806412] niks3-server[1049]: 2026/09/21 09:13:58 INFO Signed narinfos id=6 count=14150lb # [ 45.813828] niks3-server[1049]: 2026/09/21 09:13:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4151lb # [ 45.827854] niks3-server[1049]: 2026/09/21 09:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4152lb # [ 45.833566] niks3-server[1049]: 2026/09/21 09:13:58 INFO Completed upload id=64153worker2 # [ 45.782141] nix-grpc-daemon[874]: time=2026-09-21T09:13:58.378Z level=INFO msg="Upload complete. (317ms)"4154lb # [ 45.837250] niks3-server[1049]: 2026/09/21 09:13:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4155worker2 # [ 45.798667] nix-grpc-daemon[777]: ts=2026-09-21T09:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14156worker2 # [ 45.817065] nix-daemon[862]: reaped child process 970, status = succeeded4157client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 6.95 seconds)4158client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4159client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4160client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4161client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4162client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4163client # copying 3 paths...4164client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4165client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4166client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4167client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4168(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.61 seconds)4169subtest: repeat is answered Cached by the scheduler without building4170worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4171worker1 # finding garbage collector roots...4172worker1 # deleting specified paths...4173worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4174worker1 # deleting unused links...4175worker1 # note: hard linking is currently saving 0.0 KiB4176worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4177worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4178worker2 # finding garbage collector roots...4179worker2 # deleting specified paths...4180worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4181worker2 # deleting unused links...4182worker2 # note: hard linking is currently saving 0.0 KiB4183worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4184client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24185worker1 # [ 47.355462] nix-grpc-daemon[776]: ts=2026-09-21T09:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:361964186client # don't know how to build these paths:4187client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4188worker2 # [ 47.389328] nix-grpc-daemon[777]: ts=2026-09-21T09:13:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364189worker2 # [ 47.397178] nix-grpc-daemon[874]: time=2026-09-21T09:13:59.991Z level=INFO msg="All 2 paths already cached"4190worker2 # [ 47.411412] nix-grpc-daemon[777]: ts=2026-09-21T09:14:00Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14191client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.33 seconds)4192client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4193client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4194lb: must succeed: curl -sf localhost:9901/clusters4195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4196lb: must succeed: curl -sf localhost:9901/clusters4197lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4198worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4199worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4200(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.46 seconds)4201subtest: inputs already in the cache are not sent to the scheduler4202lb: must succeed: curl -sf localhost:9901/clusters4203lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4204lb: must succeed: curl -sf localhost:9901/clusters4205lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4206worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4207worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4208client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24209worker2 # [ 48.447312] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354364210client # don't know how to build these paths:4211client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4212worker2 # [ 48.458008] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364213worker2 # [ 48.471883] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.067Z level=INFO msg="All 2 paths already cached"4214worker2 # [ 48.488149] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04215worker2 # [ 48.494882] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14216worker2 # [ 48.533862] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364217worker2 # [ 48.549866] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364218worker2 # [ 48.564821] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.161Z level=INFO msg="All 2 paths already cached"4219client # copying 1 paths to 'lb:50051'...4220client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4221worker2 # [ 48.574965] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364222worker2 # [ 48.590032] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4223worker2 # [ 48.795876] nix-daemon[862]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4224worker2 # [ 48.815432] nix-daemon[862]: reaped child process 1023, status = succeeded4225worker2 # [ 48.993000] nix-daemon[862]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)4226worker2 # [ 49.011276] nix-daemon[862]: reaped child process 1028, status = succeeded4227lb # [ 49.074300] niks3-server[1049]: 2026/09/21 09:14:01 INFO Received uploads request method=POST path=/api/pending_closures4228worker2 # [ 49.033801] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.629Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4229worker2 # [ 49.035266] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.630Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4230lb # [ 49.102328] niks3-server[1049]: 2026/09/21 09:14:01 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4231lb # [ 49.117000] niks3-server[1049]: 2026/09/21 09:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4232worker2 # [ 49.071918] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.668Z level=INFO msg="Uploading 1 narinfos"4233lb # [ 49.127727] niks3-server[1049]: 2026/09/21 09:14:01 INFO Signed narinfos id=7 count=14234lb # [ 49.136913] niks3-server[1049]: 2026/09/21 09:14:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4235worker2 # [ 49.096241] nix-grpc-daemon[874]: time=2026-09-21T09:14:01.692Z level=INFO msg="Upload complete. (489ms)"4236lb # [ 49.147209] niks3-server[1049]: 2026/09/21 09:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4237lb # [ 49.153220] niks3-server[1049]: 2026/09/21 09:14:01 INFO Completed upload id=74238worker2 # [ 49.104120] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=7524239lb # [ 49.157247] niks3-server[1049]: 2026/09/21 09:14:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4240worker2 # [ 49.116244] nix-grpc-daemon[777]: ts=2026-09-21T09:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364241worker2 # [ 49.122917] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4242worker2 # [ 49.128009] nix-daemon[862]: reaped child process 1018, status = succeeded4243client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4244worker2 # [ 49.140126] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4245client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4246lb # [ 49.908262] envoy[911]: conn peer=[2001:db8:1::1]:59268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9506 tx=3829 ms=66704247lb # [ 49.908656] envoy[911]: conn peer=[2001:db8:1::1]:39274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=503 ms=554248client # farm-top> LOG-farm-top-t1b4249worker2 # [ 50.180291] nix-daemon[862]: reaped child process 1032, status = succeeded4250worker2 # [ 50.351638] nix-daemon[862]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4251worker2 # [ 50.372791] nix-daemon[862]: reaped child process 1050, status = succeeded4252lb # [ 50.435907] niks3-server[1049]: 2026/09/21 09:14:03 INFO Received uploads request method=POST path=/api/pending_closures4253worker2 # [ 50.396430] nix-grpc-daemon[874]: time=2026-09-21T09:14:02.991Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4254worker2 # [ 50.399240] nix-grpc-daemon[874]: time=2026-09-21T09:14:02.991Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4255lb # [ 50.482431] niks3-server[1049]: 2026/09/21 09:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4256worker2 # [ 50.441072] nix-grpc-daemon[874]: time=2026-09-21T09:14:03.037Z level=INFO msg="Uploading 1 narinfos"4257lb # [ 50.498371] niks3-server[1049]: 2026/09/21 09:14:03 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4258lb # [ 50.514054] niks3-server[1049]: 2026/09/21 09:14:03 INFO Signed narinfos id=8 count=14259worker2 # [ 50.468554] nix-grpc-daemon[874]: time=2026-09-21T09:14:03.064Z level=INFO msg="Upload complete. (315ms)"4260lb # [ 50.520084] niks3-server[1049]: 2026/09/21 09:14:03 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4261lb # [ 50.525659] niks3-server[1049]: 2026/09/21 09:14:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4262lb # [ 50.530244] niks3-server[1049]: 2026/09/21 09:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4263lb # [ 50.535099] niks3-server[1049]: 2026/09/21 09:14:03 INFO Completed upload id=84264lb # [ 50.537233] niks3-server[1049]: 2026/09/21 09:14:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4265worker2 # [ 50.484978] nix-grpc-daemon[777]: ts=2026-09-21T09:14:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14266worker2 # [ 50.496150] nix-daemon[862]: reaped child process 1030, status = succeeded4267client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.34 seconds)4268client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4269client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4270lb: must succeed: curl -sf localhost:9901/clusters4271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4272lb: must succeed: curl -sf localhost:9901/clusters4273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4274worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4275worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4276(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.00 seconds)4277subtest: two clients wanting the same drv build it once4278client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4279client # [ 51.082963] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4280client # Running as unit: dedup1.service; invocation ID: 6cdb6fec763643e387f480f488394e854281client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4282client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4283client # [ 51.353606] nix[879]: don't know how to build these paths:4284client # [ 51.354639] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4285worker1 # [ 51.310625] nix-grpc-daemon[776]: ts=2026-09-21T09:14:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:361964286worker2 # [ 51.344323] nix-grpc-daemon[777]: ts=2026-09-21T09:14:03Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04287worker2 # [ 51.344750] nix-grpc-daemon[777]: ts=2026-09-21T09:14:03Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14288worker1 # [ 51.339222] nix-grpc-daemon[776]: ts=2026-09-21T09:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964289worker1 # [ 51.359585] nix-grpc-daemon[776]: ts=2026-09-21T09:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964290client # [ 51.415750] nix[879]: copying 1 paths to 'lb:50051'...4291client # [ 51.419714] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4292worker1 # [ 51.377724] nix-grpc-daemon[776]: ts=2026-09-21T09:14:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361964293worker1 # [ 51.393724] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4294worker1 # [ 51.421993] nix-grpc-daemon[776]: ts=2026-09-21T09:14:04Z level=info event=niks3_push_started pid=8664295worker1 # [ 51.686194] nix-grpc-daemon[866]: time=2026-09-21T09:14:04.274Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4296worker1 # [ 51.889750] nix-daemon[840]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4297worker1 # [ 51.910694] nix-daemon[840]: reaped child process 882, status = succeeded4298worker1 # [ 52.097474] nix-daemon[840]: accepted connection from pid 884, user nix-grpc-daemon (trusted)4299worker1 # [ 52.116210] nix-daemon[840]: reaped child process 887, status = succeeded4300lb # [ 52.203793] niks3-server[1049]: 2026/09/21 09:14:04 INFO Received uploads request method=POST path=/api/pending_closures4301worker1 # [ 52.142670] nix-grpc-daemon[866]: time=2026-09-21T09:14:04.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4302worker1 # [ 52.144510] nix-grpc-daemon[866]: time=2026-09-21T09:14:04.734Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4303client # [ 52.201449] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4304client # Running as unit: dedup2.service; invocation ID: cee011784dfc44db8adbc4b671cd92884305lb # [ 52.251189] niks3-server[1049]: 2026/09/21 09:14:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4306worker1 # [ 52.189035] nix-grpc-daemon[866]: time=2026-09-21T09:14:04.778Z level=INFO msg="Uploading 1 narinfos"4307lb # [ 52.264133] niks3-server[1049]: 2026/09/21 09:14:04 INFO Signed narinfos id=9 count=14308client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4309lb # [ 52.270768] niks3-server[1049]: 2026/09/21 09:14:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4310lb # [ 52.277724] niks3-server[1049]: 2026/09/21 09:14:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4311lb # [ 52.284523] niks3-server[1049]: 2026/09/21 09:14:04 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4312worker1 # [ 52.216862] nix-grpc-daemon[866]: time=2026-09-21T09:14:04.806Z level=INFO msg="Upload complete. (521ms)"4313lb # [ 52.289394] niks3-server[1049]: 2026/09/21 09:14:04 INFO Completed upload id=94314lb # [ 52.293248] niks3-server[1049]: 2026/09/21 09:14:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4315worker1 # [ 52.224271] nix-grpc-daemon[776]: ts=2026-09-21T09:14:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=1 nar_bytes_in=4324316worker1 # [ 52.230725] nix-grpc-daemon[776]: ts=2026-09-21T09:14:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964317worker1 # [ 52.242827] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4318worker1 # [ 52.246758] nix-daemon[840]: reaped child process 864, status = succeeded4319client # [ 52.301360] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4320worker1 # [ 52.261337] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4321worker2 # [ 52.541658] nix-grpc-daemon[777]: ts=2026-09-21T09:14:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354364322client # [ 52.571899] nix[895]: don't know how to build these paths:4323client # [ 52.573370] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4324worker2 # [ 52.559740] nix-grpc-daemon[777]: ts=2026-09-21T09:14:05Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04325worker1 # [ 52.545921] nix-grpc-daemon[776]: ts=2026-09-21T09:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964326client # [ 52.595507] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4327client: must succeed: sleep 24328client: (finished: must succeed: sleep 2, in 2.06 seconds)4329client: waiting for success: ! systemctl is-active dedup1 dedup24330worker1 # [ 55.907427] nix-daemon[840]: reaped child process 895, status = succeeded4331worker1 # [ 56.094096] nix-daemon[840]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4332worker1 # [ 56.112756] nix-daemon[840]: reaped child process 923, status = succeeded4333lb # [ 56.202262] niks3-server[1049]: 2026/09/21 09:14:08 INFO Received uploads request method=POST path=/api/pending_closures4334worker1 # [ 56.139445] nix-grpc-daemon[866]: time=2026-09-21T09:14:08.728Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4335worker1 # [ 56.141546] nix-grpc-daemon[866]: time=2026-09-21T09:14:08.728Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4336lb # [ 56.235695] niks3-server[1049]: 2026/09/21 09:14:08 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4337lb # [ 56.253784] niks3-server[1049]: 2026/09/21 09:14:08 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4338lb # [ 56.270386] niks3-server[1049]: 2026/09/21 09:14:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4339worker1 # [ 56.205498] nix-grpc-daemon[866]: time=2026-09-21T09:14:08.794Z level=INFO msg="Uploading 1 narinfos"4340lb # [ 56.278469] niks3-server[1049]: 2026/09/21 09:14:08 INFO Signed narinfos id=10 count=14341lb # [ 56.286757] niks3-server[1049]: 2026/09/21 09:14:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4342lb # [ 56.296130] niks3-server[1049]: 2026/09/21 09:14:08 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4343lb # [ 56.308595] niks3-server[1049]: 2026/09/21 09:14:08 INFO Completed upload id=104344worker1 # [ 56.236700] nix-grpc-daemon[866]: time=2026-09-21T09:14:08.826Z level=INFO msg="Upload complete. (362ms)"4345lb # [ 56.311224] niks3-server[1049]: 2026/09/21 09:14:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4346worker1 # [ 56.249217] nix-grpc-daemon[776]: ts=2026-09-21T09:14:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14347worker1 # [ 56.268338] nix-grpc-daemon[776]: ts=2026-09-21T09:14:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14348worker1 # [ 56.275386] nix-daemon[840]: reaped child process 893, status = succeeded4349client # [ 56.355639] systemd[1]: dedup1.service: Deactivated successfully.4350client # [ 56.360132] systemd[1]: dedup1.service: Consumed 174ms CPU time over 5.276s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.4351client # [ 56.367881] systemd[1]: dedup2.service: Deactivated successfully.4352client # [ 56.373461] systemd[1]: dedup2.service: Consumed 209ms CPU time over 4.167s wall clock time, 33.2M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4353client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4354client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4355client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4356worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4357worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4358worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4359worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4360worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4361worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4362worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4363worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4364(finished: subtest: two clients wanting the same drv build it once, in 6.32 seconds)4365subtest: build hook: nix-daemon with builders = grpc://lb4366client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4367client # [ 57.567529] systemd[1]: Started Nix Daemon.4368client # [ 57.767897] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4369worker1 # [ 57.967095] nix-grpc-daemon[776]: ts=2026-09-21T09:14:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:361964370worker1 # [ 57.977180] nix-grpc-daemon[776]: ts=2026-09-21T09:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964371worker2 # [ 58.024190] nix-grpc-daemon[777]: ts=2026-09-21T09:14:10Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04372worker2 # [ 58.024661] nix-grpc-daemon[777]: ts=2026-09-21T09:14:10Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14373worker2 # [ 58.072219] nix-grpc-daemon[777]: ts=2026-09-21T09:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364374worker2 # [ 58.096072] nix-grpc-daemon[777]: ts=2026-09-21T09:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364375worker2 # [ 58.116230] nix-grpc-daemon[777]: ts=2026-09-21T09:14:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364376worker2 # [ 58.132726] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4377worker2 # [ 58.353534] nix-daemon[862]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4378worker2 # [ 58.374105] nix-daemon[862]: reaped child process 1089, status = succeeded4379worker2 # [ 58.569589] nix-daemon[862]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4380worker2 # [ 58.589298] nix-daemon[862]: reaped child process 1094, status = succeeded4381lb # [ 58.654015] niks3-server[1049]: 2026/09/21 09:14:11 INFO Received uploads request method=POST path=/api/pending_closures4382worker2 # [ 58.613675] nix-grpc-daemon[874]: time=2026-09-21T09:14:11.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4383worker2 # [ 58.615166] nix-grpc-daemon[874]: time=2026-09-21T09:14:11.209Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4384lb # [ 58.686040] niks3-server[1049]: 2026/09/21 09:14:11 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4385lb # [ 58.706834] niks3-server[1049]: 2026/09/21 09:14:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4386worker2 # [ 58.661269] nix-grpc-daemon[874]: time=2026-09-21T09:14:11.257Z level=INFO msg="Uploading 1 narinfos"4387lb # [ 58.717991] niks3-server[1049]: 2026/09/21 09:14:11 INFO Signed narinfos id=11 count=14388lb # [ 58.723872] niks3-server[1049]: 2026/09/21 09:14:11 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4389lb # [ 58.735314] niks3-server[1049]: 2026/09/21 09:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4390lb # [ 58.740726] niks3-server[1049]: 2026/09/21 09:14:11 INFO Completed upload id=114391worker2 # [ 58.689263] nix-grpc-daemon[874]: time=2026-09-21T09:14:11.285Z level=INFO msg="Upload complete. (536ms)"4392lb # [ 58.743283] niks3-server[1049]: 2026/09/21 09:14:11 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4393worker2 # [ 58.700924] nix-grpc-daemon[777]: ts=2026-09-21T09:14:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=5124394worker2 # [ 58.712498] nix-grpc-daemon[777]: ts=2026-09-21T09:14:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364395worker2 # [ 58.721139] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4396worker2 # [ 58.726674] nix-daemon[862]: reaped child process 1084, status = succeeded4397worker2 # [ 58.741771] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4398lb # [ 59.912788] envoy[911]: conn peer=[2001:db8:1::1]:39286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1705 ms=20484399worker2 # [ 59.990513] nix-daemon[862]: reaped child process 1098, status = succeeded4400worker2 # [ 60.168133] nix-daemon[862]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4401worker2 # [ 60.185736] nix-daemon[862]: reaped child process 1116, status = succeeded4402lb # [ 59.917738] envoy[911]: conn peer=[2001:db8:1::1]:44146 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1082 ms=37454403lb # [ 59.917944] envoy[911]: conn peer=[2001:db8:1::1]:44136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1372 ms=49554404lb # [ 60.250141] niks3-server[1049]: 2026/09/21 09:14:12 INFO Received uploads request method=POST path=/api/pending_closures4405worker2 # [ 60.220474] nix-grpc-daemon[874]: time=2026-09-21T09:14:12.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4406worker2 # [ 60.222166] nix-grpc-daemon[874]: time=2026-09-21T09:14:12.815Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4407lb # [ 60.313166] niks3-server[1049]: 2026/09/21 09:14:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4408worker2 # [ 60.270040] nix-grpc-daemon[874]: time=2026-09-21T09:14:12.865Z level=INFO msg="Uploading 1 narinfos"4409lb # [ 60.322676] niks3-server[1049]: 2026/09/21 09:14:12 INFO Signed narinfos id=12 count=14410lb # [ 60.331489] niks3-server[1049]: 2026/09/21 09:14:12 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4411lb # [ 60.347342] niks3-server[1049]: 2026/09/21 09:14:12 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4412worker2 # [ 60.301672] nix-grpc-daemon[874]: time=2026-09-21T09:14:12.898Z level=INFO msg="Upload complete. (348ms)"4413lb # [ 60.353303] niks3-server[1049]: 2026/09/21 09:14:12 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4414lb # [ 60.359922] niks3-server[1049]: 2026/09/21 09:14:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4415lb # [ 60.364813] niks3-server[1049]: 2026/09/21 09:14:12 INFO Completed upload id=124416lb # [ 60.367971] niks3-server[1049]: 2026/09/21 09:14:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4417worker2 # [ 60.315797] nix-grpc-daemon[777]: ts=2026-09-21T09:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14418worker1 # [ 60.303637] nix-grpc-daemon[776]: ts=2026-09-21T09:14:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:361964419worker2 # [ 60.333969] nix-daemon[862]: reaped child process 1096, status = succeeded4420worker1 # [ 60.321213] nix-grpc-daemon[776]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4421worker2 # [ 60.398689] nix-grpc-daemon[777]: ts=2026-09-21T09:14:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:354364422worker2 # [ 60.405643] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_out=1284423worker2 # [ 60.703263] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:354364424worker2 # [ 60.708665] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364425worker2 # [ 60.713099] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.308Z level=INFO msg="All 1 paths already cached"4426worker2 # [ 60.725374] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04427worker2 # [ 60.731695] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14428worker2 # [ 60.737100] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364429worker2 # [ 60.741903] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364430worker2 # [ 60.749801] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.346Z level=INFO msg="All 1 paths already cached"4431worker2 # [ 60.756592] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364432worker2 # [ 60.766875] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4433worker2 # [ 60.953409] nix-daemon[862]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4434worker2 # [ 60.972425] nix-daemon[862]: reaped child process 1125, status = succeeded4435worker2 # [ 61.146647] nix-daemon[862]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4436worker2 # [ 61.164845] nix-daemon[862]: reaped child process 1130, status = succeeded4437lb # [ 61.227533] niks3-server[1049]: 2026/09/21 09:14:13 INFO Received uploads request method=POST path=/api/pending_closures4438worker2 # [ 61.186753] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.782Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4439worker2 # [ 61.188357] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.782Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4440lb # [ 61.256850] niks3-server[1049]: 2026/09/21 09:14:13 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4441worker2 # [ 61.219029] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.815Z level=INFO msg="Uploading 1 narinfos"4442lb # [ 61.274972] niks3-server[1049]: 2026/09/21 09:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4443lb # [ 61.284215] niks3-server[1049]: 2026/09/21 09:14:13 INFO Signed narinfos id=13 count=14444lb # [ 61.288453] niks3-server[1049]: 2026/09/21 09:14:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4445lb # [ 61.295203] niks3-server[1049]: 2026/09/21 09:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4446worker2 # [ 61.249068] nix-grpc-daemon[874]: time=2026-09-21T09:14:13.844Z level=INFO msg="Upload complete. (470ms)"4447lb # [ 61.301317] niks3-server[1049]: 2026/09/21 09:14:13 INFO Completed upload id=134448lb # [ 61.305083] niks3-server[1049]: 2026/09/21 09:14:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4449worker2 # [ 61.256120] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=6244450worker2 # [ 61.267842] nix-grpc-daemon[777]: ts=2026-09-21T09:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364451worker2 # [ 61.276314] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4452worker2 # [ 61.279426] nix-daemon[862]: reaped child process 1120, status = succeeded4453worker2 # [ 61.293226] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4454worker2 # [ 62.739999] nix-daemon[862]: reaped child process 1134, status = succeeded4455worker2 # [ 62.918178] nix-daemon[862]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)4456worker2 # [ 62.936685] nix-daemon[862]: reaped child process 1152, status = succeeded4457lb # [ 62.999204] niks3-server[1049]: 2026/09/21 09:14:15 INFO Received uploads request method=POST path=/api/pending_closures4458worker2 # [ 62.960491] nix-grpc-daemon[874]: time=2026-09-21T09:14:15.554Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4459worker2 # [ 62.962344] nix-grpc-daemon[874]: time=2026-09-21T09:14:15.554Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4460lb # [ 63.038574] niks3-server[1049]: 2026/09/21 09:14:15 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4461lb # [ 63.054089] niks3-server[1049]: 2026/09/21 09:14:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4462worker2 # [ 63.009214] nix-grpc-daemon[874]: time=2026-09-21T09:14:15.605Z level=INFO msg="Uploading 1 narinfos"4463lb # [ 63.065340] niks3-server[1049]: 2026/09/21 09:14:15 INFO Signed narinfos id=14 count=14464lb # [ 63.070741] niks3-server[1049]: 2026/09/21 09:14:15 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4465lb # [ 63.075704] niks3-server[1049]: 2026/09/21 09:14:15 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4466lb # [ 63.082014] niks3-server[1049]: 2026/09/21 09:14:15 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4467lb # [ 63.086001] niks3-server[1049]: 2026/09/21 09:14:15 INFO Completed upload id=144468worker2 # [ 63.034769] nix-grpc-daemon[874]: time=2026-09-21T09:14:15.631Z level=INFO msg="Upload complete. (330ms)"4469lb # [ 63.088575] niks3-server[1049]: 2026/09/21 09:14:15 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4470worker2 # [ 63.049426] nix-grpc-daemon[777]: ts=2026-09-21T09:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14471worker2 # [ 63.067568] nix-grpc-daemon[777]: ts=2026-09-21T09:14:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:354364472worker2 # [ 63.072626] nix-daemon[862]: reaped child process 1132, status = succeeded4473worker1 # [ 63.053004] nix-grpc-daemon[776]: ts=2026-09-21T09:14:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:361964474worker1 # [ 63.066850] nix-grpc-daemon[776]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4475worker1 # [ 63.086576] nix-grpc-daemon[776]: ts=2026-09-21T09:14:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=1 nar_bytes_out=1764476worker1 # [ 63.401996] nix-grpc-daemon[776]: ts=2026-09-21T09:14:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:361964477worker2 # [ 63.427677] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364478worker2 # [ 63.437855] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.034Z level=INFO msg="All 2 paths already cached"4479worker2 # [ 63.455360] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04480worker2 # [ 63.463063] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14481worker2 # [ 63.471748] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364482worker2 # [ 63.484411] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364483worker2 # [ 63.498801] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.095Z level=INFO msg="All 2 paths already cached"4484worker2 # [ 63.505287] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364485worker2 # [ 63.520993] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4486worker2 # [ 63.726473] nix-daemon[862]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4487worker2 # [ 63.745263] nix-daemon[862]: reaped child process 1161, status = succeeded4488worker2 # [ 63.922238] nix-daemon[862]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4489worker2 # [ 63.940161] nix-daemon[862]: reaped child process 1166, status = succeeded4490lb # [ 64.002792] niks3-server[1049]: 2026/09/21 09:14:16 INFO Received uploads request method=POST path=/api/pending_closures4491worker2 # [ 63.964511] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.560Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4492worker2 # [ 63.966801] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.560Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4493lb # [ 64.038562] niks3-server[1049]: 2026/09/21 09:14:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4494worker2 # [ 63.995928] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.592Z level=INFO msg="Uploading 1 narinfos"4495lb # [ 64.050990] niks3-server[1049]: 2026/09/21 09:14:16 INFO Signed narinfos id=15 count=14496lb # [ 64.057560] niks3-server[1049]: 2026/09/21 09:14:16 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4497lb # [ 64.067529] niks3-server[1049]: 2026/09/21 09:14:16 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4498lb # [ 64.074698] niks3-server[1049]: 2026/09/21 09:14:16 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4499worker2 # [ 64.028856] nix-grpc-daemon[874]: time=2026-09-21T09:14:16.625Z level=INFO msg="Upload complete. (491ms)"4500lb # [ 64.082391] niks3-server[1049]: 2026/09/21 09:14:16 INFO Completed upload id=154501lb # [ 64.085844] niks3-server[1049]: 2026/09/21 09:14:16 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4502worker2 # [ 64.036146] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_in=7524503worker2 # [ 64.049485] nix-grpc-daemon[777]: ts=2026-09-21T09:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364504worker2 # [ 64.060662] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4505worker2 # [ 64.066073] nix-daemon[862]: reaped child process 1156, status = succeeded4506worker2 # [ 64.079766] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4507worker2 # [ 65.502262] nix-daemon[862]: reaped child process 1170, status = succeeded4508worker2 # [ 65.662267] nix-daemon[862]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)4509worker2 # [ 65.682494] nix-daemon[862]: reaped child process 1188, status = succeeded4510lb # [ 65.747676] niks3-server[1049]: 2026/09/21 09:14:18 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 65.708715] nix-grpc-daemon[874]: time=2026-09-21T09:14:18.302Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4512worker2 # [ 65.710234] nix-grpc-daemon[874]: time=2026-09-21T09:14:18.302Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4513lb # [ 65.787786] niks3-server[1049]: 2026/09/21 09:14:18 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4514worker2 # [ 65.748852] nix-grpc-daemon[874]: time=2026-09-21T09:14:18.345Z level=INFO msg="Uploading 1 narinfos"4515lb # [ 65.803680] niks3-server[1049]: 2026/09/21 09:14:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4516lb # [ 65.812078] niks3-server[1049]: 2026/09/21 09:14:18 INFO Signed narinfos id=16 count=14517lb # [ 65.814244] niks3-server[1049]: 2026/09/21 09:14:18 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4518lb # [ 65.823267] niks3-server[1049]: 2026/09/21 09:14:18 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4519lb # [ 65.826159] niks3-server[1049]: 2026/09/21 09:14:18 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4520lb # [ 65.829973] niks3-server[1049]: 2026/09/21 09:14:18 INFO Completed upload id=164521lb # [ 65.831874] niks3-server[1049]: 2026/09/21 09:14:18 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4522worker2 # [ 65.779046] nix-grpc-daemon[874]: time=2026-09-21T09:14:18.375Z level=INFO msg="Upload complete. (289ms)"4523worker2 # [ 65.787681] nix-grpc-daemon[777]: ts=2026-09-21T09:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14524worker2 # [ 65.803073] nix-grpc-daemon[777]: ts=2026-09-21T09:14:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:354364525worker2 # [ 65.806631] nix-daemon[862]: reaped child process 1168, status = succeeded4526worker2 # [ 65.822322] nix-grpc-daemon[777]: ts=2026-09-21T09:14:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:354364527worker2 # [ 65.827046] nix-grpc-daemon[777]: ts=2026-09-21T09:14:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_out=2324528client # [ 65.983986] nix-daemon[937]: reaped child process 939, status = succeeded4529client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.68 seconds)4530client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4531client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4532client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4533client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4534client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4535client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4536(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.81 seconds)4537subtest: an input only one worker has still reaches the builder4538client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4539client # this derivation will be built:4540client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4541client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4542client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.45 seconds)4543client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4544client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4545worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4546worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.26 seconds)4547worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4548worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4549client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24550client # [ 67.312715] nix-daemon[937]: accepted connection from pid 1019, user root (trusted)4551client # this derivation will be built:4552client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4553client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4554worker1 # [ 67.492964] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:361964555worker2 # [ 67.522090] nix-grpc-daemon[777]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364556client # copying 1 paths...4557client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4558worker1 # [ 67.512540] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361964559worker1 # [ 67.528483] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4560worker1 # [ 67.741107] nix-daemon[840]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4561worker1 # [ 67.759379] nix-daemon[840]: reaped child process 946, status = succeeded4562lb # [ 67.843662] niks3-server[1049]: 2026/09/21 09:14:20 INFO Received uploads request method=POST path=/api/pending_closures4563worker1 # [ 67.781348] nix-grpc-daemon[866]: time=2026-09-21T09:14:20.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4564worker1 # [ 67.783268] nix-grpc-daemon[866]: time=2026-09-21T09:14:20.370Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4565lb # [ 67.869901] niks3-server[1049]: 2026/09/21 09:14:20 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4566lb # [ 67.884973] niks3-server[1049]: 2026/09/21 09:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4567worker1 # [ 67.818780] nix-grpc-daemon[866]: time=2026-09-21T09:14:20.408Z level=INFO msg="Uploading 1 narinfos"4568lb # [ 67.896359] niks3-server[1049]: 2026/09/21 09:14:20 INFO Signed narinfos id=17 count=14569lb # [ 67.904433] niks3-server[1049]: 2026/09/21 09:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4570lb # [ 67.943958] niks3-server[1049]: 2026/09/21 09:14:20 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4571lb # [ 67.950482] niks3-server[1049]: 2026/09/21 09:14:20 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4572lb # [ 67.955932] niks3-server[1049]: 2026/09/21 09:14:20 INFO Completed upload id=174573worker1 # [ 67.884039] nix-grpc-daemon[866]: time=2026-09-21T09:14:20.474Z level=INFO msg="Upload complete. (340ms)"4574worker1 # [ 67.896166] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=1 nar_bytes_in=1204575worker2 # [ 67.928936] nix-grpc-daemon[777]: ts=2026-09-21T09:14:20Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04576worker1 # [ 67.913338] nix-daemon[840]: reaped child process 941, status = succeeded4577worker2 # [ 67.935828] nix-grpc-daemon[777]: ts=2026-09-21T09:14:20Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14578worker1 # [ 67.952930] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964579worker1 # [ 67.968801] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964580client # copying 2 paths to 'lb:50051'...4581client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4582worker1 # [ 67.987235] nix-grpc-daemon[776]: ts=2026-09-21T09:14:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361964583client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4584worker1 # [ 68.001634] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4585worker1 # [ 68.205749] nix-daemon[840]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4586worker1 # [ 68.222879] nix-daemon[840]: reaped child process 953, status = succeeded4587worker1 # [ 68.400341] nix-daemon[840]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4588worker1 # [ 68.416555] nix-daemon[840]: reaped child process 958, status = succeeded4589lb # [ 68.501039] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received uploads request method=POST path=/api/pending_closures4590lb # [ 68.510043] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received uploads request method=POST path=/api/pending_closures4591worker1 # [ 68.447159] nix-grpc-daemon[866]: time=2026-09-21T09:14:21.036Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4592worker1 # [ 68.448738] nix-grpc-daemon[866]: time=2026-09-21T09:14:21.036Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4593worker1 # [ 68.450438] nix-grpc-daemon[866]: time=2026-09-21T09:14:21.036Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4594lb # [ 68.545033] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4595lb # [ 68.560165] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4596lb # [ 68.573509] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4597lb # [ 68.579940] niks3-server[1049]: 2026/09/21 09:14:21 INFO Signed narinfos id=18 count=14598worker1 # [ 68.506810] nix-grpc-daemon[866]: time=2026-09-21T09:14:21.095Z level=INFO msg="Uploading 3 narinfos"4599lb # [ 68.584514] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4600lb # [ 68.587369] niks3-server[1049]: 2026/09/21 09:14:21 INFO Signed narinfos id=19 count=24601lb # [ 68.595061] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4602lb # [ 68.602783] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4603lb # [ 68.608199] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4604lb # [ 68.613326] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4605lb # [ 68.618788] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4606lb # [ 68.624813] niks3-server[1049]: 2026/09/21 09:14:21 INFO Completed upload id=184607lb # [ 68.627964] niks3-server[1049]: 2026/09/21 09:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4608lb # [ 68.631065] niks3-server[1049]: 2026/09/21 09:14:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4609lb # [ 68.633844] niks3-server[1049]: 2026/09/21 09:14:21 INFO Completed upload id=194610worker1 # [ 68.559094] nix-grpc-daemon[866]: time=2026-09-21T09:14:21.149Z level=INFO msg="Upload complete. (538ms)"4611worker1 # [ 68.570987] nix-grpc-daemon[776]: ts=2026-09-21T09:14:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=2 nar_bytes_in=9844612worker1 # [ 68.590217] nix-grpc-daemon[776]: ts=2026-09-21T09:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964613worker1 # [ 68.598471] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4614worker1 # [ 68.604457] nix-daemon[840]: reaped child process 948, status = succeeded4615client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4616client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4617worker1 # [ 68.620152] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4618lb # [ 69.907331] envoy[911]: conn peer=[2001:db8:1::1]:44172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=374619worker1 # [ 69.845476] nix-daemon[840]: reaped child process 962, status = succeeded4620worker1 # [ 70.020055] nix-daemon[840]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4621worker1 # [ 70.036619] nix-daemon[840]: reaped child process 980, status = succeeded4622lb # [ 69.913052] envoy[911]: conn peer=[2001:db8:1::1]:44158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3411 tx=1794 ms=24454623lb # [ 69.913260] envoy[911]: conn peer=[2001:db8:1::1]:44180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=1850 ms=24654624lb # [ 69.913369] envoy[911]: conn peer=[2001:db8:1::1]:33950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=984625lb # [ 69.913471] envoy[911]: conn peer=[2001:db8:1::1]:33952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4325 tx=2108 ms=24634626lb # [ 69.913596] envoy[911]: conn peer=[2001:db8:1::1]:33964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=724627lb # [ 70.123576] niks3-server[1049]: 2026/09/21 09:14:22 INFO Received uploads request method=POST path=/api/pending_closures4628worker1 # [ 70.083776] nix-grpc-daemon[866]: time=2026-09-21T09:14:22.672Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4629lb # [ 70.174513] niks3-server[1049]: 2026/09/21 09:14:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4630worker1 # [ 70.109600] nix-grpc-daemon[866]: time=2026-09-21T09:14:22.699Z level=INFO msg="Uploading 1 narinfos"4631lb # [ 70.188268] niks3-server[1049]: 2026/09/21 09:14:22 INFO Signed narinfos id=20 count=14632lb # [ 70.190931] niks3-server[1049]: 2026/09/21 09:14:22 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4633lb # [ 70.203278] niks3-server[1049]: 2026/09/21 09:14:22 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4634lb # [ 70.206795] niks3-server[1049]: 2026/09/21 09:14:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4635lb # [ 70.211003] niks3-server[1049]: 2026/09/21 09:14:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4636lb # [ 70.214694] niks3-server[1049]: 2026/09/21 09:14:22 INFO Completed upload id=204637worker1 # [ 70.142504] nix-grpc-daemon[866]: time=2026-09-21T09:14:22.732Z level=INFO msg="Upload complete. (309ms)"4638worker1 # [ 70.153149] nix-grpc-daemon[776]: ts=2026-09-21T09:14:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14639worker1 # [ 70.170033] nix-daemon[840]: reaped child process 960, status = succeeded4640client # copying 1 paths...4641worker2 # [ 70.219107] nix-grpc-daemon[777]: ts=2026-09-21T09:14:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:354364642worker2 # [ 70.236104] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4643client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4644worker2 # [ 70.286593] nix-grpc-daemon[777]: ts=2026-09-21T09:14:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:354364645worker2 # [ 70.291922] nix-grpc-daemon[777]: ts=2026-09-21T09:14:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_out=1204646client # [ 70.435790] nix-daemon[937]: reaped child process 1022, status = succeeded4647client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.37 seconds)4648client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24649client # [ 70.656088] nix-daemon[937]: accepted connection from pid 1041, user root (trusted)4650client # this derivation will be built:4651client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4652worker1 # [ 70.830405] nix-grpc-daemon[776]: ts=2026-09-21T09:14:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:361964653client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4654worker1 # [ 70.838251] nix-grpc-daemon[776]: ts=2026-09-21T09:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964655worker1 # [ 70.845017] nix-grpc-daemon[866]: time=2026-09-21T09:14:23.434Z level=INFO msg="All 1 paths already cached"4656worker2 # [ 70.878443] nix-grpc-daemon[777]: ts=2026-09-21T09:14:23Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04657worker2 # [ 70.886149] nix-grpc-daemon[777]: ts=2026-09-21T09:14:23Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14658worker2 # [ 70.894272] nix-grpc-daemon[777]: ts=2026-09-21T09:14:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364659worker2 # [ 70.909268] nix-grpc-daemon[777]: ts=2026-09-21T09:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354364660client # copying 1 paths to 'lb:50051'...4661client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4662worker2 # [ 70.930879] nix-grpc-daemon[777]: ts=2026-09-21T09:14:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354364663worker2 # [ 70.946063] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4664worker2 # [ 70.968099] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)4665worker2 # [ 71.034942] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)4666worker2 # [ 71.237626] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 255 ms (attempt 3/5)4667worker2 # [ 71.500383] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 4/5)4668worker2 # [ 71.548412] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4669worker2 # [ 71.559178] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4670worker2 # [ 71.794244] nix-daemon[862]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)4671worker2 # [ 71.813559] nix-daemon[862]: reaped child process 1207, status = succeeded4672worker2 # [ 71.987378] nix-daemon[862]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)4673worker2 # [ 72.006625] nix-daemon[862]: reaped child process 1212, status = succeeded4674lb # [ 72.069875] niks3-server[1049]: 2026/09/21 09:14:24 INFO Received uploads request method=POST path=/api/pending_closures4675worker2 # [ 72.031279] nix-grpc-daemon[874]: time=2026-09-21T09:14:24.627Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4676worker2 # [ 72.032912] nix-grpc-daemon[874]: time=2026-09-21T09:14:24.627Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4677lb # [ 72.099378] niks3-server[1049]: 2026/09/21 09:14:24 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4678worker2 # [ 72.062381] nix-grpc-daemon[874]: time=2026-09-21T09:14:24.658Z level=INFO msg="Uploading 1 narinfos"4679lb # [ 72.115620] niks3-server[1049]: 2026/09/21 09:14:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4680lb # [ 72.126485] niks3-server[1049]: 2026/09/21 09:14:24 INFO Signed narinfos id=21 count=14681worker2 # [ 72.085244] nix-grpc-daemon[874]: time=2026-09-21T09:14:24.681Z level=INFO msg="Upload complete. (494ms)"4682lb # [ 72.131646] niks3-server[1049]: 2026/09/21 09:14:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4683lb # [ 72.140761] niks3-server[1049]: 2026/09/21 09:14:24 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4684worker2 # [ 72.092124] nix-grpc-daemon[777]: ts=2026-09-21T09:14:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 paths=1 nar_bytes_in=5684685lb # [ 72.145801] niks3-server[1049]: 2026/09/21 09:14:24 INFO Completed upload id=214686lb # [ 72.148210] niks3-server[1049]: 2026/09/21 09:14:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4687worker2 # [ 72.100680] nix-grpc-daemon[777]: ts=2026-09-21T09:14:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354364688worker2 # [ 72.113829] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4689worker2 # [ 72.116441] nix-daemon[862]: reaped child process 1198, status = succeeded4690worker2 # [ 72.130349] nix-grpc-daemon[777]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4691worker2 # [ 72.144417] nix-daemon[862]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4692client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4693client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4694worker2 # [ 73.394551] nix-daemon[862]: reaped child process 1217, status = succeeded4695worker2 # [ 73.581580] nix-daemon[862]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)4696worker2 # [ 73.601030] nix-daemon[862]: reaped child process 1235, status = succeeded4697lb # [ 73.663774] niks3-server[1049]: 2026/09/21 09:14:26 INFO Received uploads request method=POST path=/api/pending_closures4698worker2 # [ 73.624940] nix-grpc-daemon[874]: time=2026-09-21T09:14:26.220Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4699lb # [ 73.697202] niks3-server[1049]: 2026/09/21 09:14:26 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4700worker2 # [ 73.657339] nix-grpc-daemon[874]: time=2026-09-21T09:14:26.253Z level=INFO msg="Uploading 1 narinfos"4701lb # [ 73.710209] niks3-server[1049]: 2026/09/21 09:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4702lb # [ 73.719760] niks3-server[1049]: 2026/09/21 09:14:26 INFO Signed narinfos id=22 count=14703lb # [ 73.723463] niks3-server[1049]: 2026/09/21 09:14:26 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4704lb # [ 73.731325] niks3-server[1049]: 2026/09/21 09:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4705lb # [ 73.738510] niks3-server[1049]: 2026/09/21 09:14:26 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4706lb # [ 73.742523] niks3-server[1049]: 2026/09/21 09:14:26 INFO Completed upload id=224707worker2 # [ 73.691548] nix-grpc-daemon[874]: time=2026-09-21T09:14:26.287Z level=INFO msg="Upload complete. (333ms)"4708worker2 # [ 73.705108] nix-grpc-daemon[777]: ts=2026-09-21T09:14:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14709worker2 # [ 73.718601] nix-daemon[862]: reaped child process 1214, status = succeeded4710client # copying 1 paths...4711worker1 # [ 73.726911] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:361964712worker1 # [ 73.741382] nix-grpc-daemon[776]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4713client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4714worker1 # [ 73.790409] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:361964715worker1 # [ 73.795681] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=1 nar_bytes_out=1204716client # [ 73.961513] nix-daemon[937]: reaped child process 1044, status = succeeded4717client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.52 seconds)4718client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24719client # [ 74.180884] nix-daemon[937]: accepted connection from pid 1063, user root (trusted)4720client # this derivation will be built:4721client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4722worker2 # [ 74.373118] nix-grpc-daemon[777]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:354364723client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4724worker1 # [ 74.357308] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964725worker1 # [ 74.370593] nix-grpc-daemon[866]: time=2026-09-21T09:14:26.959Z level=INFO msg="All 1 paths already cached"4726worker2 # [ 74.409717] nix-grpc-daemon[777]: ts=2026-09-21T09:14:27Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04727worker1 # [ 74.392510] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964728worker2 # [ 74.416145] nix-grpc-daemon[777]: ts=2026-09-21T09:14:27Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14729worker1 # [ 74.407705] nix-grpc-daemon[776]: ts=2026-09-21T09:14:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361964730worker1 # [ 74.422316] nix-grpc-daemon[866]: time=2026-09-21T09:14:27.012Z level=INFO msg="All 1 paths already cached"4731client # copying 1 paths to 'lb:50051'...4732client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4733worker1 # [ 74.432002] nix-grpc-daemon[776]: ts=2026-09-21T09:14:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361964734worker1 # [ 74.447252] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4735worker1 # [ 74.651912] nix-daemon[840]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4736worker1 # [ 74.672327] nix-daemon[840]: reaped child process 988, status = succeeded4737worker1 # [ 74.847596] nix-daemon[840]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4738worker1 # [ 74.863573] nix-daemon[840]: reaped child process 993, status = succeeded4739lb # [ 74.953115] niks3-server[1049]: 2026/09/21 09:14:27 INFO Received uploads request method=POST path=/api/pending_closures4740worker1 # [ 74.891461] nix-grpc-daemon[866]: time=2026-09-21T09:14:27.481Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4741worker1 # [ 74.893033] nix-grpc-daemon[866]: time=2026-09-21T09:14:27.481Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4742lb # [ 74.992127] niks3-server[1049]: 2026/09/21 09:14:27 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4743worker1 # [ 74.931575] nix-grpc-daemon[866]: time=2026-09-21T09:14:27.521Z level=INFO msg="Uploading 1 narinfos"4744lb # [ 75.007038] niks3-server[1049]: 2026/09/21 09:14:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4745lb # [ 75.015293] niks3-server[1049]: 2026/09/21 09:14:27 INFO Signed narinfos id=23 count=14746lb # [ 75.023449] niks3-server[1049]: 2026/09/21 09:14:27 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4747lb # [ 75.027819] niks3-server[1049]: 2026/09/21 09:14:27 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4748lb # [ 75.032935] niks3-server[1049]: 2026/09/21 09:14:27 INFO Completed upload id=234749worker1 # [ 74.959995] nix-grpc-daemon[866]: time=2026-09-21T09:14:27.550Z level=INFO msg="Upload complete. (495ms)"4750lb # [ 75.035403] niks3-server[1049]: 2026/09/21 09:14:27 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4751worker1 # [ 74.968775] nix-grpc-daemon[776]: ts=2026-09-21T09:14:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=0 paths=1 nar_bytes_in=5684752worker1 # [ 74.982190] nix-grpc-daemon[776]: ts=2026-09-21T09:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361964753worker1 # [ 74.989101] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4754worker1 # [ 74.994373] nix-daemon[840]: reaped child process 983, status = succeeded4755client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4756worker1 # [ 75.006052] nix-daemon[840]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4757client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4758worker1 # [ 76.054293] nix-daemon[840]: reaped child process 997, status = succeeded4759worker1 # [ 76.238366] nix-daemon[840]: accepted connection from pid 1012, user nix-grpc-daemon (trusted)4760worker1 # [ 76.255669] nix-daemon[840]: reaped child process 1015, status = succeeded4761lb # [ 76.341838] niks3-server[1049]: 2026/09/21 09:14:28 INFO Received uploads request method=POST path=/api/pending_closures4762worker1 # [ 76.280070] nix-grpc-daemon[866]: time=2026-09-21T09:14:28.868Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4763lb # [ 76.370724] niks3-server[1049]: 2026/09/21 09:14:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4764worker1 # [ 76.315893] nix-grpc-daemon[866]: time=2026-09-21T09:14:28.905Z level=INFO msg="Uploading 1 narinfos"4765lb # [ 76.380891] niks3-server[1049]: 2026/09/21 09:14:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4766lb # [ 76.391803] niks3-server[1049]: 2026/09/21 09:14:28 INFO Signed narinfos id=24 count=14767lb # [ 76.398063] niks3-server[1049]: 2026/09/21 09:14:28 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4768lb # [ 76.406736] niks3-server[1049]: 2026/09/21 09:14:28 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4769lb # [ 76.413902] niks3-server[1049]: 2026/09/21 09:14:29 INFO Completed upload id=244770worker1 # [ 76.341858] nix-grpc-daemon[866]: time=2026-09-21T09:14:28.932Z level=INFO msg="Upload complete. (323ms)"4771lb # [ 76.416807] niks3-server[1049]: 2026/09/21 09:14:29 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4772worker1 # [ 76.352978] nix-grpc-daemon[776]: ts=2026-09-21T09:14:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36196 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14773worker1 # [ 76.365950] nix-daemon[840]: reaped child process 995, status = succeeded4774client # copying 1 paths...4775worker2 # [ 76.423385] nix-grpc-daemon[777]: ts=2026-09-21T09:14:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:354364776worker2 # [ 76.435977] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4777client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4778worker2 # [ 76.481721] nix-grpc-daemon[777]: ts=2026-09-21T09:14:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:354364779worker2 # [ 76.487038] nix-grpc-daemon[777]: ts=2026-09-21T09:14:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35436 duration_s=0 paths=1 nar_bytes_out=1204780client # [ 76.632458] nix-daemon[937]: reaped child process 1066, status = succeeded4781client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.67 seconds)4782(finished: subtest: an input only one worker has still reaches the builder, in 10.51 seconds)4783subtest: upload whose reference the worker lacks is completed from the cache4784client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4785client # this derivation will be built:4786client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4787client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4788client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)4789client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4790client # this derivation will be built:4791client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4792client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4793client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4794client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4795client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4796worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4797worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4798worker2 # [ 78.025949] systemd[1]: Stopping Nix worker-protocol over gRPC...4799worker2 # [ 78.435620] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=signal_exit4800worker2 # [ 78.671671] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_yield4801worker1 # [ 78.655183] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_restarting addr=lb:500514802worker2 # [ 78.678380] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:354344803worker2 # [ 78.683069] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=worker_gone addr=192.168.1.3:500514804lb # [ 78.727068] niks3-server[1049]: 2026/09/21 09:14:31 INFO lead: released remote=192.168.1.4:529844805worker2 # [ 78.689803] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_restarting addr=lb:500514806worker2 # [ 78.695869] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:354344807worker2 # [ 78.700478] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=worker_gone addr=192.168.1.4:500514808worker1 # [ 78.855337] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514809worker1 # [ 78.860771] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4810worker2 # [ 78.891526] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514811worker2 # [ 78.896743] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4812worker1 # [ 79.061632] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514813worker1 # [ 79.066851] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4814worker2 # [ 79.097993] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514815worker2 # [ 79.103035] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4816worker1 # [ 79.267474] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514817worker1 # [ 79.272734] nix-grpc-daemon[776]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4818worker2 # [ 79.304181] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_connected addr=lb:500514819worker2 # [ 79.309138] nix-grpc-daemon[777]: ts=2026-09-21T09:14:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4820worker1 # [ 79.473438] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514821worker1 # [ 79.478717] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4822worker2 # [ 79.511979] nix-grpc-daemon[777]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514823worker2 # [ 79.516122] nix-grpc-daemon[777]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4824worker2 # [ 79.677146] nix-grpc-daemon[777]: ts=2026-09-21T09:14:32Z level=info event=scheduler_restarting4825worker1 # [ 79.681422] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514826worker1 # [ 79.685563] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4827worker2 # [ 79.710312] nix-daemon[862]: reaped child process 864, status = succeeded4828worker2 # [ 79.739044] nix-grpc-daemon[777]: ts=2026-09-21T09:14:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4829lb # [ 79.906742] envoy[911]: conn peer=[2001:db8:1::1]:33966 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4262 tx=1881 ms=28324830lb # [ 79.907119] envoy[911]: conn peer=[2001:db8:1::1]:33968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=644831lb # [ 79.907249] envoy[911]: conn peer=[2001:db8:1::1]:45290 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=1919 ms=30184832lb # [ 79.907368] envoy[911]: conn peer=[2001:db8:1::1]:45298 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=644833lb # [ 79.907467] envoy[911]: conn peer=[2001:db8:1::1]:45304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=1916 ms=21694834lb # [ 79.907612] envoy[911]: conn peer=[2001:db8:1::1]:45314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=654835lb # [ 79.907731] envoy[911]: conn peer=[2001:db8:1::4]:51212 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53665 tx=3109 ms=592444836worker1 # [ 79.882231] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514837worker1 # [ 79.887352] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4838worker1 # [ 80.088124] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514839worker1 # [ 80.092912] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4840worker1 # [ 80.293840] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_connected addr=lb:500514841worker1 # [ 80.299470] nix-grpc-daemon[776]: ts=2026-09-21T09:14:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4842worker1 # [ 80.500194] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_connected addr=lb:500514843worker1 # [ 80.505478] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4844worker2 # [ 80.557469] nix-grpc-daemon[777]: E0000 00:00:1789982073.152397 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4845worker2 # [ 80.564236] nix-grpc-daemon[777]: E0000 00:00:1789982073.152761 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4846worker1 # [ 80.708293] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_connected addr=lb:500514847worker1 # [ 80.712492] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4848worker1 # [ 80.909253] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_connected addr=lb:500514849worker1 # [ 80.914494] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4850worker1 # [ 81.115393] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_connected addr=lb:500514851worker1 # [ 81.120760] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4852worker1 # [ 81.321647] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_connected addr=lb:500514853worker1 # [ 81.326977] nix-grpc-daemon[776]: ts=2026-09-21T09:14:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4854worker2 # [ 81.369834] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4855worker2 # [ 81.376741] systemd[1]: Stopped Nix worker-protocol over gRPC.4856worker2 # [ 81.379053] systemd[1]: nix-grpc-daemon.service: Consumed 3.655s CPU time over 1min 1.315s wall clock time, 65.5M memory peak, 95.1K incoming IP traffic, 242.8K outgoing IP traffic.4857worker2 # [ 81.390532] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4858worker2 # [ 81.395764] systemd[1]: Closed Nix worker-protocol over gRPC.4859worker2 # [ 81.398918] systemd[1]: nix-grpc-daemon.socket: Consumed 567us CPU time over 1min 9.967s wall clock time, 256K memory peak, 94.8K incoming IP traffic, 75.5K outgoing IP traffic.4860lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4861lb: must succeed: curl -sf localhost:9901/clusters4862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4863lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4864client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24865worker1 # [ 81.527728] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_connected addr=lb:500514866worker1 # [ 81.533274] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4867worker1 # [ 81.737931] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_connected addr=lb:500514868client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4869worker1 # [ 81.744265] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:361964870worker1 # [ 81.749629] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4871worker1 # [ 81.754570] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:361964872client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4873worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4874worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4875worker2 # [ 81.906045] systemd[1]: Listening on Nix worker-protocol over gRPC.4876worker1 # [ 81.937341] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_connected addr=lb:500514877worker1 # [ 81.946305] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4878worker1 # [ 81.980654] systemd[1]: Stopping Nix worker-protocol over gRPC...4879worker1 # [ 82.145813] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_connected addr=lb:500514880worker1 # [ 82.151110] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker1 # [ 82.258818] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=signal_exit4882worker2 # [ 82.354242] systemd[1]: Starting scheduler-token.service...4883worker1 # [ 82.351917] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_connected addr=lb:500514884worker1 # [ 82.357041] nix-grpc-daemon[776]: ts=2026-09-21T09:14:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4885worker2 # [ 82.462358] systemd[1]: scheduler-token.service: Deactivated successfully.4886worker2 # [ 82.471643] systemd[1]: Finished scheduler-token.service.4887worker2 # [ 82.474473] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 112ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4888worker2 # [ 82.482158] systemd[1]: Starting Nix worker-protocol over gRPC...4889worker1 # [ 82.557922] nix-grpc-daemon[776]: ts=2026-09-21T09:14:35Z level=info event=scheduler_connected addr=lb:500514890worker1 # [ 82.563205] nix-grpc-daemon[776]: ts=2026-09-21T09:14:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4891worker2 # [ 82.673389] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4892worker2 # [ 82.676317] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4893worker2 # [ 82.680365] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14894worker2 # [ 82.687704] nix-grpc-daemon[1262]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4895worker2 # [ 82.700076] nix-grpc-daemon[1262]: E0000 00:00:1789982075.283518 1262 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4896worker2 # [ 82.705690] nix-grpc-daemon[1262]: E0000 00:00:1789982075.287327 1262 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4897lb # [ 82.757290] niks3-server[1049]: 2026/09/21 09:14:35 INFO lead: acquired remote=192.168.1.4:442084898worker2 # [ 82.708866] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4899worker2 # [ 82.710997] systemd[1]: Started Nix worker-protocol over gRPC.4900worker2 # [ 82.711925] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=scheduler_take_over4901worker2 # [ 82.720206] nix-grpc-daemon[1262]: E0000 00:00:1789982075.312483 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4902worker2 # [ 82.747290] nix-grpc-daemon[1262]: E0000 00:00:1789982075.330292 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4903worker2 # [ 82.751929] nix-grpc-daemon[1262]: E0000 00:00:1789982075.330514 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4904worker2 # [ 82.755057] nix-grpc-daemon[1262]: E0000 00:00:1789982075.347530 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4905worker2 # [ 82.764334] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=scheduler_connected addr=lb:500514906worker2 # [ 82.773815] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=oidc_jwks provider=mock keys=14907worker2 # [ 82.775200] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:344704908worker2 # [ 82.777231] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04909worker2 # [ 82.786520] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:344704910worker2 # [ 82.788232] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04911worker1 # [ 82.765247] nix-grpc-daemon[776]: ts=2026-09-21T09:14:35Z level=info event=scheduler_connected addr=lb:500514912worker1 # [ 83.519983] nix-grpc-daemon[776]: ts=2026-09-21T09:14:36Z level=info event=scheduler_restarting4913worker2 # [ 83.552553] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344704914worker2 # [ 83.552924] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:36Z level=info event=worker_gone addr=192.168.1.3:500514915worker1 # [ 83.556448] nix-grpc-daemon[776]: ts=2026-09-21T09:14:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4916worker1 # [ 83.575799] nix-daemon[840]: reaped child process 843, status = succeeded4917worker1 # [ 84.377952] nix-grpc-daemon[776]: E0000 00:00:1789982076.966734 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4918worker1 # [ 84.405808] nix-grpc-daemon[776]: E0000 00:00:1789982076.995807 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4919worker1 # [ 85.269467] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4920worker1 # [ 85.273071] systemd[1]: Stopped Nix worker-protocol over gRPC.4921worker1 # [ 85.275380] systemd[1]: nix-grpc-daemon.service: Consumed 2.026s CPU time over 1min 5.415s wall clock time, 51.6M memory peak, 56.6K incoming IP traffic, 103.7K outgoing IP traffic.4922worker1 # [ 85.290729] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4923worker1 # [ 85.293840] systemd[1]: Closed Nix worker-protocol over gRPC.4924worker1 # [ 85.295494] systemd[1]: nix-grpc-daemon.socket: Consumed 578us CPU time over 1min 13.740s wall clock time, 256K memory peak, 62.3K incoming IP traffic, 55.5K outgoing IP traffic.4925lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4926lb: must succeed: curl -sf localhost:9901/clusters4927lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4928lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4929client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24930worker2 # [ 85.679413] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344824931worker2 # [ 85.695125] nix-daemon[862]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)4932worker2 # [ 85.720764] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)4933worker2 # [ 85.811344] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)4934worker2 # [ 85.948802] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 214 ms (attempt 3/5)4935worker2 # [ 86.170223] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 4/5)4936worker2 # [ 86.186982] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4937client # copying 2 paths...4938client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4939worker2 # [ 86.212212] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344824940client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4941worker2 # [ 86.230677] nix-daemon[862]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)4942worker2 # [ 86.260254] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:38Z level=info event=niks3_push_started pid=13214943worker2 # [ 86.478134] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.073Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4944worker2 # [ 86.653865] nix-daemon[862]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)4945worker2 # [ 86.671087] nix-daemon[862]: reaped child process 1337, status = succeeded4946lb # [ 86.737826] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received uploads request method=POST path=/api/pending_closures4947lb # [ 86.748486] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received uploads request method=POST path=/api/pending_closures4948worker2 # [ 86.708069] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.303Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4949worker2 # [ 86.711155] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.303Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4950worker2 # [ 86.713019] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.303Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4951lb # [ 86.799638] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4952lb # [ 86.814564] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4953lb # [ 86.824240] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4954lb # [ 86.831501] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4955worker2 # [ 86.788285] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.383Z level=INFO msg="Uploading 3 narinfos"4956lb # [ 86.841136] niks3-server[1049]: 2026/09/21 09:14:39 INFO Signed narinfos id=25 count=14957lb # [ 86.843876] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4958lb # [ 86.855580] niks3-server[1049]: 2026/09/21 09:14:39 INFO Signed narinfos id=26 count=24959lb # [ 86.861018] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4960lb # [ 86.870699] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4961lb # [ 86.876867] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4962lb # [ 86.885139] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4963lb # [ 86.890371] niks3-server[1049]: 2026/09/21 09:14:39 INFO Completed upload id=254964lb # [ 86.893402] niks3-server[1049]: 2026/09/21 09:14:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4965lb # [ 86.896321] niks3-server[1049]: 2026/09/21 09:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4966worker2 # [ 86.845662] nix-grpc-daemon[1321]: time=2026-09-21T09:14:39.441Z level=INFO msg="Upload complete. (363ms)"4967lb # [ 86.898734] niks3-server[1049]: 2026/09/21 09:14:39 INFO Completed upload id=264968worker2 # [ 86.857285] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34482 duration_s=0 paths=2 nar_bytes_in=3124969worker2 # [ 86.874479] nix-daemon[862]: reaped child process 1319, status = succeeded4970client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.48 seconds)4971worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4972worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4973worker1 # [ 86.995182] systemd[1]: Listening on Nix worker-protocol over gRPC.4974worker1 # [ 87.005591] systemd[1]: Starting Nix worker-protocol over gRPC...4975worker1 # [ 87.213264] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4976worker1 # [ 87.215914] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4977worker1 # [ 87.220315] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14978worker1 # [ 87.228077] nix-grpc-daemon[1038]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4979worker1 # [ 87.229540] nix-grpc-daemon[1038]: E0000 00:00:1789982079.816879 1038 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4980worker1 # [ 87.239972] nix-grpc-daemon[1038]: E0000 00:00:1789982079.823367 1038 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4981lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4982lb: must succeed: curl -sf localhost:9901/clusters4983worker1 # [ 87.281528] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4984worker1 # [ 87.283304] systemd[1]: Started Nix worker-protocol over gRPC.4985worker1 # [ 87.286855] nix-grpc-daemon[1038]: E0000 00:00:1789982079.838827 1077 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4986worker1 # [ 87.289834] nix-grpc-daemon[1038]: E0000 00:00:1789982079.858870 1077 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4987worker1 # [ 87.292314] nix-grpc-daemon[1038]: E0000 00:00:1789982079.859029 1077 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4988worker1 # [ 87.294668] nix-grpc-daemon[1038]: E0000 00:00:1789982079.876190 1077 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4989worker1 # [ 87.303294] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:39Z level=info event=scheduler_connected addr=lb:500514990worker2 # [ 87.326286] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:344704991worker2 # [ 87.331837] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04992lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4993lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4994(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.75 seconds)4995subtest: scheduler down: the other node takes the lock and keeps it4996lb: must succeed: curl -sf localhost:9901/clusters4997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4998lb: must succeed: curl -sf localhost:9901/clusters4999lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5000lb: must succeed: curl -sf localhost:9901/clusters5001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5002lb: must succeed: curl -sf localhost:9901/clusters5003lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5004worker2 # [ 87.624300] systemd[1]: Stopping Nix worker-protocol over gRPC...5005worker2 # [ 87.700520] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=signal_exit5006worker2 # [ 87.706644] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_yield5007lb # [ 87.761247] niks3-server[1049]: 2026/09/21 09:14:40 INFO lead: released remote=192.168.1.4:442085008worker1 # [ 87.691110] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_restarting addr=lb:500515009worker2 # [ 87.716197] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344705010worker2 # [ 87.722856] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=worker_gone addr=192.168.1.3:500515011worker2 # [ 87.727023] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_restarting addr=lb:500515012worker2 # [ 87.731106] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344705013worker2 # [ 87.737635] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=worker_gone addr=192.168.1.4:500515014worker1 # [ 87.892433] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515015worker1 # [ 87.897537] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker2 # [ 87.925331] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515017worker2 # [ 87.929391] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5018worker1 # [ 88.098442] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515019worker1 # [ 88.103715] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5020worker2 # [ 88.126000] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515021worker2 # [ 88.131195] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5022worker1 # [ 88.306326] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515023worker2 # [ 88.332261] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_connected addr=lb:500515024worker1 # [ 88.310329] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5025worker2 # [ 88.337120] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker1 # [ 88.506833] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515027worker1 # [ 88.511701] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5028worker2 # [ 88.538174] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515029worker2 # [ 88.542895] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5030worker2 # [ 88.711583] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:41Z level=info event=scheduler_restarting5031worker1 # [ 88.712517] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515032worker1 # [ 88.717572] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5033worker2 # [ 88.741150] nix-daemon[862]: reaped child process 1314, status = succeeded5034worker2 # [ 88.765637] nix-grpc-daemon[1262]: E0000 00:00:1789982081.344847 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5035worker2 # [ 88.772397] nix-grpc-daemon[1262]: E0000 00:00:1789982081.344993 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5036worker2 # [ 88.778854] nix-grpc-daemon[1262]: ts=2026-09-21T09:14:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5037worker1 # [ 88.918454] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515038worker1 # [ 88.923384] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5039worker1 # [ 89.124306] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515040worker1 # [ 89.129414] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker1 # [ 89.332093] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_connected addr=lb:500515042worker1 # [ 89.336221] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker1 # [ 89.532477] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_connected addr=lb:500515044worker1 # [ 89.537715] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker1 # [ 89.738483] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_connected addr=lb:500515046worker1 # [ 89.743705] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047lb # [ 89.911328] envoy[911]: conn peer=[2001:db8:1::1]:37178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225048lb # [ 89.916229] envoy[911]: conn peer=[2001:db8:1::3]:45716 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12469 tx=3249 ms=632235049lb # [ 89.921206] envoy[911]: conn peer=[2001:db8:1::1]:37184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=305 ms=11975050lb # [ 89.928599] envoy[911]: conn peer=[2001:db8:1::4]:52254 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4158 tx=349 ms=59915051worker1 # [ 89.944582] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_connected addr=lb:500515052worker1 # [ 89.949687] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5053worker1 # [ 90.150505] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_connected addr=lb:500515054worker1 # [ 90.155627] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5055worker1 # [ 90.358281] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_connected addr=lb:500515056worker1 # [ 90.362513] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5057worker1 # [ 90.559054] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_connected addr=lb:500515058worker1 # [ 90.564422] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5059worker2 # [ 90.733130] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5060worker2 # [ 90.736986] systemd[1]: Stopped Nix worker-protocol over gRPC.5061worker2 # [ 90.739355] systemd[1]: nix-grpc-daemon.service: Consumed 475ms CPU time over 8.248s wall clock time, 33.5M memory peak, 18.8K incoming IP traffic, 25.1K outgoing IP traffic.5062worker2 # [ 90.756804] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5063worker2 # [ 90.758986] systemd[1]: Closed Nix worker-protocol over gRPC.5064worker2 # [ 90.763794] systemd[1]: nix-grpc-daemon.socket: Consumed 940us CPU time over 8.862s wall clock time, 256K memory peak, 19.3K incoming IP traffic, 13.8K outgoing IP traffic.5065worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5066worker1 # [ 90.765818] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_connected addr=lb:500515067worker1 # [ 90.772659] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5068worker1 # [ 90.972556] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_connected addr=lb:500515069worker1 # [ 90.977757] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5070worker1 # [ 91.178481] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_connected addr=lb:500515071worker1 # [ 91.183660] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5072worker1 # [ 91.386444] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_connected addr=lb:500515073worker1 # [ 91.390641] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5074worker1 # [ 91.587020] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_connected addr=lb:500515075worker1 # [ 91.592270] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5076worker1 # [ 91.792964] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_connected addr=lb:500515077worker1 # [ 91.797831] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5078worker1 # [ 91.998951] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_connected addr=lb:500515079worker1 # [ 92.004687] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5080worker1 # [ 92.205304] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_connected addr=lb:500515081worker1 # [ 92.210750] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5082worker1 # [ 92.242223] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:44Z level=info event=scheduler_take_over5083lb # [ 92.315426] niks3-server[1049]: 2026/09/21 09:14:44 INFO lead: acquired remote=192.168.1.3:506365084worker1 # [ 92.412889] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:45Z level=info event=scheduler_connected addr=lb:500515085worker1 # [ 92.423301] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:577485086worker1 # [ 92.428754] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05087worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 2.27 seconds)5088worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5089worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5090client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25091worker1 # [ 93.448328] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577625092worker1 # [ 93.465705] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5093client # don't know how to build these paths:5094client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5095worker1 # [ 93.495656] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577625096worker1 # [ 93.515333] nix-grpc-daemon[1038]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5097worker1 # [ 93.557985] nix-grpc-daemon[1038]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5098worker1 # [ 93.612075] nix-grpc-daemon[1038]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 277 ms (attempt 3/5)5099worker1 # [ 93.896901] nix-grpc-daemon[1038]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 376 ms (attempt 4/5)5100worker1 # [ 94.274733] nix-grpc-daemon[1038]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5101worker1 # [ 94.310878] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05102worker1 # [ 94.318287] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15103worker1 # [ 94.357369] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625104worker1 # [ 94.374175] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577625105client # copying 1 paths to 'lb:50051'...5106client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5107worker1 # [ 94.388780] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577625108worker1 # [ 94.405466] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5109worker1 # [ 94.432833] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:47Z level=info event=niks3_push_started pid=11205110worker1 # [ 94.657548] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.246Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5111worker1 # [ 94.835574] nix-daemon[840]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5112worker1 # [ 94.852760] nix-daemon[840]: reaped child process 1136, status = succeeded5113worker1 # [ 95.032566] nix-daemon[840]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5114worker1 # [ 95.049222] nix-daemon[840]: reaped child process 1141, status = succeeded5115lb # [ 95.134128] niks3-server[1049]: 2026/09/21 09:14:47 INFO Received uploads request method=POST path=/api/pending_closures5116worker1 # [ 95.074390] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5117worker1 # [ 95.075880] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.663Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5118lb # [ 95.175480] niks3-server[1049]: 2026/09/21 09:14:47 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5119lb # [ 95.189125] niks3-server[1049]: 2026/09/21 09:14:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5120worker1 # [ 95.123788] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.713Z level=INFO msg="Uploading 1 narinfos"5121lb # [ 95.202394] niks3-server[1049]: 2026/09/21 09:14:47 INFO Signed narinfos id=27 count=15122lb # [ 95.211302] niks3-server[1049]: 2026/09/21 09:14:47 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5123lb # [ 95.222047] niks3-server[1049]: 2026/09/21 09:14:47 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5124worker1 # [ 95.157020] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.747Z level=INFO msg="Upload complete. (494ms)"5125lb # [ 95.231762] niks3-server[1049]: 2026/09/21 09:14:47 INFO Completed upload id=275126worker1 # [ 95.166945] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=0 paths=1 nar_bytes_in=5205127worker1 # [ 95.181167] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625128worker1 # [ 95.184779] nix-grpc-daemon[1120]: time=2026-09-21T09:14:47.763Z level=WARN msg="Failed to register uploaded object" key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo error="server returned 404: object is not pending upload\n"5129worker1 # [ 95.194969] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5130worker1 # [ 95.201130] nix-daemon[840]: reaped child process 1118, status = succeeded5131client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5132client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5133worker1 # [ 95.208793] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5134client # farm-a> LOG-farm-a-on-standby5135worker1 # [ 96.852083] nix-daemon[840]: reaped child process 1145, status = succeeded5136worker1 # [ 97.039771] nix-daemon[840]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5137worker1 # [ 97.055793] nix-daemon[840]: reaped child process 1163, status = succeeded5138lb # [ 97.143589] niks3-server[1049]: 2026/09/21 09:14:49 INFO Received uploads request method=POST path=/api/pending_closures5139worker1 # [ 97.088488] nix-grpc-daemon[1120]: time=2026-09-21T09:14:49.676Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5140worker1 # [ 97.090107] nix-grpc-daemon[1120]: time=2026-09-21T09:14:49.676Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5141lb # [ 97.206310] niks3-server[1049]: 2026/09/21 09:14:49 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5142worker1 # [ 97.146102] nix-grpc-daemon[1120]: time=2026-09-21T09:14:49.734Z level=INFO msg="Uploading 1 narinfos"5143lb # [ 97.219589] niks3-server[1049]: 2026/09/21 09:14:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5144lb # [ 97.233206] niks3-server[1049]: 2026/09/21 09:14:49 INFO Signed narinfos id=28 count=15145lb # [ 97.235771] niks3-server[1049]: 2026/09/21 09:14:49 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5146lb # [ 97.258550] niks3-server[1049]: 2026/09/21 09:14:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5147lb # [ 97.261400] niks3-server[1049]: 2026/09/21 09:14:49 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5148lb # [ 97.264901] niks3-server[1049]: 2026/09/21 09:14:49 INFO Completed upload id=285149worker1 # [ 97.192477] nix-grpc-daemon[1120]: time=2026-09-21T09:14:49.782Z level=INFO msg="Upload complete. (372ms)"5150lb # [ 97.272423] niks3-server[1049]: 2026/09/21 09:14:49 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5151worker1 # [ 97.217118] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=2 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15152worker1 # [ 97.235088] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05153worker1 # [ 97.239389] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15154worker1 # [ 97.243638] nix-daemon[840]: reaped child process 1143, status = succeeded5155worker1 # [ 97.260764] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625156worker1 # [ 97.275809] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577625157worker1 # [ 97.292907] nix-grpc-daemon[1120]: time=2026-09-21T09:14:49.883Z level=INFO msg="All 1 paths already cached"5158client # copying 1 paths to 'lb:50051'...5159worker1 # [ 97.301561] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577625160client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5161worker1 # [ 97.320583] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5162worker1 # [ 97.528738] nix-daemon[840]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5163worker1 # [ 97.546061] nix-daemon[840]: reaped child process 1172, status = succeeded5164worker1 # [ 97.725636] nix-daemon[840]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5165worker1 # [ 97.742149] nix-daemon[840]: reaped child process 1177, status = succeeded5166lb # [ 97.825753] niks3-server[1049]: 2026/09/21 09:14:50 INFO Received uploads request method=POST path=/api/pending_closures5167worker1 # [ 97.767998] nix-grpc-daemon[1120]: time=2026-09-21T09:14:50.357Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5168worker1 # [ 97.769507] nix-grpc-daemon[1120]: time=2026-09-21T09:14:50.357Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5169lb # [ 97.859614] niks3-server[1049]: 2026/09/21 09:14:50 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5170lb # [ 97.872749] niks3-server[1049]: 2026/09/21 09:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5171worker1 # [ 97.807734] nix-grpc-daemon[1120]: time=2026-09-21T09:14:50.397Z level=INFO msg="Uploading 1 narinfos"5172lb # [ 97.884346] niks3-server[1049]: 2026/09/21 09:14:50 INFO Signed narinfos id=29 count=15173lb # [ 97.890079] niks3-server[1049]: 2026/09/21 09:14:50 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5174lb # [ 97.896810] niks3-server[1049]: 2026/09/21 09:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5175lb # [ 97.904181] niks3-server[1049]: 2026/09/21 09:14:50 INFO Completed upload id=295176worker1 # [ 97.832206] nix-grpc-daemon[1120]: time=2026-09-21T09:14:50.422Z level=INFO msg="Upload complete. (495ms)"5177lb # [ 97.906775] niks3-server[1049]: 2026/09/21 09:14:50 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5178worker1 # [ 97.840161] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=0 paths=1 nar_bytes_in=6405179worker1 # [ 97.854552] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625180worker1 # [ 97.865279] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5181worker1 # [ 97.868952] nix-daemon[840]: reaped child process 1167, status = succeeded5182client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5183client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5184worker1 # [ 97.882217] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5185client # farm-b> LOG-farm-b-on-standby5186worker1 # [ 99.327664] nix-daemon[840]: reaped child process 1181, status = succeeded5187worker1 # [ 99.511642] nix-daemon[840]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5188worker1 # [ 99.528937] nix-daemon[840]: reaped child process 1199, status = succeeded5189lb # [ 99.618151] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received uploads request method=POST path=/api/pending_closures5190worker1 # [ 99.555796] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.144Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5191worker1 # [ 99.558007] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.144Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5192lb # [ 99.656417] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5193lb # [ 99.667554] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5194worker1 # [ 99.602977] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.192Z level=INFO msg="Uploading 1 narinfos"5195lb # [ 99.678208] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5196lb # [ 99.687631] niks3-server[1049]: 2026/09/21 09:14:52 INFO Signed narinfos id=30 count=15197lb # [ 99.691522] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5198lb # [ 99.702209] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5199lb # [ 99.708753] niks3-server[1049]: 2026/09/21 09:14:52 INFO Completed upload id=305200lb # [ 99.711731] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5201worker1 # [ 99.636849] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.227Z level=INFO msg="Upload complete. (343ms)"5202worker1 # [ 99.655045] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15203worker1 # [ 99.672070] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05204worker1 # [ 99.677052] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15205worker1 # [ 99.681926] nix-daemon[840]: reaped child process 1179, status = succeeded5206worker1 # [ 99.700374] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625207worker1 # [ 99.715801] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577625208worker1 # [ 99.730359] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.320Z level=INFO msg="All 2 paths already cached"5209client # copying 1 paths to 'lb:50051'...5210client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5211worker1 # [ 99.740438] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577625212worker1 # [ 99.753841] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5213worker1 # [ 99.950175] nix-daemon[840]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5214worker1 # [ 99.967231] nix-daemon[840]: reaped child process 1208, status = succeeded5215worker1 # [ 100.139659] nix-daemon[840]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5216worker1 # [ 100.155317] nix-daemon[840]: reaped child process 1213, status = succeeded5217lb # [ 100.239664] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received uploads request method=POST path=/api/pending_closures5218worker1 # [ 100.177355] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.767Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5219worker1 # [ 100.178828] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.767Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5220lb # [ 100.267461] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5221worker1 # [ 100.213958] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.803Z level=INFO msg="Uploading 1 narinfos"5222lb # [ 100.280581] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5223lb # [ 100.292202] niks3-server[1049]: 2026/09/21 09:14:52 INFO Signed narinfos id=31 count=15224lb # [ 100.294956] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5225lb # [ 100.304318] niks3-server[1049]: 2026/09/21 09:14:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5226lb # [ 100.310924] niks3-server[1049]: 2026/09/21 09:14:52 INFO Completed upload id=315227worker1 # [ 100.238543] nix-grpc-daemon[1120]: time=2026-09-21T09:14:52.828Z level=INFO msg="Upload complete. (468ms)"5228lb # [ 100.313749] niks3-server[1049]: 2026/09/21 09:14:52 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5229worker1 # [ 100.246637] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=0 paths=1 nar_bytes_in=7685230worker1 # [ 100.259371] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625231worker1 # [ 100.265834] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5232worker1 # [ 100.270472] nix-daemon[840]: reaped child process 1203, status = succeeded5233client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5234client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5235worker1 # [ 100.283779] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5236client # farm-top> LOG-farm-top-on-standby5237worker1 # [ 101.324537] nix-daemon[840]: reaped child process 1217, status = succeeded5238worker1 # [ 101.499650] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5239worker1 # [ 101.516165] nix-daemon[840]: reaped child process 1235, status = succeeded5240lb # [ 101.603906] niks3-server[1049]: 2026/09/21 09:14:54 INFO Received uploads request method=POST path=/api/pending_closures5241worker1 # [ 101.542827] nix-grpc-daemon[1120]: time=2026-09-21T09:14:54.131Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5242worker1 # [ 101.544384] nix-grpc-daemon[1120]: time=2026-09-21T09:14:54.131Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5243lb # [ 101.646217] niks3-server[1049]: 2026/09/21 09:14:54 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5244lb # [ 101.656392] niks3-server[1049]: 2026/09/21 09:14:54 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5245worker1 # [ 101.592712] nix-grpc-daemon[1120]: time=2026-09-21T09:14:54.182Z level=INFO msg="Uploading 1 narinfos"5246lb # [ 101.666324] niks3-server[1049]: 2026/09/21 09:14:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5247lb # [ 101.674061] niks3-server[1049]: 2026/09/21 09:14:54 INFO Signed narinfos id=32 count=15248lb # [ 101.675905] niks3-server[1049]: 2026/09/21 09:14:54 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5249lb # [ 101.683352] niks3-server[1049]: 2026/09/21 09:14:54 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5250lb # [ 101.688599] niks3-server[1049]: 2026/09/21 09:14:54 INFO Completed upload id=325251worker1 # [ 101.616173] nix-grpc-daemon[1120]: time=2026-09-21T09:14:54.206Z level=INFO msg="Upload complete. (320ms)"5252lb # [ 101.690572] niks3-server[1049]: 2026/09/21 09:14:54 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5253worker1 # [ 101.629041] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15254worker1 # [ 101.643586] nix-daemon[840]: reaped child process 1215, status = succeeded5255client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.60 seconds)5256client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5257client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5258worker2 # [ 102.034704] systemd[1]: Listening on Nix worker-protocol over gRPC.5259worker2 # [ 102.043059] systemd[1]: Starting scheduler-token.service...5260worker2 # [ 102.140251] systemd[1]: scheduler-token.service: Deactivated successfully.5261worker2 # [ 102.144480] systemd[1]: Finished scheduler-token.service.5262worker2 # [ 102.145764] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 100ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5263worker2 # [ 102.154247] systemd[1]: Starting Nix worker-protocol over gRPC...5264worker2 # [ 102.334803] nix-grpc-daemon[1361]: ts=2026-09-21T09:14:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5265worker2 # [ 102.337319] nix-grpc-daemon[1361]: ts=2026-09-21T09:14:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5266worker2 # [ 102.341654] nix-grpc-daemon[1361]: ts=2026-09-21T09:14:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15267worker2 # [ 102.350843] nix-grpc-daemon[1361]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5268worker2 # [ 102.359151] nix-grpc-daemon[1361]: E0000 00:00:1789982094.944535 1361 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5269worker2 # [ 102.363596] nix-grpc-daemon[1361]: E0000 00:00:1789982094.947102 1361 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5270worker2 # [ 102.367481] nix-grpc-daemon[1361]: ts=2026-09-21T09:14:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5271lb: must succeed: curl -sf localhost:9901/clusters5272worker2 # [ 102.369989] systemd[1]: Started Nix worker-protocol over gRPC.5273worker2 # [ 102.384941] nix-grpc-daemon[1361]: E0000 00:00:1789982094.966900 1400 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5274worker2 # [ 102.405845] nix-grpc-daemon[1361]: E0000 00:00:1789982094.986685 1400 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5275worker2 # [ 102.408879] nix-grpc-daemon[1361]: E0000 00:00:1789982094.986880 1400 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5276worker2 # [ 102.411369] nix-grpc-daemon[1361]: E0000 00:00:1789982095.001398 1400 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5277worker2 # [ 102.419369] nix-grpc-daemon[1361]: ts=2026-09-21T09:14:55Z level=info event=scheduler_connected addr=lb:500515278worker1 # [ 102.408170] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:54Z level=info event=oidc_jwks provider=mock keys=15279worker1 # [ 102.408548] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:577485280worker1 # [ 102.418049] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05281lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5282worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5283worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5284lb: must succeed: curl -sf localhost:9901/clusters5285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5286lb: must succeed: curl -sf localhost:9901/clusters5287lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5288worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5289worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5290(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 15.39 seconds)5291subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5292client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5293client # [ 102.855071] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5294client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5295worker1 # [ 103.063900] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577625296client # Running as unit: rs.service; invocation ID: 201397f43f4c4b5c9f234bdfb4e1cfab5297client # [ 103.113988] nix[1167]: don't know how to build these paths:5298client # [ 103.115873] nix[1167]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5299worker1 # [ 103.084138] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05300worker1 # [ 103.089987] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15301worker1 # [ 103.135694] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625302worker1 # [ 103.151248] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577625303client # [ 103.205457] nix[1167]: copying 1 paths to 'lb:50051'...5304client # [ 103.209932] nix[1167]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5305worker1 # [ 103.168101] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577625306worker1 # [ 103.182847] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5307worker1 # [ 103.395178] nix-daemon[840]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5308worker1 # [ 103.414129] nix-daemon[840]: reaped child process 1251, status = succeeded5309worker1 # [ 103.588308] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5310worker1 # [ 103.603579] nix-daemon[840]: reaped child process 1256, status = succeeded5311lb # [ 103.690487] niks3-server[1049]: 2026/09/21 09:14:56 INFO Received uploads request method=POST path=/api/pending_closures5312worker1 # [ 103.625516] nix-grpc-daemon[1120]: time=2026-09-21T09:14:56.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5313worker1 # [ 103.627149] nix-grpc-daemon[1120]: time=2026-09-21T09:14:56.215Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5314lb # [ 103.707533] niks3-server[1049]: 2026/09/21 09:14:56 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5315lb # [ 103.714955] niks3-server[1049]: 2026/09/21 09:14:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5316worker1 # [ 103.645795] nix-grpc-daemon[1120]: time=2026-09-21T09:14:56.235Z level=INFO msg="Uploading 1 narinfos"5317lb # [ 103.720906] niks3-server[1049]: 2026/09/21 09:14:56 INFO Signed narinfos id=33 count=15318lb # [ 103.722504] niks3-server[1049]: 2026/09/21 09:14:56 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5319lb # [ 103.730995] niks3-server[1049]: 2026/09/21 09:14:56 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5320lb # [ 103.734470] niks3-server[1049]: 2026/09/21 09:14:56 INFO Completed upload id=335321worker1 # [ 103.662424] nix-grpc-daemon[1120]: time=2026-09-21T09:14:56.252Z level=INFO msg="Upload complete. (463ms)"5322lb # [ 103.736438] niks3-server[1049]: 2026/09/21 09:14:56 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5323worker1 # [ 103.668261] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=0 paths=1 nar_bytes_in=4165324worker1 # [ 103.677663] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577625325worker1 # [ 103.681979] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5326worker1 # [ 103.685542] nix-daemon[840]: reaped child process 1246, status = succeeded5327client # [ 103.735319] nix[1167]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5328worker1 # [ 103.695360] nix-daemon[840]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5329lb: must succeed: curl -sf localhost:9901/clusters5330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5331lb: must succeed: curl -sf localhost:9901/clusters5332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5333lb: must succeed: curl -sf localhost:9901/clusters5334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5335lb: must succeed: curl -sf localhost:9901/clusters5336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5337worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5338worker2 # [ 105.803642] systemd-journald[275]: Received client request to rotate journal, rotating.5339worker2 # [ 105.828630] systemd-journald[275]: Vacuuming done, freed 0B of archived journals from /var/log/journal/c51b609640d9479a81ac4afc5d3f0223.5340worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5341worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5342worker1 # [ 105.862699] systemd[1]: Stopping Nix worker-protocol over gRPC...5343worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5344client: must succeed: sleep 5.05345worker1 # [ 106.246396] nix-grpc-daemon[1038]: ts=2026-09-21T09:14:58Z level=info event=draining5346lb # [ 109.921054] envoy[911]: conn peer=[2001:db8:1::1]:46282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9610 tx=3861 ms=82095347client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5348worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5349worker1 # [ 111.344784] nix-daemon[840]: reaped child process 1260, status = succeeded5350worker1 # [ 111.523721] nix-daemon[840]: accepted connection from pid 1300, user nix-grpc-daemon (trusted)5351worker1 # [ 111.538882] nix-daemon[840]: reaped child process 1303, status = succeeded5352lb # [ 111.627115] niks3-server[1049]: 2026/09/21 09:15:04 INFO Received uploads request method=POST path=/api/pending_closures5353worker1 # [ 111.568451] nix-grpc-daemon[1120]: time=2026-09-21T09:15:04.156Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5354worker1 # [ 111.570278] nix-grpc-daemon[1120]: time=2026-09-21T09:15:04.156Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5355lb # [ 111.674473] niks3-server[1049]: 2026/09/21 09:15:04 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5356lb # [ 111.686135] niks3-server[1049]: 2026/09/21 09:15:04 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5357lb # [ 111.696426] niks3-server[1049]: 2026/09/21 09:15:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5358worker1 # [ 111.629819] nix-grpc-daemon[1120]: time=2026-09-21T09:15:04.219Z level=INFO msg="Uploading 1 narinfos"5359lb # [ 111.705371] niks3-server[1049]: 2026/09/21 09:15:04 INFO Signed narinfos id=34 count=15360lb # [ 111.710847] niks3-server[1049]: 2026/09/21 09:15:04 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5361lb # [ 111.721720] niks3-server[1049]: 2026/09/21 09:15:04 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5362lb # [ 111.727082] niks3-server[1049]: 2026/09/21 09:15:04 INFO Completed upload id=345363worker1 # [ 111.654238] nix-grpc-daemon[1120]: time=2026-09-21T09:15:04.244Z level=INFO msg="Upload complete. (345ms)"5364lb # [ 111.729154] niks3-server[1049]: 2026/09/21 09:15:04 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5365worker1 # [ 111.669227] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57762 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15366worker1 # [ 111.685417] nix-daemon[840]: reaped child process 1258, status = succeeded5367client # [ 111.749492] systemd[1]: rs.service: Deactivated successfully.5368client # [ 111.754532] systemd[1]: rs.service: Consumed 172ms CPU time over 8.897s wall clock time, 39.2M memory peak, 6.9K incoming IP traffic, 8.3K outgoing IP traffic.5369worker1 # [ 112.247375] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:04Z level=info event=signal_exit5370worker1 # [ 112.466206] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_yield5371lb # [ 112.540220] niks3-server[1049]: 2026/09/21 09:15:05 INFO lead: released remote=192.168.1.3:506365372worker2 # [ 112.493194] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_restarting addr=lb:500515373worker1 # [ 112.480527] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577485374worker1 # [ 112.487307] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=worker_gone addr=192.168.1.4:500515375worker1 # [ 112.491522] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_restarting addr=lb:500515376worker1 # [ 112.495599] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577485377worker1 # [ 112.500158] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=worker_gone addr=192.168.1.3:500515378worker2 # [ 112.693568] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515379worker2 # [ 112.698897] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker1 # [ 112.688841] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515381worker1 # [ 112.693030] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker2 # [ 112.899904] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 112.905149] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 112.889214] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515385worker1 # [ 112.894218] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker2 # [ 113.106403] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515387worker2 # [ 113.111719] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5388worker1 # [ 113.095194] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515389worker1 # [ 113.100348] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5390worker2 # [ 113.313139] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515391worker2 # [ 113.318447] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker1 # [ 113.300949] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_connected addr=lb:500515393worker1 # [ 113.306017] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5394worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5395client: waiting for success: ! systemctl is-active rs5396worker1 # [ 113.472052] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:06Z level=info event=scheduler_restarting5397worker2 # [ 113.521513] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_connected addr=lb:500515398worker1 # [ 113.503258] nix-daemon[840]: reaped child process 1111, status = succeeded5399worker2 # [ 113.525840] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5401client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5402worker1 # [ 113.528724] nix-grpc-daemon[1038]: ts=2026-09-21T09:15:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5403client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5404client: must fail: journalctl -u rs | grep -q 'reconnecting'5405client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5406lb: must succeed: curl -sf localhost:9901/clusters5407lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5408worker2 # [ 113.722171] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 113.727469] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker2 # [ 113.928649] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_connected addr=lb:500515411worker2 # [ 113.933771] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 114.135158] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_connected addr=lb:500515413worker2 # [ 114.140571] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker1 # [ 114.314274] nix-grpc-daemon[1038]: E0000 00:00:1789982106.903063 1083 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5415worker2 # [ 114.341607] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_connected addr=lb:500515416worker1 # [ 114.321014] nix-grpc-daemon[1038]: E0000 00:00:1789982106.903329 1083 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5417worker2 # [ 114.346779] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418worker2 # [ 114.550041] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515419worker2 # [ 114.554196] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420worker1 # [ 114.563468] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5421worker1 # [ 114.571536] systemd[1]: Stopped Nix worker-protocol over gRPC.5422worker1 # [ 114.580404] systemd[1]: nix-grpc-daemon.service: Consumed 1.902s CPU time over 27.556s wall clock time, 37.4M memory peak, 50.2K incoming IP traffic, 92.4K outgoing IP traffic.5423worker1 # [ 114.590971] systemd[1]: Starting Nix worker-protocol over gRPC...5424lb: must succeed: curl -sf localhost:9901/clusters5425worker2 # [ 114.750915] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515426worker2 # [ 114.756161] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5428worker1 # [ 114.786544] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5429worker1 # [ 114.789097] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5430worker1 # [ 114.793294] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15431worker1 # [ 114.800058] nix-grpc-daemon[1310]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5432worker1 # [ 114.801516] nix-grpc-daemon[1310]: E0000 00:00:1789982107.389901 1310 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5433worker1 # [ 114.809611] nix-grpc-daemon[1310]: E0000 00:00:1789982107.396247 1310 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5434worker1 # [ 114.813303] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5435worker1 # [ 114.815747] systemd[1]: Started Nix worker-protocol over gRPC.5436worker1 # [ 114.819105] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_take_over5437lb # [ 114.890815] niks3-server[1049]: 2026/09/21 09:15:07 INFO lead: acquired remote=192.168.1.3:373965438worker1 # [ 114.822400] nix-grpc-daemon[1310]: E0000 00:00:1789982107.411292 1349 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5439worker1 # [ 114.852156] nix-grpc-daemon[1310]: E0000 00:00:1789982107.429070 1349 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5440worker1 # [ 114.854428] nix-grpc-daemon[1310]: E0000 00:00:1789982107.429237 1349 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5441worker1 # [ 114.859316] nix-grpc-daemon[1310]: E0000 00:00:1789982107.448570 1349 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5442worker1 # [ 114.871918] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker2 # [ 114.957203] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515444worker2 # [ 114.962319] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5445worker1 # [ 115.073132] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515446worker1 # [ 115.078233] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker2 # [ 115.163396] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515448worker2 # [ 115.168569] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5449worker1 # [ 115.278967] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515450worker1 # [ 115.284179] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5451worker2 # [ 115.369747] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_connected addr=lb:500515452worker2 # [ 115.374841] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5453worker1 # [ 115.485033] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=scheduler_connected addr=lb:500515454worker1 # [ 115.490156] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker2 # [ 115.577651] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:08Z level=info event=scheduler_connected addr=lb:500515456worker2 # [ 115.581798] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker1 # [ 115.690893] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 115.699513] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386445459worker1 # [ 115.704802] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05460worker2 # [ 115.778202] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:08Z level=info event=scheduler_connected addr=lb:500515461worker1 # [ 115.766573] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=oidc_jwks provider=mock keys=15462lb: must succeed: curl -sf localhost:9901/clusters5463worker1 # [ 115.770980] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386445464worker1 # [ 115.776368] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05465lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5466worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5467worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5468(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.19 seconds)5469subtest: scheduler restart mid-build: build finishes, no double build5470client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5471client # [ 116.045261] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5472client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5473worker1 # [ 116.269910] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386605474worker1 # [ 116.286002] nix-daemon[840]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5475client # Running as unit: mid.service; invocation ID: 3020e26444f94774b03356f6533d3ad25476client # [ 116.345660] nix[1203]: don't know how to build these paths:5477client # [ 116.350315] nix[1203]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5478worker1 # [ 116.317222] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05479worker1 # [ 116.324373] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15480worker1 # [ 116.364442] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386605481worker1 # [ 116.380993] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)5482worker1 # [ 116.450344] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)5483worker1 # [ 116.530982] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 226 ms (attempt 3/5)5484worker1 # [ 116.764865] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 284 ms (attempt 4/5)5485worker1 # [ 117.057006] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5486worker1 # [ 117.077485] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386605487client # [ 117.135560] nix[1203]: copying 1 paths to 'lb:50051'...5488worker1 # [ 117.092780] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386605489client # [ 117.135983] nix[1203]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5490worker1 # [ 117.107588] nix-daemon[840]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5491worker1 # [ 117.136923] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:09Z level=info event=niks3_push_started pid=13785492worker1 # [ 117.366452] nix-grpc-daemon[1378]: time=2026-09-21T09:15:09.955Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5493worker1 # [ 117.538520] nix-daemon[840]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5494worker1 # [ 117.557034] nix-daemon[840]: reaped child process 1398, status = succeeded5495worker1 # [ 117.732679] nix-daemon[840]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5496worker1 # [ 117.750943] nix-daemon[840]: reaped child process 1403, status = succeeded5497lb # [ 117.835581] niks3-server[1049]: 2026/09/21 09:15:10 INFO Received uploads request method=POST path=/api/pending_closures5498worker1 # [ 117.776064] nix-grpc-daemon[1378]: time=2026-09-21T09:15:10.365Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5499worker1 # [ 117.777551] nix-grpc-daemon[1378]: time=2026-09-21T09:15:10.365Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5500lb # [ 117.871283] niks3-server[1049]: 2026/09/21 09:15:10 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5501worker1 # [ 117.818591] nix-grpc-daemon[1378]: time=2026-09-21T09:15:10.408Z level=INFO msg="Uploading 1 narinfos"5502lb # [ 117.884083] niks3-server[1049]: 2026/09/21 09:15:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5503lb # [ 117.896595] niks3-server[1049]: 2026/09/21 09:15:10 INFO Signed narinfos id=35 count=15504lb # [ 117.903046] niks3-server[1049]: 2026/09/21 09:15:10 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5505lb # [ 117.914588] niks3-server[1049]: 2026/09/21 09:15:10 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5506lb # [ 117.921574] niks3-server[1049]: 2026/09/21 09:15:10 INFO Completed upload id=355507worker1 # [ 117.848797] nix-grpc-daemon[1378]: time=2026-09-21T09:15:10.438Z level=INFO msg="Upload complete. (478ms)"5508lb # [ 117.923857] niks3-server[1049]: 2026/09/21 09:15:10 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5509worker1 # [ 117.856759] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38660 duration_s=0 paths=1 nar_bytes_in=4245510worker1 # [ 117.871658] nix-grpc-daemon[1310]: ts=2026-09-21T09:15:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386605511worker1 # [ 117.880533] nix-daemon[840]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5512worker1 # [ 117.883411] nix-daemon[840]: reaped child process 1376, status = succeeded5513worker1 # [ 117.894208] nix-daemon[840]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5514client # [ 117.936568] nix[1203]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5515lb: must succeed: curl -sf localhost:9901/clusters5516lb # [ 119.918416] envoy[911]: conn peer=[2001:db8:1::1]:38406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1349 ms=86155517lb # [ 119.924327] envoy[911]: conn peer=[2001:db8:1::3]:55118 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4957 tx=1460 ms=262175518lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5519lb: must succeed: curl -sf localhost:9901/clusters5520lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5521worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5522worker1 # [ 120.010276] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1310 (nix-grpc-daemon) on client request.5523worker1 # [ 120.012141] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5524lb # [ 120.098016] niks3-server[1049]: 2026/09/21 09:15:12 INFO lead: released remote=192.168.1.3:373965525client # [ 120.082108] nix[1203]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5526worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5527client: must succeed: sleep 5.05528worker2 # [ 120.060831] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:12Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5529client # [ 120.088456] nix[1203]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5530worker1 # [ 120.054012] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5531worker1 # [ 120.055582] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5532worker1 # [ 120.063812] systemd[1]: nix-grpc-daemon.service: Consumed 543ms CPU time over 5.461s wall clock time, 34M memory peak, 14.3K incoming IP traffic, 18.5K outgoing IP traffic.5533worker1 # [ 120.072560] nix-daemon[840]: reaped child process 1369, status = succeeded5534worker1 # [ 120.073799] nix-daemon[840]: reaped child process 1405, status = succeeded5535worker1 # [ 120.080610] nix-daemon[1407]: unexpected Nix daemon error: error: interrupted by the user5536worker1 # [ 120.086622] nix-daemon[840]: reaped child process 1407, status = failed with exit code 15537worker2 # [ 120.269824] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:12Z level=info event=scheduler_connected addr=lb:500515538worker1 # [ 120.252514] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5539worker1 # [ 120.266030] systemd[1]: Starting Nix worker-protocol over gRPC...5540worker1 # [ 120.474318] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5541worker1 # [ 120.476931] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5542worker1 # [ 120.481256] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15543worker1 # [ 120.488057] nix-grpc-daemon[1443]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5544worker1 # [ 120.489497] nix-grpc-daemon[1443]: E0000 00:00:1789982113.077675 1443 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5545worker1 # [ 120.499546] nix-grpc-daemon[1443]: E0000 00:00:1789982113.083815 1443 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5546worker1 # [ 120.503575] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5547worker1 # [ 120.507084] systemd[1]: Started Nix worker-protocol over gRPC.5548lb # [ 120.577531] niks3-server[1049]: 2026/09/21 09:15:13 INFO lead: acquired remote=192.168.1.3:374545549worker1 # [ 120.509634] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=scheduler_take_over5550worker1 # [ 120.512563] nix-grpc-daemon[1443]: E0000 00:00:1789982113.102746 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5551worker1 # [ 120.540181] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=oidc_jwks provider=mock keys=15552worker1 # [ 120.541568] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386765553worker1 # [ 120.552983] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05554worker1 # [ 120.554883] nix-grpc-daemon[1443]: E0000 00:00:1789982113.122163 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5555worker1 # [ 120.557168] nix-grpc-daemon[1443]: E0000 00:00:1789982113.122318 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5556worker1 # [ 120.562337] nix-grpc-daemon[1443]: E0000 00:00:1789982113.141823 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5557worker1 # [ 120.566649] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05558worker1 # [ 120.568862] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15559worker1 # [ 120.570908] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=scheduler_connected addr=lb:500515560worker1 # [ 120.572207] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386765561worker1 # [ 120.573700] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05562worker2 # [ 120.610954] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415245563worker2 # [ 120.625773] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5564worker2 # [ 120.651037] nix-grpc-daemon[1361]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5565worker2 # [ 120.716524] nix-grpc-daemon[1361]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 2/5)5566worker2 # [ 120.905080] nix-grpc-daemon[1361]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 3/5)5567worker2 # [ 121.019308] nix-grpc-daemon[1361]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 748 ms (attempt 4/5)5568worker2 # [ 121.778112] nix-grpc-daemon[1361]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5569worker2 # [ 121.795333] nix-grpc-daemon[1361]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5570worker2 # [ 121.828771] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5571client # [ 121.868222] nix[1203]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5572worker2 # [ 121.850802] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5573client: (finished: must succeed: sleep 5.0, in 5.07 seconds)5574client: waiting for success: ! systemctl is-active mid5575worker2 # [ 125.474046] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:18Z level=info event=niks3_push_started pid=15155576worker2 # [ 125.492474] nix-daemon[862]: reaped child process 1491, status = succeeded5577worker2 # [ 125.670638] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.266Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5578worker2 # [ 125.843012] nix-daemon[862]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)5579worker2 # [ 125.861647] nix-daemon[862]: reaped child process 1531, status = succeeded5580lb # [ 125.927151] niks3-server[1049]: 2026/09/21 09:15:18 INFO Received uploads request method=POST path=/api/pending_closures5581worker2 # [ 125.888080] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.483Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5582worker2 # [ 125.889582] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.483Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5583lb # [ 125.964345] niks3-server[1049]: 2026/09/21 09:15:18 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5584lb # [ 125.982419] niks3-server[1049]: 2026/09/21 09:15:18 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5585lb # [ 125.992465] niks3-server[1049]: 2026/09/21 09:15:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5586worker2 # [ 125.947733] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.542Z level=INFO msg="Uploading 1 narinfos"5587lb # [ 126.000333] niks3-server[1049]: 2026/09/21 09:15:18 INFO Signed narinfos id=36 count=15588lb # [ 126.007952] niks3-server[1049]: 2026/09/21 09:15:18 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5589lb # [ 126.016285] niks3-server[1049]: 2026/09/21 09:15:18 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5590worker2 # [ 125.972514] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.568Z level=INFO msg="Upload complete. (297ms)"5591lb # [ 126.025245] niks3-server[1049]: 2026/09/21 09:15:18 INFO Completed upload id=365592worker2 # [ 125.984128] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41524 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15593worker2 # [ 125.997333] nix-grpc-daemon[1515]: time=2026-09-21T09:15:18.586Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5594worker2 # [ 126.003164] nix-daemon[862]: reaped child process 1489, status = succeeded5595client # [ 126.047962] systemd[1]: mid.service: Deactivated successfully.5596client # [ 126.053141] systemd[1]: mid.service: Consumed 187ms CPU time over 10.004s wall clock time, 41.2M memory peak, 8.2K incoming IP traffic, 9.8K outgoing IP traffic.5597client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5598client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5599client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5600(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.66 seconds)5601subtest: interrupting the client stops the build on the worker5602client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5603client # [ 126.674818] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5604client # Running as unit: intr.service; invocation ID: 71a8fde9d1d94c65baa4444cae68dc215605client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5606worker1 # [ 126.874651] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444425607worker1 # [ 126.889606] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5608client # [ 126.950715] nix[1239]: don't know how to build these paths:5609client # [ 126.953373] nix[1239]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5610worker1 # [ 126.922345] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:19Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05611worker1 # [ 126.929416] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:19Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15612worker1 # [ 126.976927] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425613worker1 # [ 126.994447] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)5614worker1 # [ 127.075162] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 2/5)5615worker1 # [ 127.112759] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 3/5)5616worker1 # [ 127.363312] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 207 ms (attempt 4/5)5617worker1 # [ 127.578533] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5618worker1 # [ 127.601673] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425619client # [ 127.659353] nix[1239]: copying 1 paths to 'lb:50051'...5620worker1 # [ 127.616659] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444425621client # [ 127.659784] nix[1239]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5622worker1 # [ 127.630811] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5623worker1 # [ 127.657808] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:20Z level=info event=niks3_push_started pid=15185624worker1 # [ 127.884237] nix-grpc-daemon[1518]: time=2026-09-21T09:15:20.473Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5625worker1 # [ 128.068216] nix-daemon[840]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)5626worker1 # [ 128.083853] nix-daemon[840]: reaped child process 1538, status = succeeded5627worker1 # [ 128.260057] nix-daemon[840]: accepted connection from pid 1540, user nix-grpc-daemon (trusted)5628worker1 # [ 128.275774] nix-daemon[840]: reaped child process 1543, status = succeeded5629lb # [ 128.361893] niks3-server[1049]: 2026/09/21 09:15:20 INFO Received uploads request method=POST path=/api/pending_closures5630worker1 # [ 128.307229] nix-grpc-daemon[1518]: time=2026-09-21T09:15:20.896Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5631worker1 # [ 128.308932] nix-grpc-daemon[1518]: time=2026-09-21T09:15:20.896Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5632lb # [ 128.421102] niks3-server[1049]: 2026/09/21 09:15:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5633lb # [ 128.432813] niks3-server[1049]: 2026/09/21 09:15:21 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5634lb # [ 128.440766] niks3-server[1049]: 2026/09/21 09:15:21 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5635lb # [ 128.447458] niks3-server[1049]: 2026/09/21 09:15:21 INFO Signed narinfos id=37 count=15636worker1 # [ 128.375920] nix-grpc-daemon[1518]: time=2026-09-21T09:15:20.965Z level=INFO msg="Uploading 1 narinfos"5637lb # [ 128.456359] niks3-server[1049]: 2026/09/21 09:15:21 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5638lb # [ 128.463317] niks3-server[1049]: 2026/09/21 09:15:21 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5639lb # [ 128.475663] niks3-server[1049]: 2026/09/21 09:15:21 INFO Completed upload id=375640worker1 # [ 128.403899] nix-grpc-daemon[1518]: time=2026-09-21T09:15:20.993Z level=INFO msg="Upload complete. (514ms)"5641worker1 # [ 128.417117] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=4245642worker1 # [ 128.432588] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425643worker1 # [ 128.442708] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5644worker1 # [ 128.446714] nix-daemon[840]: reaped child process 1516, status = succeeded5645client # [ 128.499800] nix[1239]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5646worker1 # [ 128.460266] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5647lb # [ 129.917516] envoy[911]: conn peer=[2001:db8:1::3]:58952 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=897 tx=411 ms=51745648lb # [ 129.922567] envoy[911]: conn peer=[2001:db8:1::1]:44762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3828 tx=1683 ms=97105649client: must succeed: systemctl kill -s INT intr5650client # [ 130.416440] systemd[1]: intr.service: Sent signal SIGINT to main process 1239 (nix) on client request.5651worker1 # [ 130.381748] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:22Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5652client # [ 130.426524] nix[1239]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5653client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5654client # [ 130.434302] nix[1239]: error: interrupted by the user5655client # [ 130.468326] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5656client # [ 130.472980] systemd[1]: intr.service: Failed with result 'exit-code'.5657client # [ 130.475092] systemd[1]: intr.service: Consumed 161ms CPU time over 3.796s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic.5658worker1 # [ 130.485860] nix-daemon[840]: reaped child process 1545, status = succeeded5659worker1 # [ 130.489344] nix-daemon[1547]: unexpected Nix daemon error: error: interrupted by the user5660worker1 # [ 130.496591] nix-daemon[840]: reaped child process 1547, status = failed with exit code 15661(finished: subtest: interrupting the client stops the build on the worker, in 5.12 seconds)5662subtest: a deploy mid-build drains: the build finishes, the new generation takes over5663client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5664client # [ 131.827597] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5665client # Running as unit: sw.service; invocation ID: dfc55eae9ea64bf680a013bfe7e37e415666client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5667worker1 # [ 132.041868] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444425668client # [ 132.092171] nix[1259]: don't know how to build these paths:5669client # [ 132.093128] nix[1259]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5670worker1 # [ 132.063481] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:24Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05671worker1 # [ 132.069578] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:24Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15672worker2 # [ 132.130611] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415245673worker2 # [ 132.147071] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415245674client # [ 132.180342] nix[1259]: copying 1 paths to 'lb:50051'...5675client # [ 132.185873] nix[1259]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5676worker2 # [ 132.164120] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415245677worker2 # [ 132.179351] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5678worker2 # [ 132.388775] nix-daemon[862]: accepted connection from pid 1563, user nix-grpc-daemon (trusted)5679worker2 # [ 132.406652] nix-daemon[862]: reaped child process 1566, status = succeeded5680worker2 # [ 132.577280] nix-daemon[862]: accepted connection from pid 1568, user nix-grpc-daemon (trusted)5681worker2 # [ 132.594919] nix-daemon[862]: reaped child process 1571, status = succeeded5682lb # [ 132.657906] niks3-server[1049]: 2026/09/21 09:15:25 INFO Received uploads request method=POST path=/api/pending_closures5683worker2 # [ 132.620309] nix-grpc-daemon[1515]: time=2026-09-21T09:15:25.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5684worker2 # [ 132.621833] nix-grpc-daemon[1515]: time=2026-09-21T09:15:25.216Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5685lb # [ 132.694445] niks3-server[1049]: 2026/09/21 09:15:25 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5686worker2 # [ 132.656577] nix-grpc-daemon[1515]: time=2026-09-21T09:15:25.252Z level=INFO msg="Uploading 1 narinfos"5687lb # [ 132.711098] niks3-server[1049]: 2026/09/21 09:15:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5688lb # [ 132.719713] niks3-server[1049]: 2026/09/21 09:15:25 INFO Signed narinfos id=38 count=15689lb # [ 132.723428] niks3-server[1049]: 2026/09/21 09:15:25 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5690lb # [ 132.733137] niks3-server[1049]: 2026/09/21 09:15:25 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5691worker2 # [ 132.682267] nix-grpc-daemon[1515]: time=2026-09-21T09:15:25.278Z level=INFO msg="Upload complete. (485ms)"5692lb # [ 132.738175] niks3-server[1049]: 2026/09/21 09:15:25 INFO Completed upload id=385693worker2 # [ 132.688415] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41524 duration_s=0 paths=1 nar_bytes_in=4165694lb # [ 132.741457] niks3-server[1049]: 2026/09/21 09:15:25 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5695worker2 # [ 132.703724] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415245696worker2 # [ 132.710004] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5697worker2 # [ 132.714927] nix-daemon[862]: reaped child process 1561, status = succeeded5698worker2 # [ 132.725926] nix-daemon[862]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5699client # [ 132.745723] nix[1259]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5700worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25701worker2 # Checking switch inhibitors... done5702worker2 # [ 135.285069] nixos[1606]: switching to system configuration /nix/store/40rw7cdwyigbhkcn0m0j4w1x14h7fl4w-nixos-system-worker2-test5703worker2 # [ 135.288598] systemd[1]: Stopped target Remote File Systems.5704worker2 # activating the configuration...5705worker2 # [ 135.290872] systemd[1]: Stopped target Local File Systems.5706worker2 # setting up /etc...5707worker2 # [ 136.067481] systemd[1]: Reload requested from client PID 1606 ('.switch-to-conf') (unit backdoor.service)...5708worker2 # [ 136.069435] systemd[1]: Reloading...5709worker2 # [ 136.970297] systemd[1]: Reloading finished in 897 ms.5710worker2 # restarting sysinit-reactivation.target5711worker2 # [ 137.049319] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5712worker2 # [ 137.206962] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5713worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5714worker2 # [ 137.212653] systemd[1]: Reached target Reactivate sysinit units.5715worker2 # [ 137.213501] systemd[1]: Reloading D-Bus System Message Bus...5716worker2 # [ 137.214546] dbus-broker-launch[502]: Caught SIGHUP, trigger reload.5717worker2 # [ 137.225895] systemd[1]: Reloading Nix worker-protocol over gRPC...5718worker2 # [ 137.249303] dbus-broker-launch[502]: Looking up NSS user entry for 'systemd-timesync'...5719worker2 # [ 137.251350] dbus-broker-launch[502]: NSS returned no entry for 'systemd-timesync'5720worker2 # [ 137.261081] dbus-broker-launch[502]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5721worker2 # [ 137.279999] systemd[1]: Reloaded Nix worker-protocol over gRPC.5722worker2 # [ 137.294516] systemd[1]: Reloaded D-Bus System Message Bus.5723worker2 # [ 137.302175] systemd[1]: Reached target Remote File Systems.5724worker2 # [ 137.305206] systemd[1]: Starting linger-users.service...5725worker2 # [ 137.313003] systemd[1]: Reached target Local File Systems.5726worker2 # [ 137.313974] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5727worker2 # [ 137.323076] systemd[1]: Starting Create SUID/SGID Wrappers...5728worker2 # [ 137.331273] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5729worker2 # [ 137.336050] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675730worker2 # [ 137.344144] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5731worker2 # [ 137.345676] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5732worker2 # [ 137.346891] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5733worker2 # [ 137.354018] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5734worker2 # [ 137.359314] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5735worker2 # [ 137.367764] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:29Z level=info event=draining5736worker2 # [ 137.400384] systemd[1]: linger-users.service: Deactivated successfully.5737worker2 # [ 137.401321] systemd[1]: Finished linger-users.service.5738worker2 # [ 137.850082] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5739worker2 # [ 137.851209] systemd[1]: Finished Create SUID/SGID Wrappers.5740worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5741worker2 # [ 138.103951] nixos[1606]: finished switching to system configuration /nix/store/40rw7cdwyigbhkcn0m0j4w1x14h7fl4w-nixos-system-worker2-test5742worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.65 seconds)5743worker2: must succeed: pgrep -f 'read -t [9]0 x'5744worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5745client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25746client # don't know how to build these paths:5747client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5748worker1 # [ 138.433939] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444425749worker1 # [ 138.441881] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425750worker1 # [ 138.463087] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05751worker1 # [ 138.470428] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15752worker1 # [ 138.512679] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425753worker1 # [ 138.528533] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425754client # copying 1 paths to 'lb:50051'...5755client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5756worker1 # [ 138.544970] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444425757worker1 # [ 138.566165] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5758worker1 # [ 138.766425] nix-daemon[840]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)5759worker1 # [ 138.784852] nix-daemon[840]: reaped child process 1604, status = succeeded5760worker1 # [ 138.966048] nix-daemon[840]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)5761worker1 # [ 138.983027] nix-daemon[840]: reaped child process 1609, status = succeeded5762lb # [ 139.067245] niks3-server[1049]: 2026/09/21 09:15:31 INFO Received uploads request method=POST path=/api/pending_closures5763worker1 # [ 139.005349] nix-grpc-daemon[1518]: time=2026-09-21T09:15:31.594Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5764worker1 # [ 139.006805] nix-grpc-daemon[1518]: time=2026-09-21T09:15:31.594Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5765lb # [ 139.095504] niks3-server[1049]: 2026/09/21 09:15:31 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5766worker1 # [ 139.037305] nix-grpc-daemon[1518]: time=2026-09-21T09:15:31.627Z level=INFO msg="Uploading 1 narinfos"5767lb # [ 139.112215] niks3-server[1049]: 2026/09/21 09:15:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5768lb # [ 139.123497] niks3-server[1049]: 2026/09/21 09:15:31 INFO Signed narinfos id=39 count=15769lb # [ 139.132680] niks3-server[1049]: 2026/09/21 09:15:31 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5770worker1 # [ 139.064941] nix-grpc-daemon[1518]: time=2026-09-21T09:15:31.655Z level=INFO msg="Upload complete. (482ms)"5771lb # [ 139.137215] niks3-server[1049]: 2026/09/21 09:15:31 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5772lb # [ 139.142339] niks3-server[1049]: 2026/09/21 09:15:31 INFO Completed upload id=395773worker1 # [ 139.072114] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=5285774lb # [ 139.145457] niks3-server[1049]: 2026/09/21 09:15:31 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5775worker1 # [ 139.082863] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425776worker1 # [ 139.090939] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5777worker1 # [ 139.093395] nix-daemon[840]: reaped child process 1599, status = succeeded5778client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5779worker1 # [ 139.103646] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5780client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5781lb # [ 139.917524] envoy[911]: conn peer=[2001:db8:1::1]:39114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=730 ms=35385782client # farm-a> LOG-farm-a-during-drain5783worker1 # [ 140.331771] nix-daemon[840]: reaped child process 1613, status = succeeded5784worker1 # [ 140.500785] nix-daemon[840]: accepted connection from pid 1628, user nix-grpc-daemon (trusted)5785worker1 # [ 140.518022] nix-daemon[840]: reaped child process 1631, status = succeeded5786lb # [ 140.603726] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received uploads request method=POST path=/api/pending_closures5787worker1 # [ 140.544178] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.131Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5788worker1 # [ 140.545686] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.131Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5789lb # [ 140.637486] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5790lb # [ 140.652741] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5791lb # [ 140.668462] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5792worker1 # [ 140.602174] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.191Z level=INFO msg="Uploading 1 narinfos"5793lb # [ 140.679219] niks3-server[1049]: 2026/09/21 09:15:33 INFO Signed narinfos id=40 count=15794lb # [ 140.685419] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5795lb # [ 140.693399] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5796lb # [ 140.698174] niks3-server[1049]: 2026/09/21 09:15:33 INFO Completed upload id=405797worker1 # [ 140.625940] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.216Z level=INFO msg="Upload complete. (310ms)"5798lb # [ 140.701091] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5799worker1 # [ 140.639853] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15800worker1 # [ 140.657120] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05801worker1 # [ 140.661222] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15802worker1 # [ 140.665315] nix-daemon[840]: reaped child process 1611, status = succeeded5803worker1 # [ 140.696566] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425804worker1 # [ 140.712540] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425805worker1 # [ 140.728233] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.318Z level=INFO msg="All 1 paths already cached"5806client # copying 1 paths to 'lb:50051'...5807worker1 # [ 140.738560] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444425808client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5809worker1 # [ 140.761379] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5810worker1 # [ 140.963272] nix-daemon[840]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)5811worker1 # [ 140.983317] nix-daemon[840]: reaped child process 1640, status = succeeded5812worker1 # [ 141.158118] nix-daemon[840]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)5813worker1 # [ 141.174168] nix-daemon[840]: reaped child process 1645, status = succeeded5814lb # [ 141.257329] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received uploads request method=POST path=/api/pending_closures5815worker1 # [ 141.197149] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.786Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5816worker1 # [ 141.198606] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.786Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5817lb # [ 141.299526] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5818worker1 # [ 141.242244] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.832Z level=INFO msg="Uploading 1 narinfos"5819lb # [ 141.315832] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5820lb # [ 141.325456] niks3-server[1049]: 2026/09/21 09:15:33 INFO Signed narinfos id=41 count=15821lb # [ 141.331073] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5822lb # [ 141.336848] niks3-server[1049]: 2026/09/21 09:15:33 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5823worker1 # [ 141.270575] nix-grpc-daemon[1518]: time=2026-09-21T09:15:33.860Z level=INFO msg="Upload complete. (491ms)"5824lb # [ 141.343439] niks3-server[1049]: 2026/09/21 09:15:33 INFO Completed upload id=415825lb # [ 141.346061] niks3-server[1049]: 2026/09/21 09:15:33 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5826worker1 # [ 141.281280] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=6405827worker1 # [ 141.293589] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425828worker1 # [ 141.299971] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5829worker1 # [ 141.305063] nix-daemon[840]: reaped child process 1635, status = succeeded5830client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5831client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5832worker1 # [ 141.319104] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5833client # farm-b> LOG-farm-b-during-drain5834worker1 # [ 141.961741] nix-daemon[840]: reaped child process 1649, status = succeeded5835worker1 # [ 142.141506] nix-daemon[840]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5836worker1 # [ 142.159620] nix-daemon[840]: reaped child process 1667, status = succeeded5837lb # [ 142.247781] niks3-server[1049]: 2026/09/21 09:15:34 INFO Received uploads request method=POST path=/api/pending_closures5838worker1 # [ 142.185963] nix-grpc-daemon[1518]: time=2026-09-21T09:15:34.775Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5839worker1 # [ 142.188572] nix-grpc-daemon[1518]: time=2026-09-21T09:15:34.778Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5840lb # [ 142.283406] niks3-server[1049]: 2026/09/21 09:15:34 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5841lb # [ 142.299454] niks3-server[1049]: 2026/09/21 09:15:34 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5842worker1 # [ 142.239608] nix-grpc-daemon[1518]: time=2026-09-21T09:15:34.829Z level=INFO msg="Uploading 1 narinfos"5843lb # [ 142.312286] niks3-server[1049]: 2026/09/21 09:15:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5844lb # [ 142.315836] niks3-server[1049]: 2026/09/21 09:15:34 INFO Signed narinfos id=42 count=15845lb # [ 142.328061] niks3-server[1049]: 2026/09/21 09:15:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5846lb # [ 142.333106] niks3-server[1049]: 2026/09/21 09:15:34 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5847worker1 # [ 142.261029] nix-grpc-daemon[1518]: time=2026-09-21T09:15:34.851Z level=INFO msg="Upload complete. (332ms)"5848lb # [ 142.335978] niks3-server[1049]: 2026/09/21 09:15:34 INFO Completed upload id=425849lb # [ 142.338960] niks3-server[1049]: 2026/09/21 09:15:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5850worker1 # [ 142.280598] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15851worker1 # [ 142.293704] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05852worker1 # [ 142.297887] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15853worker1 # [ 142.301941] nix-daemon[840]: reaped child process 1647, status = succeeded5854worker1 # [ 142.324114] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425855worker1 # [ 142.340955] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425856client # copying 1 paths to 'lb:50051'...5857worker1 # [ 142.355673] nix-grpc-daemon[1518]: time=2026-09-21T09:15:34.945Z level=INFO msg="All 2 paths already cached"5858worker1 # [ 142.362371] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444425859client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5860worker1 # [ 142.385438] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5861worker1 # [ 142.588439] nix-daemon[840]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5862worker1 # [ 142.605686] nix-daemon[840]: reaped child process 1676, status = succeeded5863worker1 # [ 142.784180] nix-daemon[840]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)5864worker1 # [ 142.800706] nix-daemon[840]: reaped child process 1681, status = succeeded5865lb # [ 142.886760] niks3-server[1049]: 2026/09/21 09:15:35 INFO Received uploads request method=POST path=/api/pending_closures5866worker1 # [ 142.825796] nix-grpc-daemon[1518]: time=2026-09-21T09:15:35.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5867worker1 # [ 142.827269] nix-grpc-daemon[1518]: time=2026-09-21T09:15:35.415Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5868lb # [ 142.914458] niks3-server[1049]: 2026/09/21 09:15:35 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5869worker1 # [ 142.855194] nix-grpc-daemon[1518]: time=2026-09-21T09:15:35.445Z level=INFO msg="Uploading 1 narinfos"5870lb # [ 142.929315] niks3-server[1049]: 2026/09/21 09:15:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5871lb # [ 142.939624] niks3-server[1049]: 2026/09/21 09:15:35 INFO Signed narinfos id=43 count=15872lb # [ 142.947175] niks3-server[1049]: 2026/09/21 09:15:35 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5873worker1 # [ 142.882707] nix-grpc-daemon[1518]: time=2026-09-21T09:15:35.472Z level=INFO msg="Upload complete. (480ms)"5874lb # [ 142.954959] niks3-server[1049]: 2026/09/21 09:15:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5875lb # [ 142.959580] niks3-server[1049]: 2026/09/21 09:15:35 INFO Completed upload id=435876lb # [ 142.962284] niks3-server[1049]: 2026/09/21 09:15:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5877worker1 # [ 142.890800] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=7685878worker1 # [ 142.896729] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425879worker1 # [ 142.908141] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5880worker1 # [ 142.910330] nix-daemon[840]: reaped child process 1671, status = succeeded5881client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5882worker1 # [ 142.922510] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5883client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5884client # farm-top> LOG-farm-top-during-drain5885worker1 # [ 144.551194] nix-daemon[840]: reaped child process 1685, status = succeeded5886worker1 # [ 144.714843] nix-daemon[840]: accepted connection from pid 1700, user nix-grpc-daemon (trusted)5887worker1 # [ 144.733551] nix-daemon[840]: reaped child process 1703, status = succeeded5888lb # [ 144.818134] niks3-server[1049]: 2026/09/21 09:15:37 INFO Received uploads request method=POST path=/api/pending_closures5889worker1 # [ 144.758208] nix-grpc-daemon[1518]: time=2026-09-21T09:15:37.346Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5890worker1 # [ 144.759682] nix-grpc-daemon[1518]: time=2026-09-21T09:15:37.346Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5891lb # [ 144.861840] niks3-server[1049]: 2026/09/21 09:15:37 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5892worker1 # [ 144.800347] nix-grpc-daemon[1518]: time=2026-09-21T09:15:37.390Z level=INFO msg="Uploading 1 narinfos"5893lb # [ 144.873816] niks3-server[1049]: 2026/09/21 09:15:37 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5894lb # [ 144.884789] niks3-server[1049]: 2026/09/21 09:15:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5895lb # [ 144.891148] niks3-server[1049]: 2026/09/21 09:15:37 INFO Signed narinfos id=44 count=15896lb # [ 144.897904] niks3-server[1049]: 2026/09/21 09:15:37 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5897worker1 # [ 144.830099] nix-grpc-daemon[1518]: time=2026-09-21T09:15:37.420Z level=INFO msg="Upload complete. (296ms)"5898lb # [ 144.902581] niks3-server[1049]: 2026/09/21 09:15:37 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5899lb # [ 144.907655] niks3-server[1049]: 2026/09/21 09:15:37 INFO Completed upload id=445900lb # [ 144.911294] niks3-server[1049]: 2026/09/21 09:15:37 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5901worker1 # [ 144.840942] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15902worker1 # [ 144.857239] nix-daemon[840]: reaped child process 1683, status = succeeded5903client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.75 seconds)5904client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5905client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.24 seconds)5906worker2: must succeed: pkill -f 'read -t [9]0 x'5907worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5908client: waiting for success: ! systemctl is-active sw5909worker2 # [ 145.386529] nix-daemon[862]: reaped child process 1575, status = succeeded5910worker2 # [ 145.571426] nix-daemon[862]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)5911worker2 # [ 145.589958] nix-daemon[862]: reaped child process 1749, status = succeeded5912lb # [ 145.651903] niks3-server[1049]: 2026/09/21 09:15:38 INFO Received uploads request method=POST path=/api/pending_closures5913worker2 # [ 145.611629] nix-grpc-daemon[1515]: time=2026-09-21T09:15:38.207Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5914worker2 # [ 145.613239] nix-grpc-daemon[1515]: time=2026-09-21T09:15:38.207Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5915lb # [ 145.679775] niks3-server[1049]: 2026/09/21 09:15:38 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5916lb # [ 145.694274] niks3-server[1049]: 2026/09/21 09:15:38 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5917worker2 # [ 145.661606] nix-grpc-daemon[1515]: time=2026-09-21T09:15:38.256Z level=INFO msg="Uploading 1 narinfos"5918lb # [ 145.711144] niks3-server[1049]: 2026/09/21 09:15:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5919lb # [ 145.724383] niks3-server[1049]: 2026/09/21 09:15:38 INFO Signed narinfos id=45 count=15920lb # [ 145.731126] niks3-server[1049]: 2026/09/21 09:15:38 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5921worker2 # [ 145.689079] nix-grpc-daemon[1515]: time=2026-09-21T09:15:38.285Z level=INFO msg="Upload complete. (332ms)"5922lb # [ 145.740272] niks3-server[1049]: 2026/09/21 09:15:38 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5923lb # [ 145.746318] niks3-server[1049]: 2026/09/21 09:15:38 INFO Completed upload id=455924lb # [ 145.749929] niks3-server[1049]: 2026/09/21 09:15:38 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5925worker2 # [ 145.701109] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41524 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15926worker2 # [ 145.718096] nix-daemon[862]: reaped child process 1573, status = succeeded5927client # [ 145.767710] systemd[1]: sw.service: Deactivated successfully.5928client # [ 145.773190] systemd[1]: sw.service: Consumed 174ms CPU time over 13.943s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.5929client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5930client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5931worker2 # [ 146.371754] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:38Z level=info event=signal_exit5932client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5933worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5934worker2 # [ 146.503903] systemd[1]: Starting scheduler-token.service...5935worker2 # [ 146.596549] systemd[1]: scheduler-token.service: Deactivated successfully.5936worker2 # [ 146.599808] systemd[1]: Finished scheduler-token.service.5937worker2 # [ 146.601181] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 93ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5938worker2 # [ 147.414537] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:40Z level=info event=scheduler_restarting5939worker1 # [ 147.393795] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386765940worker2 # [ 147.441160] nix-grpc-daemon[1361]: ts=2026-09-21T09:15:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5941worker2 # [ 147.465773] nix-daemon[862]: reaped child process 1483, status = succeeded5942worker2 # [ 147.842378] nix-grpc-daemon[1361]: E0000 00:00:1789982140.438479 1406 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5943worker2 # [ 147.867334] nix-grpc-daemon[1361]: E0000 00:00:1789982140.463614 1406 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5944worker1 # [ 147.398653] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:39Z level=info event=worker_gone addr=192.168.1.4:500515945worker2 # [ 148.853351] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5946worker2 # [ 148.858959] systemd[1]: nix-grpc-daemon.service: Consumed 942ms CPU time over 46.696s wall clock time, 33.4M memory peak, 24.4K incoming IP traffic, 46.2K outgoing IP traffic.5947worker2 # [ 148.879904] systemd[1]: Starting Nix worker-protocol over gRPC...5948worker2 # [ 149.074814] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5949worker2 # [ 149.077272] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5950worker2 # [ 149.081310] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15951worker2 # [ 149.088059] nix-grpc-daemon[1766]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5952worker2 # [ 149.098364] nix-grpc-daemon[1766]: E0000 00:00:1789982141.684132 1766 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5953worker2 # [ 149.102973] nix-grpc-daemon[1766]: E0000 00:00:1789982141.687896 1766 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5954worker2 # [ 149.107120] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5955worker2 # [ 149.116138] systemd[1]: Started Nix worker-protocol over gRPC.5956worker2 # [ 149.117035] nix-grpc-daemon[1766]: E0000 00:00:1789982141.707603 1808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5957worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.73 seconds)5958lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5959lb: must succeed: curl -sf localhost:9901/clusters5960worker2 # [ 149.146831] nix-grpc-daemon[1766]: E0000 00:00:1789982141.727034 1808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5961worker2 # [ 149.149775] nix-grpc-daemon[1766]: E0000 00:00:1789982141.727235 1808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5962worker2 # [ 149.152260] nix-grpc-daemon[1766]: E0000 00:00:1789982141.742562 1808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5963worker2 # [ 149.165795] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:41Z level=info event=scheduler_connected addr=lb:500515964worker1 # [ 149.151152] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386765965worker1 # [ 149.157229] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05966lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5967lb # [ 149.918393] envoy[911]: conn peer=[2001:db8:1::1]:42586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9595 tx=3842 ms=64355968lb: must succeed: curl -sf localhost:9901/clusters5969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5970lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)5971lb: must succeed: curl -sf localhost:9901/clusters5972lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5973worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5974worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5975(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.71 seconds)5976subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5977client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15978lb # [ 149.923404] envoy[911]: conn peer=[2001:db8:1::1]:42570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1351 ms=136565979lb # [ 149.923535] envoy[911]: conn peer=[2001:db8:1::4]:39008 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=14470 tx=1048 ms=450045980worker2 # [ 150.726991] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546825981worker2 # [ 150.742371] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)5982worker2 # [ 150.762007] nix-grpc-daemon[1766]: ts=2026-09-21T09:15:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546825983worker2 # [ 150.777566] nix-grpc-daemon[1766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)5984worker2 # [ 150.855668] nix-grpc-daemon[1766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5)5985worker2 # [ 150.961368] nix-grpc-daemon[1766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 347 ms (attempt 3/5)5986worker2 # [ 151.315948] nix-grpc-daemon[1766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 4/5)5987worker2 # [ 151.339679] nix-grpc-daemon[1766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5988worker1 # [ 151.369943] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:43Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05989worker1 # [ 151.370314] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:43Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15990worker1 # [ 151.370455] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:43Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05991worker1 # [ 151.370570] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:43Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05992worker1 # [ 151.413003] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444425993worker1 # [ 151.427823] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444425994worker1 # [ 151.442114] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444425995worker1 # [ 151.457046] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5996worker1 # [ 151.659816] nix-daemon[840]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)5997worker1 # [ 151.679595] nix-daemon[840]: reaped child process 1715, status = succeeded5998worker1 # [ 151.852607] nix-daemon[840]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)5999worker1 # [ 151.868480] nix-daemon[840]: reaped child process 1720, status = succeeded6000lb # [ 151.955120] niks3-server[1049]: 2026/09/21 09:15:44 INFO Received uploads request method=POST path=/api/pending_closures6001worker1 # [ 151.893359] nix-grpc-daemon[1518]: time=2026-09-21T09:15:44.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6002worker1 # [ 151.895231] nix-grpc-daemon[1518]: time=2026-09-21T09:15:44.483Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6003lb # [ 151.986868] niks3-server[1049]: 2026/09/21 09:15:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6004worker1 # [ 151.921671] nix-grpc-daemon[1518]: time=2026-09-21T09:15:44.511Z level=INFO msg="Uploading 1 narinfos"6005lb # [ 151.997861] niks3-server[1049]: 2026/09/21 09:15:44 INFO Signed narinfos id=46 count=16006lb # [ 152.006570] niks3-server[1049]: 2026/09/21 09:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6007lb # [ 152.014491] niks3-server[1049]: 2026/09/21 09:15:44 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6008lb # [ 152.022793] niks3-server[1049]: 2026/09/21 09:15:44 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6009lb # [ 152.027565] niks3-server[1049]: 2026/09/21 09:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6010lb # [ 152.031220] niks3-server[1049]: 2026/09/21 09:15:44 INFO Completed upload id=466011worker1 # [ 151.959290] nix-grpc-daemon[1518]: time=2026-09-21T09:15:44.549Z level=INFO msg="Upload complete. (485ms)"6012worker1 # [ 151.966885] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=5126013worker1 # [ 151.979447] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426014worker1 # [ 151.985623] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6015worker1 # [ 151.990057] nix-daemon[840]: reaped child process 1710, status = succeeded6016worker1 # [ 152.002851] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6017worker1 # [ 153.447764] nix-daemon[840]: reaped child process 1724, status = succeeded6018worker1 # [ 153.634233] nix-daemon[840]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6019worker1 # [ 153.652267] nix-daemon[840]: reaped child process 1742, status = succeeded6020lb # [ 153.737019] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received uploads request method=POST path=/api/pending_closures6021worker1 # [ 153.679143] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6022worker1 # [ 153.680775] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.266Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6023lb # [ 153.771910] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6024lb # [ 153.787045] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6025lb # [ 153.800078] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6026worker1 # [ 153.734578] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.324Z level=INFO msg="Uploading 1 narinfos"6027lb # [ 153.809762] niks3-server[1049]: 2026/09/21 09:15:46 INFO Signed narinfos id=47 count=16028lb # [ 153.816885] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6029lb # [ 153.824881] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6030lb # [ 153.831690] niks3-server[1049]: 2026/09/21 09:15:46 INFO Completed upload id=476031worker1 # [ 153.758986] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.349Z level=INFO msg="Upload complete. (345ms)"6032lb # [ 153.834497] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6033worker1 # [ 153.777432] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16034worker1 # [ 153.796850] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16035worker1 # [ 153.801550] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426036worker1 # [ 153.806810] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06037worker1 # [ 153.811619] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426038worker1 # [ 153.816448] nix-daemon[840]: reaped child process 1722, status = succeeded6039worker1 # [ 153.819645] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426040worker1 # [ 153.830772] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6041worker1 # [ 154.013972] nix-daemon[840]: accepted connection from pid 1748, user nix-grpc-daemon (trusted)6042worker1 # [ 154.034136] nix-daemon[840]: reaped child process 1751, status = succeeded6043worker1 # [ 154.211343] nix-daemon[840]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6044worker1 # [ 154.227843] nix-daemon[840]: reaped child process 1756, status = succeeded6045lb # [ 154.313527] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received uploads request method=POST path=/api/pending_closures6046worker1 # [ 154.251192] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.841Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6047worker1 # [ 154.252778] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.841Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6048lb # [ 154.343733] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6049worker1 # [ 154.282702] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.872Z level=INFO msg="Uploading 1 narinfos"6050lb # [ 154.358052] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6051lb # [ 154.369841] niks3-server[1049]: 2026/09/21 09:15:46 INFO Signed narinfos id=48 count=16052lb # [ 154.373847] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6053lb # [ 154.382057] niks3-server[1049]: 2026/09/21 09:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6054lb # [ 154.387161] niks3-server[1049]: 2026/09/21 09:15:46 INFO Completed upload id=486055worker1 # [ 154.315171] nix-grpc-daemon[1518]: time=2026-09-21T09:15:46.905Z level=INFO msg="Upload complete. (475ms)"6056lb # [ 154.390277] niks3-server[1049]: 2026/09/21 09:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6057worker1 # [ 154.324238] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=5126058worker1 # [ 154.336682] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426059worker1 # [ 154.342465] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6060worker1 # [ 154.347440] nix-daemon[840]: reaped child process 1746, status = succeeded6061worker1 # [ 154.360978] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6062worker1 # [ 155.407781] nix-daemon[840]: reaped child process 1760, status = succeeded6063worker1 # [ 155.591329] nix-daemon[840]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6064worker1 # [ 155.607703] nix-daemon[840]: reaped child process 1778, status = succeeded6065lb # [ 155.695299] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received uploads request method=POST path=/api/pending_closures6066worker1 # [ 155.635762] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.222Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6067worker1 # [ 155.637461] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.222Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6068lb # [ 155.728182] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6069lb # [ 155.742934] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6070worker1 # [ 155.680421] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.270Z level=INFO msg="Uploading 1 narinfos"6071lb # [ 155.755064] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6072lb # [ 155.764292] niks3-server[1049]: 2026/09/21 09:15:48 INFO Signed narinfos id=49 count=16073lb # [ 155.766530] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6074lb # [ 155.771426] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6075lb # [ 155.780091] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6076lb # [ 155.785397] niks3-server[1049]: 2026/09/21 09:15:48 INFO Completed upload id=496077worker1 # [ 155.713480] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.303Z level=INFO msg="Upload complete. (342ms)"6078worker1 # [ 155.731173] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16079worker1 # [ 155.744455] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16080worker1 # [ 155.748814] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06081worker1 # [ 155.752550] nix-daemon[840]: reaped child process 1758, status = succeeded6082worker1 # [ 155.772306] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426083worker1 # [ 155.786851] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426084worker1 # [ 155.802336] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426085worker1 # [ 155.816204] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6086worker1 # [ 156.026527] nix-daemon[840]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6087worker1 # [ 156.044832] nix-daemon[840]: reaped child process 1787, status = succeeded6088worker1 # [ 156.219357] nix-daemon[840]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6089worker1 # [ 156.235525] nix-daemon[840]: reaped child process 1792, status = succeeded6090lb # [ 156.319091] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received uploads request method=POST path=/api/pending_closures6091worker1 # [ 156.255884] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.845Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6092worker1 # [ 156.257561] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.845Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6093lb # [ 156.344064] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6094worker1 # [ 156.284530] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.874Z level=INFO msg="Uploading 1 narinfos"6095lb # [ 156.360225] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6096lb # [ 156.370092] niks3-server[1049]: 2026/09/21 09:15:48 INFO Signed narinfos id=50 count=16097lb # [ 156.375799] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6098lb # [ 156.383552] niks3-server[1049]: 2026/09/21 09:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6099lb # [ 156.387846] niks3-server[1049]: 2026/09/21 09:15:48 INFO Completed upload id=506100worker1 # [ 156.315325] nix-grpc-daemon[1518]: time=2026-09-21T09:15:48.905Z level=INFO msg="Upload complete. (483ms)"6101lb # [ 156.390701] niks3-server[1049]: 2026/09/21 09:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6102worker1 # [ 156.323285] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=5126103worker1 # [ 156.334193] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426104worker1 # [ 156.340397] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6105worker1 # [ 156.344841] nix-daemon[840]: reaped child process 1782, status = succeeded6106worker1 # [ 156.356749] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6107worker1 # [ 157.799582] nix-daemon[840]: reaped child process 1796, status = succeeded6108worker1 # [ 157.975791] nix-daemon[840]: accepted connection from pid 1811, user nix-grpc-daemon (trusted)6109worker1 # [ 157.995028] nix-daemon[840]: reaped child process 1814, status = succeeded6110lb # [ 158.081311] niks3-server[1049]: 2026/09/21 09:15:50 INFO Received uploads request method=POST path=/api/pending_closures6111worker1 # [ 158.019676] nix-grpc-daemon[1518]: time=2026-09-21T09:15:50.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6112worker1 # [ 158.021443] nix-grpc-daemon[1518]: time=2026-09-21T09:15:50.608Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6113lb # [ 158.124323] niks3-server[1049]: 2026/09/21 09:15:50 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6114worker1 # [ 158.063899] nix-grpc-daemon[1518]: time=2026-09-21T09:15:50.653Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 158.135693] niks3-server[1049]: 2026/09/21 09:15:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6116lb # [ 158.144234] niks3-server[1049]: 2026/09/21 09:15:50 INFO Signed narinfos id=51 count=16117lb # [ 158.146746] niks3-server[1049]: 2026/09/21 09:15:50 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6118lb # [ 158.153953] niks3-server[1049]: 2026/09/21 09:15:50 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6119lb # [ 158.162838] niks3-server[1049]: 2026/09/21 09:15:50 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6120lb # [ 158.167810] niks3-server[1049]: 2026/09/21 09:15:50 INFO Completed upload id=516121worker1 # [ 158.094821] nix-grpc-daemon[1518]: time=2026-09-21T09:15:50.684Z level=INFO msg="Upload complete. (326ms)"6122lb # [ 158.170996] niks3-server[1049]: 2026/09/21 09:15:50 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6123worker1 # [ 158.109974] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16124worker1 # [ 158.128895] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16125worker1 # [ 158.136232] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426126worker1 # [ 158.139530] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06127worker1 # [ 158.144345] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426128worker1 # [ 158.147398] nix-daemon[840]: reaped child process 1794, status = succeeded6129worker1 # [ 158.150217] nix-grpc-daemon[1518]: time=2026-09-21T09:15:50.739Z level=INFO msg="All 1 paths already cached"6130worker1 # [ 158.154354] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426131worker1 # [ 158.164969] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6132worker1 # [ 158.345990] nix-daemon[840]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6133worker1 # [ 158.361973] nix-daemon[840]: reaped child process 1823, status = succeeded6134worker1 # [ 158.534233] nix-daemon[840]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6135worker1 # [ 158.550091] nix-daemon[840]: reaped child process 1828, status = succeeded6136lb # [ 158.633774] niks3-server[1049]: 2026/09/21 09:15:51 INFO Received uploads request method=POST path=/api/pending_closures6137worker1 # [ 158.570373] nix-grpc-daemon[1518]: time=2026-09-21T09:15:51.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6138worker1 # [ 158.571816] nix-grpc-daemon[1518]: time=2026-09-21T09:15:51.160Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6139lb # [ 158.660547] niks3-server[1049]: 2026/09/21 09:15:51 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6140lb # [ 158.674259] niks3-server[1049]: 2026/09/21 09:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6141worker1 # [ 158.609076] nix-grpc-daemon[1518]: time=2026-09-21T09:15:51.199Z level=INFO msg="Uploading 1 narinfos"6142lb # [ 158.684455] niks3-server[1049]: 2026/09/21 09:15:51 INFO Signed narinfos id=52 count=16143lb # [ 158.687093] niks3-server[1049]: 2026/09/21 09:15:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6144lb # [ 158.696278] niks3-server[1049]: 2026/09/21 09:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6145lb # [ 158.702575] niks3-server[1049]: 2026/09/21 09:15:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6146lb # [ 158.706480] niks3-server[1049]: 2026/09/21 09:15:51 INFO Completed upload id=526147worker1 # [ 158.633796] nix-grpc-daemon[1518]: time=2026-09-21T09:15:51.223Z level=INFO msg="Upload complete. (458ms)"6148worker1 # [ 158.640253] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=6246149worker1 # [ 158.648986] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426150worker1 # [ 158.660884] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6151worker1 # [ 158.663553] nix-daemon[840]: reaped child process 1818, status = succeeded6152worker1 # [ 158.676498] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6153worker1 # [ 159.697935] nix-daemon[840]: reaped child process 1832, status = succeeded6154worker1 # [ 159.857757] nix-daemon[840]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6155worker1 # [ 159.874786] nix-daemon[840]: reaped child process 1850, status = succeeded6156lb # [ 159.963732] niks3-server[1049]: 2026/09/21 09:15:52 INFO Received uploads request method=POST path=/api/pending_closures6157worker1 # [ 159.904415] nix-grpc-daemon[1518]: time=2026-09-21T09:15:52.491Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6158worker1 # [ 159.905908] nix-grpc-daemon[1518]: time=2026-09-21T09:15:52.491Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6159lb # [ 160.002619] niks3-server[1049]: 2026/09/21 09:15:52 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6160lb # [ 160.014030] niks3-server[1049]: 2026/09/21 09:15:52 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6161worker1 # [ 159.951947] nix-grpc-daemon[1518]: time=2026-09-21T09:15:52.541Z level=INFO msg="Uploading 1 narinfos"6162lb # [ 160.026782] niks3-server[1049]: 2026/09/21 09:15:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6163lb # [ 160.035477] niks3-server[1049]: 2026/09/21 09:15:52 INFO Signed narinfos id=53 count=16164lb # [ 160.038674] niks3-server[1049]: 2026/09/21 09:15:52 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6165lb # [ 160.046453] niks3-server[1049]: 2026/09/21 09:15:52 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6166lb # [ 160.050801] niks3-server[1049]: 2026/09/21 09:15:52 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6167lb # [ 160.054381] niks3-server[1049]: 2026/09/21 09:15:52 INFO Completed upload id=536168worker1 # [ 159.982204] nix-grpc-daemon[1518]: time=2026-09-21T09:15:52.572Z level=INFO msg="Upload complete. (295ms)"6169worker1 # [ 159.998894] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16170worker1 # [ 160.012509] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16171worker1 # [ 160.024121] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426172worker1 # [ 160.028664] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06173worker1 # [ 160.033695] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426174worker1 # [ 160.044300] nix-daemon[840]: reaped child process 1830, status = succeeded6175worker1 # [ 160.046111] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6176worker1 # [ 160.051278] nix-grpc-daemon[1518]: time=2026-09-21T09:15:52.613Z level=INFO msg="All 1 paths already cached"6177worker1 # [ 160.056484] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426178worker1 # [ 160.236912] nix-daemon[840]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)6179worker1 # [ 160.253943] nix-daemon[840]: reaped child process 1859, status = succeeded6180worker1 # [ 160.428732] nix-daemon[840]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6181worker1 # [ 160.445155] nix-daemon[840]: reaped child process 1864, status = succeeded6182lb # [ 160.528343] niks3-server[1049]: 2026/09/21 09:15:53 INFO Received uploads request method=POST path=/api/pending_closures6183worker1 # [ 160.464711] nix-grpc-daemon[1518]: time=2026-09-21T09:15:53.054Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6184worker1 # [ 160.466362] nix-grpc-daemon[1518]: time=2026-09-21T09:15:53.054Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6185lb # [ 160.552960] niks3-server[1049]: 2026/09/21 09:15:53 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6186lb # [ 160.568114] niks3-server[1049]: 2026/09/21 09:15:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6187worker1 # [ 160.502358] nix-grpc-daemon[1518]: time=2026-09-21T09:15:53.092Z level=INFO msg="Uploading 1 narinfos"6188lb # [ 160.580989] niks3-server[1049]: 2026/09/21 09:15:53 INFO Signed narinfos id=54 count=16189lb # [ 160.591166] niks3-server[1049]: 2026/09/21 09:15:53 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6190worker1 # [ 160.525883] nix-grpc-daemon[1518]: time=2026-09-21T09:15:53.116Z level=INFO msg="Upload complete. (466ms)"6191lb # [ 160.598365] niks3-server[1049]: 2026/09/21 09:15:53 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6192lb # [ 160.603763] niks3-server[1049]: 2026/09/21 09:15:53 INFO Completed upload id=546193worker1 # [ 160.533747] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=6246194lb # [ 160.607067] niks3-server[1049]: 2026/09/21 09:15:53 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6195worker1 # [ 160.545880] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426196worker1 # [ 160.551380] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6197worker1 # [ 160.555757] nix-daemon[840]: reaped child process 1854, status = succeeded6198worker1 # [ 160.567390] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6199worker1 # [ 161.607791] nix-daemon[840]: reaped child process 1868, status = succeeded6200worker1 # [ 161.794948] nix-daemon[840]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6201worker1 # [ 161.814862] nix-daemon[840]: reaped child process 1886, status = succeeded6202lb # [ 161.901186] niks3-server[1049]: 2026/09/21 09:15:54 INFO Received uploads request method=POST path=/api/pending_closures6203worker1 # [ 161.839474] nix-grpc-daemon[1518]: time=2026-09-21T09:15:54.429Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6204worker1 # [ 161.842436] nix-grpc-daemon[1518]: time=2026-09-21T09:15:54.431Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6205lb # [ 161.939223] niks3-server[1049]: 2026/09/21 09:15:54 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6206lb # [ 161.960150] niks3-server[1049]: 2026/09/21 09:15:54 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6207lb # [ 161.970197] niks3-server[1049]: 2026/09/21 09:15:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6208worker1 # [ 161.904347] nix-grpc-daemon[1518]: time=2026-09-21T09:15:54.493Z level=INFO msg="Uploading 1 narinfos"6209lb # [ 161.980982] niks3-server[1049]: 2026/09/21 09:15:54 INFO Signed narinfos id=55 count=16210lb # [ 161.986623] niks3-server[1049]: 2026/09/21 09:15:54 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6211lb # [ 161.996241] niks3-server[1049]: 2026/09/21 09:15:54 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6212worker1 # [ 161.930173] nix-grpc-daemon[1518]: time=2026-09-21T09:15:54.520Z level=INFO msg="Upload complete. (345ms)"6213lb # [ 162.004469] niks3-server[1049]: 2026/09/21 09:15:54 INFO Completed upload id=556214lb # [ 162.008808] niks3-server[1049]: 2026/09/21 09:15:54 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6215worker1 # [ 161.950741] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16216worker1 # [ 161.966922] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16217worker1 # [ 161.976336] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426218worker1 # [ 161.979450] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06219worker1 # [ 161.985677] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426220worker1 # [ 161.989382] nix-grpc-daemon[1518]: time=2026-09-21T09:15:54.578Z level=INFO msg="All 1 paths already cached"6221worker1 # [ 161.991512] nix-daemon[840]: reaped child process 1866, status = succeeded6222worker1 # [ 161.994072] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426223worker1 # [ 162.003274] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6224worker1 # [ 162.194228] nix-daemon[840]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6225worker1 # [ 162.213550] nix-daemon[840]: reaped child process 1895, status = succeeded6226worker1 # [ 162.395371] nix-daemon[840]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6227worker1 # [ 162.413616] nix-daemon[840]: reaped child process 1900, status = succeeded6228lb # [ 162.497961] niks3-server[1049]: 2026/09/21 09:15:55 INFO Received uploads request method=POST path=/api/pending_closures6229worker1 # [ 162.434530] nix-grpc-daemon[1518]: time=2026-09-21T09:15:55.024Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6230worker1 # [ 162.436085] nix-grpc-daemon[1518]: time=2026-09-21T09:15:55.024Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6231lb # [ 162.525229] niks3-server[1049]: 2026/09/21 09:15:55 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6232worker1 # [ 162.465555] nix-grpc-daemon[1518]: time=2026-09-21T09:15:55.055Z level=INFO msg="Uploading 1 narinfos"6233lb # [ 162.540973] niks3-server[1049]: 2026/09/21 09:15:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6234lb # [ 162.551965] niks3-server[1049]: 2026/09/21 09:15:55 INFO Signed narinfos id=56 count=16235lb # [ 162.557133] niks3-server[1049]: 2026/09/21 09:15:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6236worker1 # [ 162.492315] nix-grpc-daemon[1518]: time=2026-09-21T09:15:55.082Z level=INFO msg="Upload complete. (479ms)"6237lb # [ 162.565800] niks3-server[1049]: 2026/09/21 09:15:55 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6238lb # [ 162.571124] niks3-server[1049]: 2026/09/21 09:15:55 INFO Completed upload id=566239worker1 # [ 162.498791] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=6246240lb # [ 162.574439] niks3-server[1049]: 2026/09/21 09:15:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6241worker1 # [ 162.511213] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426242worker1 # [ 162.517703] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6243worker1 # [ 162.522462] nix-daemon[840]: reaped child process 1890, status = succeeded6244worker1 # [ 162.534720] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6245worker1 # [ 163.980322] nix-daemon[840]: reaped child process 1904, status = succeeded6246worker1 # [ 164.161259] nix-daemon[840]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6247worker1 # [ 164.177908] nix-daemon[840]: reaped child process 1922, status = succeeded6248lb # [ 164.262179] niks3-server[1049]: 2026/09/21 09:15:56 INFO Received uploads request method=POST path=/api/pending_closures6249worker1 # [ 164.200386] nix-grpc-daemon[1518]: time=2026-09-21T09:15:56.789Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6250worker1 # [ 164.201912] nix-grpc-daemon[1518]: time=2026-09-21T09:15:56.789Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6251lb # [ 164.302768] niks3-server[1049]: 2026/09/21 09:15:56 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6252lb # [ 164.314681] niks3-server[1049]: 2026/09/21 09:15:56 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6253lb # [ 164.324344] niks3-server[1049]: 2026/09/21 09:15:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6254worker1 # [ 164.259655] nix-grpc-daemon[1518]: time=2026-09-21T09:15:56.849Z level=INFO msg="Uploading 1 narinfos"6255lb # [ 164.333081] niks3-server[1049]: 2026/09/21 09:15:56 INFO Signed narinfos id=57 count=16256lb # [ 164.339773] niks3-server[1049]: 2026/09/21 09:15:56 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6257lb # [ 164.348669] niks3-server[1049]: 2026/09/21 09:15:56 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6258lb # [ 164.356590] niks3-server[1049]: 2026/09/21 09:15:56 INFO Completed upload id=576259worker1 # [ 164.283991] nix-grpc-daemon[1518]: time=2026-09-21T09:15:56.874Z level=INFO msg="Upload complete. (339ms)"6260lb # [ 164.359818] niks3-server[1049]: 2026/09/21 09:15:56 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6261worker1 # [ 164.297159] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16262worker1 # [ 164.312373] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16263worker1 # [ 164.318700] nix-daemon[840]: reaped child process 1902, status = succeeded6264worker1 # [ 164.338717] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426265worker1 # [ 164.347109] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06266worker1 # [ 164.358339] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426267worker1 # [ 164.371189] nix-grpc-daemon[1518]: time=2026-09-21T09:15:56.961Z level=INFO msg="All 2 paths already cached"6268worker1 # [ 164.380517] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426269worker1 # [ 164.393608] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6270worker1 # [ 164.591372] nix-daemon[840]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6271worker1 # [ 164.610767] nix-daemon[840]: reaped child process 1931, status = succeeded6272worker1 # [ 164.784313] nix-daemon[840]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)6273worker1 # [ 164.800698] nix-daemon[840]: reaped child process 1936, status = succeeded6274lb # [ 164.884433] niks3-server[1049]: 2026/09/21 09:15:57 INFO Received uploads request method=POST path=/api/pending_closures6275worker1 # [ 164.820690] nix-grpc-daemon[1518]: time=2026-09-21T09:15:57.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6276worker1 # [ 164.822295] nix-grpc-daemon[1518]: time=2026-09-21T09:15:57.410Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6277lb # [ 164.909285] niks3-server[1049]: 2026/09/21 09:15:57 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6278worker1 # [ 164.849607] nix-grpc-daemon[1518]: time=2026-09-21T09:15:57.439Z level=INFO msg="Uploading 1 narinfos"6279lb # [ 164.925774] niks3-server[1049]: 2026/09/21 09:15:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6280lb # [ 164.937333] niks3-server[1049]: 2026/09/21 09:15:57 INFO Signed narinfos id=58 count=16281lb # [ 164.942451] niks3-server[1049]: 2026/09/21 09:15:57 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6282lb # [ 164.950557] niks3-server[1049]: 2026/09/21 09:15:57 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6283lb # [ 164.955884] niks3-server[1049]: 2026/09/21 09:15:57 INFO Completed upload id=586284worker1 # [ 164.883850] nix-grpc-daemon[1518]: time=2026-09-21T09:15:57.474Z level=INFO msg="Upload complete. (475ms)"6285lb # [ 164.959256] niks3-server[1049]: 2026/09/21 09:15:57 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6286worker1 # [ 164.892262] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=7526287worker1 # [ 164.902436] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426288worker1 # [ 164.909180] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6289worker1 # [ 164.913937] nix-daemon[840]: reaped child process 1926, status = succeeded6290worker1 # [ 164.924655] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6291worker1 # [ 165.954745] nix-daemon[840]: reaped child process 1940, status = succeeded6292worker1 # [ 166.115163] nix-daemon[840]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6293worker1 # [ 166.131761] nix-daemon[840]: reaped child process 1958, status = succeeded6294lb # [ 166.216606] niks3-server[1049]: 2026/09/21 09:15:58 INFO Received uploads request method=POST path=/api/pending_closures6295worker1 # [ 166.155929] nix-grpc-daemon[1518]: time=2026-09-21T09:15:58.744Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6296worker1 # [ 166.157651] nix-grpc-daemon[1518]: time=2026-09-21T09:15:58.744Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6297lb # [ 166.253902] niks3-server[1049]: 2026/09/21 09:15:58 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6298lb # [ 166.265658] niks3-server[1049]: 2026/09/21 09:15:58 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6299worker1 # [ 166.203042] nix-grpc-daemon[1518]: time=2026-09-21T09:15:58.792Z level=INFO msg="Uploading 1 narinfos"6300lb # [ 166.279253] niks3-server[1049]: 2026/09/21 09:15:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6301lb # [ 166.287014] niks3-server[1049]: 2026/09/21 09:15:58 INFO Signed narinfos id=59 count=16302lb # [ 166.290242] niks3-server[1049]: 2026/09/21 09:15:58 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6303lb # [ 166.297595] niks3-server[1049]: 2026/09/21 09:15:58 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6304lb # [ 166.301897] niks3-server[1049]: 2026/09/21 09:15:58 INFO Completed upload id=596305worker1 # [ 166.229189] nix-grpc-daemon[1518]: time=2026-09-21T09:15:58.819Z level=INFO msg="Upload complete. (293ms)"6306lb # [ 166.304303] niks3-server[1049]: 2026/09/21 09:15:58 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6307worker1 # [ 166.241079] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16308worker1 # [ 166.259587] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:58Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16309worker1 # [ 166.265093] nix-daemon[840]: reaped child process 1938, status = succeeded6310worker1 # [ 166.284112] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426311worker1 # [ 166.298031] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426312worker1 # [ 166.312177] nix-grpc-daemon[1518]: time=2026-09-21T09:15:58.902Z level=INFO msg="All 2 paths already cached"6313worker1 # [ 166.321146] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426314worker1 # [ 166.337746] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6315worker1 # [ 166.538900] nix-daemon[840]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6316worker1 # [ 166.556987] nix-daemon[840]: reaped child process 1965, status = succeeded6317worker1 # [ 166.728785] nix-daemon[840]: accepted connection from pid 1967, user nix-grpc-daemon (trusted)6318worker1 # [ 166.744430] nix-daemon[840]: reaped child process 1970, status = succeeded6319lb # [ 166.828974] niks3-server[1049]: 2026/09/21 09:15:59 INFO Received uploads request method=POST path=/api/pending_closures6320worker1 # [ 166.765443] nix-grpc-daemon[1518]: time=2026-09-21T09:15:59.355Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6321worker1 # [ 166.766896] nix-grpc-daemon[1518]: time=2026-09-21T09:15:59.355Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6322lb # [ 166.853620] niks3-server[1049]: 2026/09/21 09:15:59 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6323lb # [ 166.867195] niks3-server[1049]: 2026/09/21 09:15:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6324worker1 # [ 166.803219] nix-grpc-daemon[1518]: time=2026-09-21T09:15:59.393Z level=INFO msg="Uploading 1 narinfos"6325lb # [ 166.875409] niks3-server[1049]: 2026/09/21 09:15:59 INFO Signed narinfos id=60 count=16326lb # [ 166.884525] niks3-server[1049]: 2026/09/21 09:15:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6327lb # [ 166.896684] niks3-server[1049]: 2026/09/21 09:15:59 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6328worker1 # [ 166.829866] nix-grpc-daemon[1518]: time=2026-09-21T09:15:59.420Z level=INFO msg="Upload complete. (475ms)"6329lb # [ 166.901927] niks3-server[1049]: 2026/09/21 09:15:59 INFO Completed upload id=606330lb # [ 166.905084] niks3-server[1049]: 2026/09/21 09:15:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6331worker1 # [ 166.836249] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=7526332worker1 # [ 166.844663] nix-grpc-daemon[1443]: ts=2026-09-21T09:15:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426333worker1 # [ 166.856156] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6334worker1 # [ 166.859933] nix-daemon[840]: reaped child process 1960, status = succeeded6335worker1 # [ 166.874574] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6336worker1 # [ 168.318555] nix-daemon[840]: reaped child process 1974, status = succeeded6337worker1 # [ 168.494704] nix-daemon[840]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6338worker1 # [ 168.512851] nix-daemon[840]: reaped child process 1992, status = succeeded6339lb # [ 168.601266] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received uploads request method=POST path=/api/pending_closures6340worker1 # [ 168.539984] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.129Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6341worker1 # [ 168.545352] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.132Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6342lb # [ 168.637063] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6343lb # [ 168.651376] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6344worker1 # [ 168.596679] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.186Z level=INFO msg="Uploading 1 narinfos"6345lb # [ 168.663254] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6346lb # [ 168.673840] niks3-server[1049]: 2026/09/21 09:16:01 INFO Signed narinfos id=61 count=16347lb # [ 168.680109] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6348lb # [ 168.688797] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6349lb # [ 168.693954] niks3-server[1049]: 2026/09/21 09:16:01 INFO Completed upload id=616350worker1 # [ 168.621452] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.211Z level=INFO msg="Upload complete. (336ms)"6351lb # [ 168.697151] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6352worker1 # [ 168.637310] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16353worker1 # [ 168.653493] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16354worker1 # [ 168.658364] nix-daemon[840]: reaped child process 1972, status = succeeded6355worker1 # [ 168.676596] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426356worker1 # [ 168.691628] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444426357worker1 # [ 168.706871] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.296Z level=INFO msg="All 2 paths already cached"6358worker1 # [ 168.715982] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444426359worker1 # [ 168.729853] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6360worker1 # [ 168.928136] nix-daemon[840]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6361worker1 # [ 168.948117] nix-daemon[840]: reaped child process 1999, status = succeeded6362worker1 # [ 169.119106] nix-daemon[840]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)6363worker1 # [ 169.137569] nix-daemon[840]: reaped child process 2004, status = succeeded6364lb # [ 169.221177] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received uploads request method=POST path=/api/pending_closures6365worker1 # [ 169.158281] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.748Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6366worker1 # [ 169.159734] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.748Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6367lb # [ 169.247202] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6368lb # [ 169.259564] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6369worker1 # [ 169.193812] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.783Z level=INFO msg="Uploading 1 narinfos"6370lb # [ 169.269969] niks3-server[1049]: 2026/09/21 09:16:01 INFO Signed narinfos id=62 count=16371lb # [ 169.276346] niks3-server[1049]: 2026/09/21 09:16:01 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6372lb # [ 169.284254] niks3-server[1049]: 2026/09/21 09:16:01 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6373lb # [ 169.288769] niks3-server[1049]: 2026/09/21 09:16:01 INFO Completed upload id=626374worker1 # [ 169.216268] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.806Z level=INFO msg="Upload complete. (469ms)"6375worker1 # [ 169.223298] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=0 paths=1 nar_bytes_in=7526376worker1 # [ 169.234321] nix-grpc-daemon[1518]: time=2026-09-21T09:16:01.809Z level=WARN msg="Failed to register uploaded object" key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo error="server returned 404: object is not pending upload\n"6377worker1 # [ 169.239731] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444426378worker1 # [ 169.245224] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6379worker1 # [ 169.247997] nix-daemon[840]: reaped child process 1994, status = succeeded6380worker1 # [ 169.256816] nix-daemon[840]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6381worker1 # [ 170.505908] nix-daemon[840]: reaped child process 2008, status = succeeded6382worker1 # [ 170.678924] nix-daemon[840]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6383worker1 # [ 170.698748] nix-daemon[840]: reaped child process 2026, status = succeeded6384lb # [ 170.786108] niks3-server[1049]: 2026/09/21 09:16:03 INFO Received uploads request method=POST path=/api/pending_closures6385worker1 # [ 170.723079] nix-grpc-daemon[1518]: time=2026-09-21T09:16:03.312Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6386worker1 # [ 170.724686] nix-grpc-daemon[1518]: time=2026-09-21T09:16:03.312Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6387lb # [ 170.818179] niks3-server[1049]: 2026/09/21 09:16:03 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6388lb # [ 170.833076] niks3-server[1049]: 2026/09/21 09:16:03 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6389worker1 # [ 170.777341] nix-grpc-daemon[1518]: time=2026-09-21T09:16:03.367Z level=INFO msg="Uploading 1 narinfos"6390lb # [ 170.849046] niks3-server[1049]: 2026/09/21 09:16:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6391lb # [ 170.860461] niks3-server[1049]: 2026/09/21 09:16:03 INFO Signed narinfos id=63 count=16392lb # [ 170.867330] niks3-server[1049]: 2026/09/21 09:16:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6393lb # [ 170.875835] niks3-server[1049]: 2026/09/21 09:16:03 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6394lb # [ 170.882581] niks3-server[1049]: 2026/09/21 09:16:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6395lb # [ 170.888143] niks3-server[1049]: 2026/09/21 09:16:03 INFO Completed upload id=636396worker1 # [ 170.816419] nix-grpc-daemon[1518]: time=2026-09-21T09:16:03.406Z level=INFO msg="Upload complete. (348ms)"6397worker1 # [ 170.829297] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44442 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16398worker1 # [ 170.846639] nix-daemon[840]: reaped child process 2006, status = succeeded6399client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.47 seconds)6400worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6401worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6402client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6403client # [ 171.093894] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6404client # Running as unit: gpu.service; invocation ID: fc3fc5bd52624eb290a42ed56dc564c66405client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6406lb: must succeed: curl -sf localhost:9901/clusters6407lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6408lb: must succeed: curl -sf localhost:9901/clusters6409lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6410worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6411worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6412client # [ 171.351945] nix[1338]: don't know how to build these paths:6413client # [ 171.353109] nix[1338]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6414worker1 # [ 171.307838] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444426415worker2 # [ 171.337465] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546826416worker1 # [ 171.340261] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:03Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06417client # [ 171.389705] nix[1338]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6418lb: must succeed: curl -sf localhost:9901/clusters6419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6420lb: must succeed: curl -sf localhost:9901/clusters6421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6422worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6423worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6424client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6425lb # [ 179.919400] envoy[911]: conn peer=[2001:db8:1::1]:40460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27118 tx=10668 ms=201446426client # [ 201.395918] nix[1338]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6427client # [ 201.408431] nix[1338]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6428client # [ 201.448114] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6429client # [ 201.455730] systemd[1]: gpu.service: Failed with result 'exit-code'.6430client # [ 201.459360] systemd[1]: gpu.service: Consumed 178ms CPU time over 30.359s wall clock time, 35.5M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6431client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.24 seconds)6432client: must succeed: journalctl -u gpu -o cat6433client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6434lb: must succeed: curl -sf localhost:9901/clusters6435lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6436lb: must succeed: curl -sf localhost:9901/clusters6437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6438worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6439worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6440(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.58 seconds)6441subtest: low disk drains a worker and builds go to the other6442worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6443worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.15 seconds)6444worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6445worker1 # [ 203.539934] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:36Z level=info event=unhealthy reason=min_free6446worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6447client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26448client # don't know how to build these paths:6449client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6450worker1 # [ 204.549907] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444426451worker2 # [ 204.577981] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546826452worker1 # [ 204.580999] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:37Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06453worker1 # [ 204.586745] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:37Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16454worker2 # [ 204.650058] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826455worker2 # [ 204.665542] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546826456client # copying 1 paths to 'lb:50051'...6457client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6458worker2 # [ 204.681018] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546826459worker2 # [ 204.698286] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6460worker2 # [ 204.727906] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:37Z level=info event=niks3_push_started pid=18256461worker2 # [ 204.939384] nix-grpc-daemon[1825]: time=2026-09-21T09:16:37.535Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6462worker2 # [ 205.112317] nix-daemon[862]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6463worker2 # [ 205.130070] nix-daemon[862]: reaped child process 1841, status = succeeded6464worker2 # [ 205.302121] nix-daemon[862]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6465worker2 # [ 205.317889] nix-daemon[862]: reaped child process 1846, status = succeeded6466lb # [ 205.380780] niks3-server[1049]: 2026/09/21 09:16:37 INFO Received uploads request method=POST path=/api/pending_closures6467worker2 # [ 205.340134] nix-grpc-daemon[1825]: time=2026-09-21T09:16:37.936Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker2 # [ 205.343214] nix-grpc-daemon[1825]: time=2026-09-21T09:16:37.938Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6469lb # [ 205.417822] niks3-server[1049]: 2026/09/21 09:16:38 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6470worker2 # [ 205.378161] nix-grpc-daemon[1825]: time=2026-09-21T09:16:37.974Z level=INFO msg="Uploading 1 narinfos"6471lb # [ 205.432669] niks3-server[1049]: 2026/09/21 09:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6472lb # [ 205.443512] niks3-server[1049]: 2026/09/21 09:16:38 INFO Signed narinfos id=64 count=16473lb # [ 205.449747] niks3-server[1049]: 2026/09/21 09:16:38 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6474lb # [ 205.456100] niks3-server[1049]: 2026/09/21 09:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6475lb # [ 205.464542] niks3-server[1049]: 2026/09/21 09:16:38 INFO Completed upload id=646476worker2 # [ 205.413641] nix-grpc-daemon[1825]: time=2026-09-21T09:16:38.010Z level=INFO msg="Upload complete. (470ms)"6477lb # [ 205.467489] niks3-server[1049]: 2026/09/21 09:16:38 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6478worker2 # [ 205.420297] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=0 paths=1 nar_bytes_in=5126479worker2 # [ 205.435122] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826480worker2 # [ 205.441501] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6481worker2 # [ 205.446767] nix-daemon[862]: reaped child process 1823, status = succeeded6482client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6483worker2 # [ 205.457625] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6484client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6485client # farm-a> LOG-farm-a-drain6486worker2 # [ 207.083373] nix-daemon[862]: reaped child process 1850, status = succeeded6487worker2 # [ 207.238246] nix-daemon[862]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6488worker2 # [ 207.256309] nix-daemon[862]: reaped child process 1868, status = succeeded6489lb # [ 207.320781] niks3-server[1049]: 2026/09/21 09:16:39 INFO Received uploads request method=POST path=/api/pending_closures6490worker2 # [ 207.280416] nix-grpc-daemon[1825]: time=2026-09-21T09:16:39.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6491worker2 # [ 207.281943] nix-grpc-daemon[1825]: time=2026-09-21T09:16:39.875Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6492lb # [ 207.355764] niks3-server[1049]: 2026/09/21 09:16:39 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6493lb # [ 207.367893] niks3-server[1049]: 2026/09/21 09:16:39 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6494worker2 # [ 207.334756] nix-grpc-daemon[1825]: time=2026-09-21T09:16:39.930Z level=INFO msg="Uploading 1 narinfos"6495lb # [ 207.384534] niks3-server[1049]: 2026/09/21 09:16:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6496lb # [ 207.396104] niks3-server[1049]: 2026/09/21 09:16:39 INFO Signed narinfos id=65 count=16497lb # [ 207.402364] niks3-server[1049]: 2026/09/21 09:16:39 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6498worker2 # [ 207.362293] nix-grpc-daemon[1825]: time=2026-09-21T09:16:39.958Z level=INFO msg="Upload complete. (292ms)"6499lb # [ 207.414591] niks3-server[1049]: 2026/09/21 09:16:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6500lb # [ 207.419723] niks3-server[1049]: 2026/09/21 09:16:39 INFO Completed upload id=656501lb # [ 207.422297] niks3-server[1049]: 2026/09/21 09:16:40 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6502worker2 # [ 207.372944] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16503worker1 # [ 207.357339] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:39Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06504worker1 # [ 207.363123] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:39Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16505worker2 # [ 207.388679] nix-daemon[862]: reaped child process 1848, status = succeeded6506worker2 # [ 207.429264] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826507worker2 # [ 207.445554] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546826508worker2 # [ 207.460068] nix-grpc-daemon[1825]: time=2026-09-21T09:16:40.056Z level=INFO msg="All 1 paths already cached"6509client # copying 1 paths to 'lb:50051'...6510client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6511worker2 # [ 207.470383] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546826512worker2 # [ 207.484156] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6513worker2 # [ 207.684109] nix-daemon[862]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6514worker2 # [ 207.700809] nix-daemon[862]: reaped child process 1875, status = succeeded6515worker2 # [ 207.876107] nix-daemon[862]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6516worker2 # [ 207.891196] nix-daemon[862]: reaped child process 1880, status = succeeded6517lb # [ 207.956644] niks3-server[1049]: 2026/09/21 09:16:40 INFO Received uploads request method=POST path=/api/pending_closures6518worker2 # [ 207.914738] nix-grpc-daemon[1825]: time=2026-09-21T09:16:40.510Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6519worker2 # [ 207.916266] nix-grpc-daemon[1825]: time=2026-09-21T09:16:40.510Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6520lb # [ 207.988822] niks3-server[1049]: 2026/09/21 09:16:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6521worker2 # [ 207.944735] nix-grpc-daemon[1825]: time=2026-09-21T09:16:40.540Z level=INFO msg="Uploading 1 narinfos"6522lb # [ 208.003303] niks3-server[1049]: 2026/09/21 09:16:40 INFO Signed narinfos id=66 count=16523lb # [ 208.013737] niks3-server[1049]: 2026/09/21 09:16:40 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6524worker2 # [ 207.977169] nix-grpc-daemon[1825]: time=2026-09-21T09:16:40.573Z level=INFO msg="Upload complete. (475ms)"6525lb # [ 208.028977] niks3-server[1049]: 2026/09/21 09:16:40 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6526worker2 # [ 207.983865] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=0 paths=1 nar_bytes_in=6326527lb # [ 208.035031] niks3-server[1049]: 2026/09/21 09:16:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6528lb # [ 208.040813] niks3-server[1049]: 2026/09/21 09:16:40 INFO Completed upload id=666529lb # [ 208.043524] niks3-server[1049]: 2026/09/21 09:16:40 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6530worker2 # [ 207.995726] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826531worker2 # [ 208.002169] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6532worker2 # [ 208.006656] nix-daemon[862]: reaped child process 1870, status = succeeded6533client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6534worker2 # [ 208.018410] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6535client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6536worker2 # [ 209.038794] nix-daemon[862]: reaped child process 1884, status = succeeded6537client # farm-b> LOG-farm-b-drain6538worker2 # [ 209.193697] nix-daemon[862]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6539worker2 # [ 209.210790] nix-daemon[862]: reaped child process 1902, status = succeeded6540lb # [ 209.279522] niks3-server[1049]: 2026/09/21 09:16:41 INFO Received uploads request method=POST path=/api/pending_closures6541worker2 # [ 209.237199] nix-grpc-daemon[1825]: time=2026-09-21T09:16:41.832Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6542worker2 # [ 209.238670] nix-grpc-daemon[1825]: time=2026-09-21T09:16:41.832Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6543lb # [ 209.310957] niks3-server[1049]: 2026/09/21 09:16:41 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6544lb # [ 209.328153] niks3-server[1049]: 2026/09/21 09:16:41 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6545lb # [ 209.334917] niks3-server[1049]: 2026/09/21 09:16:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6546worker2 # [ 209.289957] nix-grpc-daemon[1825]: time=2026-09-21T09:16:41.886Z level=INFO msg="Uploading 1 narinfos"6547lb # [ 209.345069] niks3-server[1049]: 2026/09/21 09:16:41 INFO Signed narinfos id=67 count=16548lb # [ 209.347312] niks3-server[1049]: 2026/09/21 09:16:41 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6549lb # [ 209.355267] niks3-server[1049]: 2026/09/21 09:16:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6550lb # [ 209.361137] niks3-server[1049]: 2026/09/21 09:16:41 INFO Completed upload id=676551worker2 # [ 209.309661] nix-grpc-daemon[1825]: time=2026-09-21T09:16:41.906Z level=INFO msg="Upload complete. (282ms)"6552lb # [ 209.364350] niks3-server[1049]: 2026/09/21 09:16:41 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6553worker2 # [ 209.320904] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16554worker1 # [ 209.304625] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:41Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06555worker1 # [ 209.305016] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:41Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16556worker2 # [ 209.335774] nix-daemon[862]: reaped child process 1882, status = succeeded6557worker2 # [ 209.369852] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826558worker2 # [ 209.384875] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546826559worker2 # [ 209.398920] nix-grpc-daemon[1825]: time=2026-09-21T09:16:41.995Z level=INFO msg="All 2 paths already cached"6560client # copying 1 paths to 'lb:50051'...6561client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6562worker2 # [ 209.409056] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546826563worker2 # [ 209.422940] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6564worker2 # [ 209.626039] nix-daemon[862]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6565worker2 # [ 209.643075] nix-daemon[862]: reaped child process 1909, status = succeeded6566worker2 # [ 209.813563] nix-daemon[862]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6567worker2 # [ 209.829893] nix-daemon[862]: reaped child process 1914, status = succeeded6568lb # [ 209.892809] niks3-server[1049]: 2026/09/21 09:16:42 INFO Received uploads request method=POST path=/api/pending_closures6569worker2 # [ 209.850577] nix-grpc-daemon[1825]: time=2026-09-21T09:16:42.446Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6570worker2 # [ 209.852121] nix-grpc-daemon[1825]: time=2026-09-21T09:16:42.446Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6571lb # [ 209.920261] niks3-server[1049]: 2026/09/21 09:16:42 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6572worker2 # [ 209.880319] nix-grpc-daemon[1825]: time=2026-09-21T09:16:42.476Z level=INFO msg="Uploading 1 narinfos"6573lb # [ 209.936328] niks3-server[1049]: 2026/09/21 09:16:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6574lb # [ 209.946073] niks3-server[1049]: 2026/09/21 09:16:42 INFO Signed narinfos id=68 count=16575lb # [ 209.954751] niks3-server[1049]: 2026/09/21 09:16:42 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6576worker2 # [ 209.908072] nix-grpc-daemon[1825]: time=2026-09-21T09:16:42.504Z level=INFO msg="Upload complete. (468ms)"6577lb # [ 209.960553] envoy[911]: conn peer=[2001:db8:1::1]:33836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300666578lb # [ 209.965036] niks3-server[1049]: 2026/09/21 09:16:42 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6579worker2 # [ 209.916134] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=0 paths=1 nar_bytes_in=7606580lb # [ 209.969234] niks3-server[1049]: 2026/09/21 09:16:42 INFO Completed upload id=686581lb # [ 209.971489] niks3-server[1049]: 2026/09/21 09:16:42 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6582worker2 # [ 209.925877] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546826583worker2 # [ 209.934747] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6584worker2 # [ 209.937006] nix-daemon[862]: reaped child process 1904, status = succeeded6585client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6586worker2 # [ 209.949685] nix-daemon[862]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6587client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6588client # farm-top> LOG-farm-top-drain6589worker2 # [ 211.385204] nix-daemon[862]: reaped child process 1918, status = succeeded6590worker2 # [ 211.562374] nix-daemon[862]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)6591worker2 # [ 211.581427] nix-daemon[862]: reaped child process 1936, status = succeeded6592lb # [ 211.646795] niks3-server[1049]: 2026/09/21 09:16:44 INFO Received uploads request method=POST path=/api/pending_closures6593worker2 # [ 211.606380] nix-grpc-daemon[1825]: time=2026-09-21T09:16:44.202Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6594worker2 # [ 211.609724] nix-grpc-daemon[1825]: time=2026-09-21T09:16:44.204Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6595lb # [ 211.685830] niks3-server[1049]: 2026/09/21 09:16:44 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6596lb # [ 211.698753] niks3-server[1049]: 2026/09/21 09:16:44 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6597lb # [ 211.709497] niks3-server[1049]: 2026/09/21 09:16:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6598worker2 # [ 211.665935] nix-grpc-daemon[1825]: time=2026-09-21T09:16:44.261Z level=INFO msg="Uploading 1 narinfos"6599lb # [ 211.719730] niks3-server[1049]: 2026/09/21 09:16:44 INFO Signed narinfos id=69 count=16600lb # [ 211.726915] niks3-server[1049]: 2026/09/21 09:16:44 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6601lb # [ 211.735621] niks3-server[1049]: 2026/09/21 09:16:44 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6602lb # [ 211.743303] niks3-server[1049]: 2026/09/21 09:16:44 INFO Completed upload id=696603worker2 # [ 211.692075] nix-grpc-daemon[1825]: time=2026-09-21T09:16:44.288Z level=INFO msg="Upload complete. (332ms)"6604lb # [ 211.746454] niks3-server[1049]: 2026/09/21 09:16:44 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6605worker2 # [ 211.705026] nix-grpc-daemon[1766]: ts=2026-09-21T09:16:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54682 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16606worker2 # [ 211.721010] nix-daemon[862]: reaped child process 1916, status = succeeded6607client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.45 seconds)6608client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6609client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6610worker1: must succeed: rm /nix/.rw-store/fill6611worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6612worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6613worker1 # [ 212.545654] nix-grpc-daemon[1443]: ts=2026-09-21T09:16:45Z level=info event=healthy reason=min_free6614worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6615(finished: subtest: low disk drains a worker and builds go to the other, in 11.33 seconds)6616(finished: run the VM test script, in 214.26 seconds)6617test script finished in 214.54s6618cleanup6619kill QemuMachine (pid 50)6620client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6621kill QemuMachine (pid 49)6622lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6623kill QemuMachine (pid 52)6624worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6625kill QemuMachine (pid 51)6626worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6627(finished: cleanup, in 1.04 seconds)6628additionally exposed symbols:6629 client, lb, worker1, worker2,6630 vlan1,6631 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh